The best way for Andy to lose a Tour wihtout Contador would need a similar situation as 03 with Armstrong.  2 riders close in time, that are completely different characteristics.  IN 03 you had Ulrich who was the diesel that sat on Armstrong, forcing Lance to chase the other rider, Vinokourov, a naturally aggressive rider. 
 The aggressive rider is close enough in the GC forcing the leader to chase, while the Top TT can sit on the favorite and maybe pipe him in the end for a couple of seconds here and there.  The aggressive rider doesn't havent to be a Top TT, but at least one of good quality to where on a normal day he may not lose much more then a minute in the TT.
Unfortunately, I am not sure we can produce that duo this year.  I think Cadel could come close to playing the role of Ulrich, he seems to be climbing well and looks more like his old TT form,  but the closest I see to the aggressive rider, would be Vino, and as much as I love to see it, I just dont think he will contend as long as Schleck is in good form and bad luck free.
I do have a feeling though that if Contador is not there, I think we will have a surprising podium result, whether if its Andy not winning, or an outsider Like a Vino, maybe a Vande Velde type, one who has finished Top 5-10, but nothing impressive to make one think they could do more, or even Tony Martin, a really good TT that can limit losses in the mountains.    The way the course is, I think he can lend itself to a surprise.