There is a defeatist element to it, but I think the statement also shows a (possibly unhealthy) degree of respect to Contador. It can easily be seen as Frank saying "Look, when Contador did this much racing, he was a shadow of his normal self. If he can't do it, what hope have I?". Yes, Contador had a tougher Giro-Tour double with a full, more brutal Giro and crashes in the Tour, but the point remains that these days, very few riders are going to have a proper crack at both the Giro and Tour.
All you Contador fans need to get a grip. Everyone knows that sportspeople are rarely 100% truthful in the media. This could all be a bluff, stop throwing a hissy fit every time anyone compares themselves to Contador.