No. Those who compete frequently against the best. Not just at TT or one or other mountain stage, but the most of them. Contador, Quintana, Froome, Nibali (GC), Cavendish, Cancellara, Tony Martin, Sagan, for example, are first class riders. Then you have 2nd class riders, such as Uran, Van den Broeck, Wiggins, etc.
I mean, just because a guy is 2nd in a GT, does that mean he should be considered a GC contender for every GT from then on? No. He must have great performances frequently, mainly in GTs, to be considered a serious GC contender. At least that's my opinion.