He was so much of an outlier (for a number of reasons, one of course being massive EPO use) nobody has come close to most of his times since. The last person to ride Alpe d'huez in less than 40 minutes was Carlos Sastre in 2008, while Pantani did it in 36'40". Armstrong needed an ITT just to break 38 mins FFS. I was a big fan of Sastre, but nobody going faster than him since, even with improvements in materials and frame design speaks volumes.
So yeah, unless massive changes in frame design that greatly change climbing speeds are introduced, nobody has any business seriously challenging Pantani's records while saying they're clean.