Really nothing more than empirical evidence...
1. The era i.e America's first real foray into international racing and definitely the grand tours - with a U.S. based pro team, 7-Eleven in 1985 - not forgetting Jonathan Boyer in '81 who managed to secure a spot on a continental team, becoming the first American to ride in the tour.
2. The company he kept and the teams he rode with Jim Ochowicz, Mike Neel, Steve Bauer, Greg Lemond, Paul Keochli... Team 7-Eleven, La Vie Claire...
In Greg's words:
"I know my old teammate, Eric Boyer, retired because he didn't want to touch the stuff, and I know many other people who made it through clean, such as Andy Hampsten and Steve Bauer. Every rider on La Vie Claire was clean, that was Paul Keochli's big deal to make sure he had a clean team."
http://www.roble.net/marquis/coaching/lemond98.html
3. His history, his path into the sport and his timely exit - as outlined in this superb interview... the whole story in his words:
http://nyvelocity.com/content/interviews/2009/andy-hampsten-interview
With cycling you can never be sure, but his history and depth of character rings true for me.