Well I don't know when it started exactly, but I do remember that there as a little debate when he was banned for amphetamine use back in 2002. Not everyone believed his story about having taking unknown pills in a club. With what we know now though, that sounds like a pretty realistic thing for Ulle to do.
There were general doubts though and I think Ullrich was regularly asked about the topic of doping.
97 I was so young I don't remember any doping debates fullstop. The first time doping became known and a reality to me was with the Festina scandal I think. There was, from then on imo, enough public debate about doping to become suspicious yourself about the peloton in general and Ullrich in specific as well.
Does growing up in East Germany was a part of the discussion, considering how tied to the mythology of doping the DDR was ?
On the other hand I can see Ullrich having some slack because he was tagged as a prodigy very early on.