I've had this particular question leveled at me literally dozens of times over the years, so luckily in my case the answer is quite simple.
Some people like the sport while others like certain riders. I'm in the first group.My interest in a race isn't influenced all that much by who is or isn't riding. It's influenced by what type of race it's likely to be.
This isn't depreciative of the latter group. People like what they like. I like the sport and enjoy watching it. Other people like certain riders and watch specifically to see them ride. Different interests, simply.
In practical terms, this translates to: people in the first group will have a stable interest in the sport and don't want cheats in it but are more willing to believe the testimony of insiders, while people in the second group are more likely to watch races where their favorite riders/teams are participating and their interest in the sport tends to have peaks and throughs month after month or year after year.
The first group believes in something......believes that the sport is interesting and exciting and can be even better if it's made fair.
The second group also believes in something.....believes that certain people are role models and finds it odd that someone can enjoy the sport without being a fan of a specific role model.
Neither of the two is better or worse, just different
Your church analogy would be apt if everyone's mentality was "The riders" = "The sport". But to many people they're not the same. You can have faith and trust in one and none at all in the other.
Hopefully I didn't make my point too muddled