A lot of sports where the Olympics are completely central to prestige will see this, as athletes will retire after an Olympics if they have a big success and don't feel that they will still be on top four years later. Somebody like, say, Darya Domracheva, falls under that category, Mark Spitz also, of course, from your examples, and there are countless others in the classic 'higher, further, faster' Olympic disciplines as well.
Ones where they retire outside of that cycle at comparatively young ages tend to be a bit rarer, like Johannes Thingnes Bø's recent retirement. However, Dahlmeier as mentioned is a counterexample, while Martin Fourcade retired in 2020 at 31 years of age, winning literally his last race and being the 2nd best on the World Cup. Neuner is a bit in between; she didn't retire in the Olympic cycle, but she was afforded the chance to retire at a home World Championships having already achieved everything there was to achieve in the sport and made noises around wanting to start a family (this is also potentially a reason why early retirement may be more common in women's sport of course), so took that opportunity.
Another example from a sport where, though an Olympic sport, it is not central to the calendar, would be Marion Bartoli retiring at 28 almost immediately after winning at Wimbledon. But even then, both Bartoli and Spitz had brief, abortive comebacks, although the latter's was rather a publicity stunt nearly 20 years after his active career.
It seems like a lot of the time the crucial factor is that there needs to be something that keeps the athlete motivated and active in their retirement. Sometimes being too close to the sport can draw you back in (take Anna van der Breggen as an example), but because a lot of these people have dedicated their whole lives to sport for their entire adult life they are listless or unsure of how to proceed with regular life.