Well, silly assumptions or not. The financial side of things among the teams, sponsors and organizers is hardly open source info that we have access to. But the fact that we can't read their spreadsheets doesn't mean they don't exist. Some factors lend themselves to more or less qualified speculation, though.
To point out but a few of the variables that we " the average Joes"

haven't touched upon yet, reducing the no. of riders and increasing the no. of teams, may foster more doping. With fragmentation you'll inevitably lose some transparency as to the various antidoping-regimes instituated in the teams. Allowing smaller "Pop'n-Mom kitchen-sink"-teams onboard, with less to lose and a big fat carrot dangling in front of them, ie. a sudden place in the spotlight, may tip the scale the wrong way.
Also, more teams would mean more sponsors, ensuring a loss of "exclusivity" for the big sponsors of today, and as a possible consequence some big sponsors might lose interest. It's a bit like increasing the no. of jerseys. (Jersey for best manners, anybody?? Tidiest team-bus?) Each team would get less exposure. Securing a spot in the big GT's is
supposed to be a big thing. Exclusive. Prestigious. Trust the french, italians and spaniards to know about how these things work.
Increasing the no. of teams may also lead to more aggressive riding, and as a consequence, a more complex and dangerous workplace for the riders. Do we really need that to feel "excitement" in our armchairs? You saw the riders' reaction to banning the radios on some stages in TdF. Not what I'd call a great success, would you? And that was just a minor adjustment, trying to reward riders with wits enough to make wise on-the-spur decisions, and a deviation from the automaton on remote from the team-car of today.
More likely than not the number of riders per team as of today is the result of a long and evoulutionary process rather than a random number picked out of a hat. Changing the number without knowing the consequences would be plain stupid.
And to what end? Because some forum members have perceived that the three GT's weren't as exciting this year as they used to be? I'd question their perception rather than the GT's teamstructures, to be honest. See what could be done about TV coverage and commentary etc. Having the podium girls in wet T-shirts or whatever.