Can’t you take into account the incredibly small size of the wizarding community? Maybe I’ve been interpreting completely wrong all these years, but I never thought that witches and wizards numbered over 3-4 thousand in the whole world.
It’s unclear. The UK wizarding community is organised politically in separation from those around the world, you get the impression that Hogwarts isn’t the only UK wizarding school because it’s seen as the elite place to go. In the triwizarding tournament it competes with other elite schools. It can’t be an elite school if it’s the only one. There are also entire wizarding communities (such as at Hogs Meade and Godric’s hollow, and Diagon Ally is a very large place that seems frequently busy and able to service more than a global population of max 4k (so presumable only a handful of hundreds in the UK?) would need.
Then factor in the presence on non-wizarding, magical species such as the Goblins, House Elves, Centaurs, Giants and so on and consider that the wizarding community maintains a dominant and often very oppressive control over these perfectly sapient beings so then we have to question how they do that and what numbers they must have to be able to do so.
Even if the wizarding community were that small that just makes the lack of political participation more abhorrent because it’d be much more functional for each person to be able to participate in some form of direct democracy, or otherwise have some form of community accountability without needing to create such a burgeoning bureaucracy to do so (it seems the vast majority of employability in the wizarding community is that of civil servant with the retail/service sector second, with a few adventurers, authors and journalists on the side).