The number of people was a political compromise between individual rights and States rights, but so was a Senate and House.
The electoral college was primarily designed to enable states to vote despite a communication delay that could take months.
It did great at that, actually. How would California have up to date info on what's going on in Washington when the fastest mode of travel was a horse? It wouldn't.
Instead of voting based on information that's outdated and potentially inaccurate, best to pick some people you trust to vote in your interests, and send them to Washington. Let them get caught up, and vote how they will as your representative.
Then States can sort out their own voting time and method, with no real concern for it being simultaneous or consistent because news travels so slow anyway. The important thing was authorized people would show up by the expected federal voting time, and if that happened, everyone did well enough.
Of course, now they can cast their vote without leaving the state, and coordination is possible, but here we are holding the bag on a lack of accounting for technological progress.