I once read that it's an epigenetic thing and it can be found across the animal kingdom. Some animals are born more sensitive and others less and this is important for the species or social group as a whole. This actually happens on a neuron level.
The less sensitive kind needs to actively search out stimulation, whereas you can leave the more sensitive one alone with a flower and they'll be a happy camper.
And there's so much more to it, for example developmental. Have you ever noticed the difference in sound levels in people's homes? In some places it's just like a warzone. TV on max, dogs barking, kids screaming. Imagine growing up with that. Like a fish in water.
And then there's all the processing disorders..
You can train yourself though if you value it. I enjoy encounters and it bothered me a lot, so I just kept going to busy cafés and bars until my brain finally got the memo. It keeps surprising me how my hearing has become like a sort of precision microphone.