I went to a Cracker Barrel a few years ago and they has a sign that read "Proudly serving people of color since 1973" or whatever year it had. Weird brag
It's to counter the racism, biggotry, and xenophobia present in pockets of American culture.
These shitty types of people avoid or hide within large cities - everyone is typically fine in these places. But small towns can sometimes have a serious problem with this if they are too "red" (Conservative).
In short, pockets of America are very tribal. People bond over similarities, and their shared disapproval of anything different, or out of the ordinary.
So where you come from, you welcome no races, no religions, no countries of origin, no sexual orientations, no genders; you don't stand with them and they're not safe, and the restrooms are a complete free for all?
I disagree. In my experience having a lack of restrooms just makes the place nasty and hostile. You can still visit places with plenty of public restrooms, and it's so much nicer. Every light rail station in Seattle smells like a toilet, the ones in Vancouver are way nicer. Removing restrooms doesn't remove the need to pee.