To be fair, there is safety in numbers and consequently I also see a benefit to combining them. I'm a dude, and I would rather walk down a busy pedestrian mall than a sketchy half-empty street or an urban trail on a moonless night.
Same thing with bathrooms. I imagine women feel better in a bathroom where other women and other people are present and aware, vs a nearly empty bathroom where a creep could walk in and isolate them at any moment. I've been to places with large unisex bathrooms (hot springs, spas, etc.) and there's enough people in there that no one is gonna try anything stupid/inappropriate without being immediately noticed. Bathroom troublemakers are a small subset of the universally large group of people who have to use the restroom anyway.
I only support gender-segregated bathrooms because I don't want a bunch of chicks clogging up the mensroom with their slow pissing/shitting/bloody paper towels on the floor. :D I am unreasonable and self-centered, but anyone who has ever been to a concert knows which line is longer.