I meant there's no point going to an office if nobody I'm working with is going to be there. I might as well just stay home too.
Just to be clear I'm not sitting here yearning for face to face contact because my colleagues are so gorgeous or to socialize with them. These aren't "hi how are you" meetings. My field is super technical with tons of back and forth questions, problem solving, and drafts, between me, my staff, other colleagues, clients, whoever. And I'm right smack in the middle conducting the orchestra trying to keep it all together. The in-person time I'm talking about is going over minute technical details with staff/partners/colleagues/whoever to review or because they have questions, ask them 'prove this, explain that, make a note that this thing over here has been fucked for twenty years,' etc, or just general problem solving and decision making etc.
These things are infinitely easier when you're not herding cats trying to get people on a zoom call, it's fast paced and we're all juggling thirty things on fire at once, sometimes you gotta catch someone for twenty seconds while they're taking a piss, that's just how it is.
So when everyone is in different offices, different time zones, some in office, some at home, it just increases bottlenecks and decreases efficiency. That's been my experience over the last few years, YMMV