I don't know, I would not say that I knew automatically when I was born what's the difference between "man" and "woman". Of course I have had clear feelings and preferences about a variety of topics, some instinctive and well defined, ever since I was born. But I don't think that's determined by a label. They clearly can fall into a particular label, but only "after the fact".
To me, "man" and "woman" can't be labels that go beyond the social/behavioral because I don't know what it feels like to be a man anymore than what I know it feels like to be a woman.. I only know myself, I can't possibly compare what I feel to what others feel, because those feelings are a "qualia" that cannot be simply be transmitted with words.
And without communication to compare and reference, I could not judge whether what I feel is "man" or "woman" at the level that you choose to do it. To me it's logically impossible to set a gender at such a deep level.
An analogy would be how I can never be sure that other people experience the same thing I experience when we both see and point to the color "green". "Green" is a construct based on our common understanding of the experience a particular wavelength that is emitted by an object we are pointing to. But the label "green" cannot go beyond that external consensus, because what I experience when the impulses caused by that wavelength reach my brain could perfectly be different than what you experience when that same wavelength causes yours.
We might even agree on what are the wavelengths that we call green, based on our own internal experience, because the experience I feel when seeing green might be similar every time I see green (and the same will happen for you)... but that does not mean that we are both having the same experience, it could be that what you experience as blue I experience as green and that what you experience as green I experience as blue, and yet every time we would agree on calling the same wavelengths the same way, because we would have learned to call them that way.
So it would be meaningless to say beyond any social agreement that I deeply think that this color should be "green" only based on my experience alone, because it would not be any different from saying that this color should be "blue"... the only thing that makes us both agree on calling a particular color experience as green and not blue is the social understanding of that experience matching a common external pattern we both agree on, and that we each match it with our respective (and possibly different) subjective experiences (qualia) when we see that color.