For me, because my homeschooling was part and parcel with a very conservative religious upbringing, the most difficult part is, in a sense, still ongoing.
I don't want to imply my childhood was like Unbreakable Kimmy Schmidt or something, I was aware of what was going on in the world and stuff, but certainly sheltered and heavily influenced by my parents in how interpreted what was going on.
Breaking away from that, interacting with society more directly and more diversely than I otherwise would have, has overlapped with breaking away from a lot of the cultural and religious beliefs that were instilled in me.
That's not an overnight process. I think anyone who has left behind a religious upbringing could relate to that, homeschooled or not, but the homeschooling and relative 'isolation' adds a twist to it.