Perhaps it’s not easy to decide where the line of legality should go though, which is why this topic is controversial.
It's not easy. Especially when you need to determine what's a controversial opinion and what's hate speech.
For example (and this is NOT anything I agree with)-- if one said 'I don't believe gay people should be allowed to adopt children, because science shows both male and female influences are more helpful when applied together for a child's development' what is that? Is that hate speech because it advocates taking rights away from gay people? Is it an opinion stated with the goal of protecting children?
Does it become illegal to express almost any position that isn't pro-gay?
It's a VERY slippery slope.
Certain speech is criminal like inciting violence. If someone said 'I'm going to buy a gun and kill gay people, and you all should kill gay people too' that is a specific statement of criminal intent and also inciting violence. That will get you cops knocking on your door (and rightly so).
You can apply a 'test' to that- does it show specific intent to commit a crime? Does it encourage others to commit crimes? Yes on both.
But how do you 'test' someone saying they don't think gay people should be allowed to adopt? How do you tell from a few words if they have a hate-filled heart, or if they legitimately think gay people can't provide a loving home? You can't.
For the record- I'm using LGBT as an example. I personally liberal-libertarian--- I believe married gay couples should have guns to defend their adopted children and pot farms from criminals, with single payer healthcare to keep them alive if they get hurt. I'm against almost any effort to take away anyone's rights.
So I'll fight for the asshat's right to say 'fuck the gays' just as hard as I'll fight for the LGBT person's right to marry, adopt, and use whatever bathroom they want (provided they wash their hands).