ChatGPT gets code questions wrong 52% of the time

Any one of us who actually codes/scripts knows ChatGPT spits out hot garbage when asked to produce anything beyond maybe a single short one or two line code snippet or bash/powershell command. Like the article said the AI lacks context of what you're trying to do. It will confidently spit out either completely wrong or made up code with commands that don't even exist.

Also, this will go really fucking well. Don't give them any ideas.

Kabir said, "From our findings and observation from this research, we would suggest that Stack Overflow may want to incorporate effective methods to detect toxicity and negative sentiments in comments and answers in order to improve sentiment and politeness.

It's interesting that the sharp fall in traffic mimics the fall of Twitter and Reddit.

Anecdotally, I would find code answrs on Reddit or Twitter, that would direct to Stack to view the full answer, or a more complete explanation of why X should be done that way.

Considering the (relatively) small decline, I'm surprised that Stack think the answer is ChatGPT(or similar), and not the loss of semantic details added by a Reddit/Twitter thread.

52% In the first year is pretty cool, excited to see how it will evolve.

I am using ChatGPT4+ with the code interpreter and I am finding it closer to 90% accurate writing 50-200 lines of c# code in Unity. Beyond 200 it starts to have more issues and the accuracy drops. It has saved me so much time refactoring my project.