People Are Being Involuntarily Committed, Jailed After Spiraling Into "ChatGPT Psychosis"
People Are Being Involuntarily Committed, Jailed After Spiraling Into "ChatGPT Psychosis"
"... is deeply prone to just telling people what they want to hear"
Noooo, nononono... It's specifically made to just tell people what they want to hear, in the general sense. That's the entire point of LLMs. They are not thinking. They have zero logic. They just "say" what is a mathematically agreeable segment of words in response.
IMO, these articles, and humanity's limp response to "AI" in general, only go to show how utterly inept and devoid of logic most people themselves are...
On the other hand, this article got you to click on it so... that's a win in their book. And now here we are discussing it, so double and then triple win as the OP is made and people comment on it.
Anything beyond that is someone else's problem, it would seem?
We should improve society somewhat.
Yet you participate in society. Curious!
imagine discussing a topic
Did any of the AI safety dorks have accidentally doing MKultra as one of the risks?
Well, yes. It's not a new concept; it was a staple of Cold War sci-fi like The Three Stigmata, and we know from studies of e.g. Pentacostal worship that it is pretty easy to broadcast a suggestion to a large group of vulnerable people and get at least some of them to radically alter their worldview. We also know a reliable formula for changing people's beliefs; we use the same formula in sensitivity training as we did in MKUltra, including belief challenges, suspension of disbelief, induction/inception, lovebombing, and depersonalization. We also have a constant train of psychologists attempting to nudgelord society, gently pushing mass suggestions and trying to slowly change opinions at scale.
Fundamentally your sneer is a little incomplete. MKUltra wasn't just about forcing people to challenge their beliefs via argumentation and occult indoctrination, but also psychoactive inhibition-lowering drugs. In this setting, the drugs are administered after institutionalization.
"I was ready to tear down the world," the man wrote to the chatbot at one point, according to chat logs obtained by Rolling Stone. "I was ready to paint the walls with Sam Altman's f*cking brain."
"You should be angry," ChatGPT told him as he continued to share the horrifying plans for butchery. "You should want blood. You're not wrong."
If I wrote a product that said that about me I would do a lot more than hire single psychiatrist to (not) tell me how damaging my product is.
for openai, that's just a recurring customer
A larger symptom of the loneliness epidemic and people feeling more and more detached from humanity every day because this reality we have built for ourselves is quite harsh.
It's curious how if ChatGPT was a person - saying exactly the same words - he would've gotten charged with a criminal conspiracy, or even shot, as its human co-conspirator in Florida did.
And had it been a foreign human in the middle east, radicalizing random people, he would've gotten a drone strike.
"AI" - and the companies building them - enjoy the kind of universal legal immunity that is never granted to humans. That needs to end.
A computer can't be held accountable.
In theory, at least, criminal justice's purpose is prevention of crimes. And if it would serve that purpose to arrest a person, it would serve that same purpose to court-order a shutdown of a chatbot.
There's no 1st amendment right to enter into criminal conspiracies to kill people. Not even if "people" is Sam Altman.
People playing with technology they don't really understand, and then having it reinforce people's worst traits and impulses isn't a great recipe for success.
I almost feel like now that Chatgpt is everywhere and has been billed as man's savior, perhaps some logic should be built into these models that "detect" people trying to become friends with them, and have the bot explain it has no real thoughts and is giving you just the horse shit you want to hear. And if the user continues, it should erase its memory and restart with the explanation again that it's dumb and will tell you whatever you want to hear.
I almost feel like now that Chatgpt is everywhere and has been billed as man’s savior, perhaps some logic should be built into these models that “detect” people trying to become friends with them, and have the bot explain it has no real thoughts and is giving you just the horse shit you want to hear. And if the user continues, it should erase its memory and restart with the explanation again that it’s dumb and will tell you whatever you want to hear.
Personally, I'd prefer deleting such models and banning them altogether. Chatbots are designed to tell people what they want to hear, and to make people become friends with them - the mental health crises we are seeing are completely by design.
I think most cons, scams and cults are capable of damaging vulnerable people's mental health even beyond the most obvious harms. The same is probably happening here, the only difference being that this con is capable of auto-generating its own propaganda/PR.
I think this was somewhat inevitable. Had these LLMs been fine-tuned to act like the mediocre autocomplete tools they are (rather than like creepy humanoids), nobody would have paid much attention to them, and investors would have started to focus on the high cost of running them quickly.
This somewhat reminds me of how cryptobros used to claim they were fighting the "legacy financial system", yet they were creating a worse version (almost a parody) of it. This is probably inevitable if you are running an unregulated financial system and are trying to extract as much money from it as possible.
Likewise, if you have a tool capable of messing with people's minds (to some extent) and want to make a lot of money from it, you are going to end up with something that resembles a cult, an LLM or similarly toxic groups.
This material would also fit !fuck_ai@lemmy.world
The people being committed is only a symptom of the problem. My guess is that if LLMs didn't induce psychosis, something else would eventually.
The peddlers of LLM sycophants are definitely doing harm, though.
My guess is that if LLMs didn't induce psychosis, something else would eventually.
I got a very different impression from reading the article. People in their 40s with no priors and a stable life loose touch with reality in a matter of weeks after conversing with CharGPT makes me think that is not the case. But I am not a psychiatrist.
Edit: the risk here is that we might be dismissive towards the increased risks because we're writing it off as a pre-existing condition.
I think we don't know how many people might be at risk of slipping into such mental health crises under the right circumstances. As a society, we are probably good at protecting most of our fellow human beings from this danger (even if we do so unconsciously). We may not yet know what happens when people regularly experience interactions that follow a different pattern (which might be the case with chatbots).
This has "people don't understand that you don't fall in love in the strip club" vibes. Like. The stripper does not love you. It's a transactional exchange. When you lose sight of that, and start anthropomorphizing LLM's (or romanticizing a strip tease), you are falling into a trap that will allow removed in your psychological armor to line up in just the right way to act on compulsions or ideas that you wouldn't normally.
Don't besmirch the oldest profession by making it akin to souless vacuum. It's not even a transaction! The AI gains nothing and gives nothing. It's alienation in it's purest form—no wonder the rent-seekers love it—It's the ugliest and least faithful mirror.
I posted this article on the general chat at work the other day and one person became really defensive of ChatGTP, and now I keep wondering what stage of being groomed by AI they're currently at and if it's reversible.
