We have to stop ignoring AI’s hallucination problem

This right here is also the reason why AI fanboys get angry when they are told that LLMs are not intelligent or even thinking at all. They don't understand that in order for rational intelligence to exist, the LLMs should be able to have an internal, referential inner world of symbols, to contrast external input (training data) against and that is also capable of changing and molding to reality and truth criteria. No, tokens are not what I'm talking about. I'm talking about an internally consistent and persistent representation of the world. An identity, which is currently antithetical with the information model used to train LLMs. Let me try to illustrate.

I don't remember the details or technical terms but essentially, animal intelligence needs to experience a lot of things first hand in order to create an individualized model of the world which is used to direct behavior (language is just one form of behavior after all). This is very slow and labor intensive, but it means that animals are extremely good, when they get good, at adapting said skills to a messy reality. LLMs are transactional, they rely entirely on the correlation of patterns of input to itself. As a result they don't need years of experience, like humans for example, to develop skilled intelligent responses. They can do it in hours of sensing training input instead. But at the same time, they can never be certain of their results, and when faced with reality, they crumble because it's harder for it to adapt intelligently and effectively to the mess of reality.

LLMs are a solipsism experiment. A child is locked in a dark cave with nothing but a dim light and millions of pages of text, assume immortality and no need for food or water. As there is nothing else to do but look at the text they eventually develop the ability to understand how the symbols marked on the text relate to each other, how they are usually and typically assembled one next to the other. One day, a slit on a wall opens and the person receives a piece of paper with a prompt, a pencil and a blank page. Out of boredom, the person looks at the prompt, it recognizes the symbols and the pattern, and starts assembling the symbols on the blank page with the pencil. They are just trying to continue from the prompt what they think would typically follow or should follow afterwards. The slit in the wall opens again, and the person intuitively pushes the paper it just wrote into the slit.

For the people outside the cave, leaving prompts and receiving the novel piece of paper, it would look like an intelligent linguistic construction, it is grammatically correct, the sentences are correctly punctuated and structured. The words even make sense and it says intelligent things in accordance to the training text left inside and the prompt given. But once in a while it seems to hallucinate weird passages. They miss the point that, it is not hallucinating, it just has no sense of reality. Their reality is just the text. When the cave is opened and the person trapped inside is left into the light of the world, it would still be profoundly ignorant about it. When given the word sun, written on a piece of paper, they would have no idea that the word refers to the bright burning ball of gas above them. It would know the word, it would know how it is usually used to assemble text next to other words. But it won't know what it is.

LLMs are just like that, they just aren't actually intelligent as the person in this mental experiment. Because there's no way, currently, for these LLMs to actually sense and correlate the real world, or several sources of sensors into a mentalese internal model. This is currently the crux and the biggest problem on the field of AI as I understand it.

I fucking hate how OpenAi and other such companies claim their models "understand" language or are "fluent" in French. These are human attributes. Unless they made a synthetic brain, they can take these claims and shove them up their square tight corporate behinds.

They do not have any comprehension of the truth or untruth of what they are saying, and this means that when they say things that are true, they do not understand why those things are true.

Which can be beautifully exploited with sponsored content.

See Google I/O '24.

Well stated and explained. I'm not an AI researcher but I develop with LLMs quite a lot right now.

Hallucination is a huge problem we face when we're trying to use LLMs for non-fiction. It's a little bit like having a friend who can lie straight-faced and convincingly. You cannot distinguish whether they are telling you the truth or they're lying until you rely on the output.

I think one of the nearest solutions to this may be the addition of extra layers or observer engines that are very deterministic and trained on only extremely reputable sources, perhaps only peer reviewed trade journals, for example, or sources we deem trustworthy. Unfortunately this could only serve to improve our confidence in the facts, not remove hallucination entirely.

It's even feasible that we could have multiple observers with different domains of expertise (i.e. training sources) and voting capability to fact check and subjectively rate the LLMs output trustworthiness.

But all this will accomplish short term is to perhaps roll the dice in our favor a bit more often.

The perceived results from the end users however may significantly improve. Consider some human examples: sometimes people disagree with their doctor so they go see another doctor and another until they get the answer they want. Sometimes two very experienced lawyers both look at the facts and disagree.

The system that prevents me from knowingly stating something as true, despite not knowing, without some ability to back up my claims is my reputation and my personal values and ethics. LLMs can only pretend to have those traits when we tell them to.

usually, what I see is that the REPL they are using is never introspective enough. The ai cant on its own revert to a prevous state or give notes to itself because the response being fast and in linear time matters for a chatbot. ChatGPT can make really cool stuff when you ask it to break it's thoght process into steps. Ones it usually fails spectacularly at. It was like pulling teeth to get it to actually do the steps and not just give the bad answer anyway.

I'm not convinced about the "a human can say 'that's a little outside my area of expertise', but an LLM cannot." I'm sure there are a lot of examples in the training data set that contains qualification of answers and expression of uncertainty, so why would the model not be able to generate that output? I don't see why it would require an "understanding" for that specifically. I would suspect that better human reinforcement would make such answers possible.

Very long layman take. Why is your guesstimation so incredibly crucial to understand, then next thing important to understand then really, really important to understand, over and over, when you are not an expert?

I think where you are going wrong here is assuming that our internal perception is not also a hallucination by your definition. It absolutely is. But our minds are embodied, thus we are able check these hallucinations against some outside stimulus. Your gripe that current LLMs are unable to do that is really a criticism of the current implementations of AI, which are trained on some data, frozen, then restricted from further learning by design. Imagine if your mind was removed from all stimulus and then tested. That is what current LLMs are, and I doubt we could expect a human mind to behave much better in such a scenario. Just look at what happens to people cut off from social stimulus; their mental capacities degrade rapidly and that is just one type of stimulus.

Another problem with your analysis is that you expect the AI to do something that humans cannot do: cite sources without an external reference. Go ahead right now and from memory cite some source for something you know. Do not Google search, just remember where you got that knowledge. Now who is the one that cannot cite sources? The way we cite sources generally requires access to the source at that moment. Current LLMs do not have that by design. Once again, this is a gripe with implementation of a very new technology.

The main problem I have with so many of these "AI isn't really able to..." arguments is that no one is offering a rigorous definition of knowledge, understanding, introspection, etc in a way that can be measured and tested. Further, we just assume that humans are able to do all these things without any tests to see if we can. Don't even get me started on the free will vs illusory free will debate that remains unsettled after centuries. But the crux of many of these arguments is the assumption that humans can do it and are somehow uniquely able to do it. We had these same debates about levels of intelligence in animals long ago, and we found that there really isn't any intelligent capability that is uniquely human.

they do not understand why those things are true.

Some researchers compared the results of questions between chat gpt 3 and 4. One of the questions was about stacking items in a stable way. Chat gpt 3 just, in line with what you are saying about "without understanding", listed the items saying to place them one on top of each other. No way it would have worked.

Chat gpt 4, however, said that you should put the book down first, put the eggs in a 3 x 3 grid on top of the book, trap them in a way with a laptop so they don't roll around, and then put the bottle on top of the laptop standing up, and then balance the nail on the top of it...even noting you have to put the flat end of the nail down. This sounds a lot like understanding to me and not just rolling the dice hoping to be correct.

Yes, AI confidently gets stuff wrong. But let's all note that there is a whole subreddit dedicated to people being confidently wrong. One doesn't need to go any further than Lemmy to see people confidently claiming to know the truth about shit they should know is outside of their actual knowledge. We're all guilty of this. Including refusing to learn when we are wrong. Additionally, the argument that they can't learn doesn't make sense because models have definitely become better.

Now I'm not saying ai is conscious, I really don't know, but all of your shortcomings you've listed humans are guilty of too. So to use it as examples as to why it's always just a hallucination, or that our thoughts are not, doesn't seem to hold much water to me.

It's a nascent stage technology that reflects the world's words back at you in statistical order by way parsing user generated prompts. It's a reactive system with no autonomy to deviate from a template upon reset. It's no Rokos Basilisk inherently, just because

Its not just a calculator though.

Image generation requires no fact checking whatsoever, and some of the tools can do it well.

That said, LLMs will always have limitations and true AI is still a ways away.

Some problems lend themselves to "guess-and-check" approaches. This calculator is great at guessing, and it's usually "close enough".

The other calculator can check efficiently, but it can't solve the original problem.

Essentially this is the entire motivation for numerical methods.

That's not really right, because verifying solutions is usually much easier than finding them. A calculator that can take in arbitrary sets of formulas and produce answers for variables, but is sometimes wrong, is an entirely different beast than a calculator that can plug values into variables and evaluate expressions to check if they're correct.

As a matter of fact, I'm pretty sure that argument would also make quantum computing pointless - because quantum computers are probability based and can provide answers for difficult problems, but not consistently, so you want to use a regular computer to verify those answers.

Perhaps a better comparison would be a dictionary that can explain entire sentences, but requires you to then check each word in a regular dictionary and make sure it didn't mix them up completely? Though I guess that's actually exactly how LLMs operate...

The problem is people thinking the tool is a "calculator" (or fact-checker or search engine) while it's just a text generator. It's great for generating text.

But even then it can't keep a paragraph stable during the conversation. For me personally, the best antidote against the hype was to use the tool.

I don't judge people believing it's more than it is though. The industry is intentionally deceiving everyone about this and we also intuitively see intelligence when someone can eloquently express themselves. Seeing that in software seems magical.

We now have a great Star Trek like human machine interface. We only need real intelligence in the backend.

It would be a great comment if it represented reality, but as an analogy it's completely off.

LLM-based AI represents functionality that nothing other than the human mind and extensive research or singular expertise can replicate. There is no already existing 'second, better calculator' that has the same breadth of capabilities, particularly in areas involving language.

If you're only using it as a calculator (which was never the strength of an LLM in the first place), for problems you could already solve with a calculator because you understand what is required, then uh... yeah i mean use a calculator, that is the appropriate tool.

People who suggest, let's say, firing employees of crisis intervention hotline and replacing them with llms...

The part that's being ignored is that it's a problem, not the existence of the hallucinations themselves. Currently a lot of enthusiasts are just brushing it off with the equivalent of boys will be boys AIs will be AIs, which is fine until an AI, say, gets someone jailed by providing garbage caselaw citations.

And, um, you're greatly overestimating what someone like my technophobic mother knows about AI ( xkcd 2501: Average Familiarity seems apropos). There are a lot of people out there who never get into a conversation about LLMs.

It really needs to be a disqualifying factor for generative AI. Even using it for my hobbies is useless when I can't trust it knows dick about fuck. Every time I test the new version out it gets things so blatantly wrong and contradictory that I give up; it's not worth the effort. It's no surprise everywhere I've worked has outright banned its use for official work.

Maybe on Lemmy and in some pockets of social media. Elsewhere it definitely doesn't.

EDIT: Also I usually talk with IRL non-tech people about AI, just to check what they feel about it. Absolutely no one so far knew what hallucinations were.

It's easily the worst problem of Lemmy. Sometimes one guy has an issue with something and suddenly the whole thread is about that thing, as if everyone thought about it. No, you didn't think about it, you just read another person's comment and made another one instead of replying to it.

I never heard anyone complain about the term "hallucination" for AIs, but suddenly in this one thread there are 100 clonic comments instead of a single upvoted ones.

I get it, you don't like "hallucinate", just upvote the existing comment about it and move on. If you have anything to add, reply to that comment.

I don't know why this specific thing is so common on Lemmy though, I don't think it happened in reddit.

They're nowadays using it to humanize neural networks, and thus oversell its capabilities.

What I don’t like about it is that it makes it sound more benign than it is. Which also points to who decided to use that term - AI promoters/proponents.

Edit: it’s like all of the bills/acts in congress where they name them something like “The Protect Children Online Act” and you ask, “well, what does it do?” And they say something like, “it lets local police read all of your messages so they can look for any dangers to children.”

that's not fun and dramatic and clickbaity though

Technically, humans are just bio machines, running very complicated software. AI just isn't there yet.

is it that difficult to look up the answer from a reliable source?

With the current state of search engines and their content (almost completely unrelated garbage and shitty blogs make in like 3 minutes with 1/4 of the content poorly copy-pasted out of context from stackoverflow and most of the rest being pop-ups and ads), YES

SEO ""engineers"" deserve the guillotine

because some jobs have to produce a bunch of bullshit text that no one will read quickly, or else parse a bunch of bullshit text for a single phrase in the midst of it all and put it in a report.

If it keeps me from going to stack and interacting with those degenerates, yes

I think you're giving a glorified encyclopedia too much credit. The difference between us and "AI" is that we can approach knowledge from a problem solving position. We do approximate the laws of physics, but we don't blindly take our beliefs and run with it. We put we come up with a theory that then gets rigorously criticized, then come up with ways to test that theory, then be critical of the test results and eventually we come to consensus that based on our understandings that thing is true. We've built entire frameworks to reduce our "hallucinations". The reason we even know we have blind spots is because we're so critical of our own "hallucinations" that we end up deliberately looking for our blind spots.

But the "AI" doesn't do that. It can't do that. The "AI" can't solve problems, it can't be critical of itself or what information its giving out. All our current "AI" can do is word vomit itself into a reasonable answer. Sometimes the word vomit is factually correct, sometimes it's just nonsense.

You are right that theoretically hallucinations cannot be solved, but in practicality we ourselves have come up with solutions to minimize it. We could probably do something similar with "AI" but not when the AI is just a LLM that fumbles into sentences.

Could not have said it better. The whole reason contemporary programs haven't been able to adapt to the ambiguity of real world situations is because they require rigidly defined parameters to function. LLMs and AI make assumptions and act on shaky info - That's the whole point. If people waited for complete understanding of every circumstance and topic, we'd constantly be trapped in indecision. Without the ability to test their assumptions in the real world, LLMs will be like children.

ok so to give you an um ackshually here.

Technically if we were to develop a real general artificial general intelligence, it would be limited to the amount of knowledge that it has, but so is any given human. And it's advantage would still be scale of operations compared to a human, since it can realistically operate on all known theoretical and practical information, where as for a human that's simply not possible.

Though presumably, it would also be influenced by AI posting that we already have now, to some degree, the question is how it responds to that, and how well it can determine the difference between that and real human posting.

the reason why hallucinations are such a big problem currently is simply due to the fact that it's literally a predictive text model, it doesn't know anything. That simply wouldn't be true for a general artificial intelligence. Not that it couldn't hallucinate, but it wouldn't hallucinate to the same degree, and possibly with greater motives in mind.

A lot of the reason human biology tends to obfuscate certain things is simply due to the way it's evolved, as well as it's potential advantages in our life. The reason we can't see our blindspots is due to the fact that it would be much more difficult to process things otherwise. It's the same reason our eyesight is flipped as well. It's the same reason pain is interpreted the way that it is.

a big mistake you are making here is stating that it must be fed information that it knows to be true, this is not inherently true. You can train a model on all of the wrong things to do, as long it has the capability to understand this, it shouldn't be a problem.

For predictive models? This is probably the case, but you can also poison the well so to speak, when it comes to those even.

You assume the physical world is all there is or that the AI has any real intelligence at all. It's a damn chinese room.

Very long layman take. Why is there always so many of these on every ai post? What do you get from guesstimating how the technology works?

Their real power is their ability to understand language and context.

...they do exactly none of that.

So if you type cats but the item is called pets get no results. Now the ai can find context based matches in the auto complete list.

Google added context search to Gmail and it's infuriating. I'm looking for an exact phrase that I even put in quotes but Gmail returns a long list of emails that are vaguely related to the search word.

Searching with synonym matching is almost.decades old at this point. I worked on it as an undergrad in the early 2000s.and it wasn't new then, just complicated. Google's version improved over other search algorithms for a long time.and then trashed it by letting AI take over.

Exactly. The big problem with LLMs is that they're so good at mimicking understanding that people forget that they don't actually have understanding of anything beyond language itself.

The thing they excel at, and should be used for, is exactly what you say - a natural language interface between humans and software.

Like in your example, an LLM doesn't know what a cat is, but it knows what words describe a cat based on training data - and for a search engine, that's all you need.

That's called "fuzzy" matching, it's existed for a long, long time. We didn't need "AI" to do that.

That's why I only use Perplexity. ChatGPT can't give me sources unless I pay, so I can't trust information it gives me and it also hallucinated a lot when coding, it was faster to search in the official documentation rather than correcting and debugging code "generated" by ChatGPT.

I use Perplexity + SearXNG, so I can search a lot faster, cite sources and it also makes summaries of your search, so it saves me time while writing introductions and so.

It sometimes hallucinates too and cites weird sources, but it's faster for me to correct and search for better sources given the context and more ideas. In summary, when/if you're correcting the prompts and searching apart from Perplexity, you already have something useful.

BTW, I try not to use it a lot, but it's way better for my workflow.

Hallucinations, like depression, is a multifaceted issue. Training data is only a piece of it. Quantized models, overfitted training models rely on memory at the cost of obviously correct training data. Poorly structured Inferences can confuse a model.

Rest assured, this isn't just training data.

I think you are spot on. I tend to think the problems may begin to outnumber the potentials.

Oh that is hilarious! Just on my first listen but I don't quite get the lyrics - am deeply disappointed that the video doesn't have subs. 🤭

Found it on Spotify. It's so much worse with lyrics. Thank you for sharing a version without them! 🙏

Here is an alternative Piped link(s):

https://www.piped.video/watch?v=RObuKTeHoxo

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source; check me out at GitHub.

The Italians actually have a name for that kind of gibberish talking that sounds real. I did some VO work on a project being directed by an Italian guy and he explained what he wanted me to do by explaining the term to me first. I'm afraid it's been way too long since he told me for me to remember it though.

Another example would be the La Linea cartoons, where the main character speaks a gibberish which seems to approximate Italian to my ears.

https://www.youtube.com/watch?v=ldff__DwMBc

hallucination refers to a specific bug (AI confidently BSing) rather than all bugs as a whole

It's not a bug, it's a natural consequence of the methodology. A language model won't always be correct when it doesn't know what it is saying.

https://en.m.wikipedia.org/wiki/Hallucination_(artificial_intelligence)

The term "hallucinations" originally came from computer researchers working with image producing AI systems. I think you might be hallucinating yourself 😉

Because hallucinations pretty much exactly describes what's happening? All of your suggested terms are less descriptive of what the issue is.

The definition of hallucination:

A hallucination is a perception in the absence of an external stimulus.

In the case of generative AI, it's generating output that doesn't match it's training data "stimulus". Or in other words, false statements, or "facts" that don't exist in reality.

Ty. As soon as I saw the headline, I knew I wouldn't be finding value in the article.

That's actually not a bad analogy. Politicians are rarely versed in topics they talk about and instead are just reiterating stuff experts told them. That's why lobbyism works as well as it does.

You do enterprise resource planning with AI hallucinations?

You just run the same query a bunch of times and see how consistent the answer is.

A lot of people are developing what I'd call superstitions on some way to overcome LLm limitations. I remember someone swearing they fixed the problem by appending "Ensure the response does not contain hallucinations" to every prompt.

In my experience, what you describe is not a reliable method. Sometimes it's really attached to the same sort of mistakes for the same query. I've seen it double down, when instructed a facet of the answer was incorrect and to revise, several times I'd get "sorry for the incorrect information", followed by exact same mistake. On the flip side, to the extent it "works", it works on valid responses too, meaning an extra pass to ward off "hallucinations" you end up gaslighting the model and it changes the previously correct answer as if it were a hallucination.

Generative AI is hardly "good" yet, either morally or as a product.