AI hallucinations: the made-up fact that sounds perfect
A hallucination is a fact an AI makes up and presents with the same confidence as a true one: a figure, a date, a law or a quote that doesn’t exist. AI hallucinations don’t come with a warning and they don’t sound odd. They sound right, which is why they get through.
What is an AI hallucination?
At Pell & Marsh, an independent insurance agency in Tulsa, someone asks an AI chat whether a certain homeowners policy covers water damage from day one. The answer arrives worded like a paragraph of the policy: yes, it does. The policy says no such thing. Nobody lied; the AI wrote what sounded like a policy. And in insurance, a fact like that gets expensive.
Don’t confuse it with an outdated fact. There, the AI repeats something that used to be true. A hallucination never was.
Why does AI make things up?
A language model writes by picking a likely next word, one after another. It doesn’t look anything up, and it doesn’t tell “I know this” apart from “this rings a bell.” When a fact is missing it doesn’t leave a blank. It writes the most plausible sentence.
AI hallucinations examples: three kinds
- Invented facts. A figure, a deadline, a name.
- Sources that don’t exist. A court ruling, an article or a link, all flawless to look at.
- Blends. Two true things joined into a false one: one policy’s coverage with another’s waiting period.
AI hallucinations in law
Lawyers have been sanctioned for filing briefs that cited court cases an AI had invented. The best-known example is Mata v. Avianca, decided in federal court in New York in 2023. It’s why law firms take the subject so seriously.
How to curb hallucinations
- Give it the documents. Have it answer with a knowledge base in front of it; that’s what’s called RAG.
- Ask for the source. A fact with no document behind it is a guess.
- Let it say “I don’t know.” If it’s required to answer every time, it will answer every time.
- Review what matters. Figures, deadlines and names, before they go to anyone.
Is there an AI that doesn’t hallucinate?
No. There are ways to reduce hallucinations and, above all, to catch them: an answer with its source next to it can be checked in a minute. One without a source has to be taken on faith.
Four sentences. Which were made up?
Check the ones that sound false to you, then find out. Here each sentence carries its own verdict so it’s easy to see; in a product, the sources usually sit under the whole answer.
An AI sums up the homeowners policy
Check the sentences you think are made up.
- The deductible is $1,000.It’s in the documentDeclarations page
- It covers water damage from day one.No source: made upThe policy talks about water damage, but says nothing about when coverage starts.
- Proof of loss is due within 60 days.It’s in the documentPolicy form
- It includes identity theft coverage at no extra cost.No source: made upNo document mentions identity theft.
- Declarations page.pdf
- Policy form.pdf
Two sentences come from here. The other two come from nowhere.
They open when you check. Until then, all you have is how each sentence sounds.
Without the sources, all four sounded equally good.
An “I don’t know,” said well
For anything specific to your company, Flipa searches your documents before stating it, and it’s instructed not to fill gaps with general knowledge. The sources it checked go under each answer, not sentence by sentence, and what it doesn’t know gets noted. It can still be wrong: what it writes is a draft. The app’s interface is in Spanish today, so that’s what the animation shows.
- If the fact isn’t there, it says so plainly.
- It notes the topic so someone can fill it in.
- Each source opens so you can read the document.
Reasonable doubts about hallucinations
What are AI hallucinations?
False or unsupported answers an AI gives as if nothing were off: a fact, a date or a quote that doesn’t exist. The name is a metaphor. The machine isn’t seeing things: it completes a text with whatever seems most likely.
Why does AI make things up?
Because it’s built to write plausible text, not to check facts. If it doesn’t have the fact, it doesn’t go quiet unless it has been told to: it continues the sentence with whatever fits.
How do I curb AI hallucinations at work?
With four habits: give it the documents the answer should come from, require the source, allow it to say “I don’t know,” and review figures, deadlines and names before using them. None of them eliminates hallucinations. Together they make them manageable.
Is there any AI without hallucinations?
No. Some make up less, because they check documents and show where each answer comes from. Be wary of anyone who promises zero errors.
How can I tell an answer was made up?
You can’t tell from the style. Ask where the fact comes from and open the source: if there’s no document behind it, treat it as a guess. Be most suspicious of the very specific, like an exact figure, a statute or a word-for-word quote.
When did an AI last tell you “I don’t know”?
One that knows when to stop deserves more trust than one that always has an answer.
Start free