"Hallucination" gets used three different ways in the same meeting: as a joke, as a vague liability, and as the reason a colleague got sacked. Since we spend our days researching them, here's the primer we wish we'd had.
A confident answer to a question the model doesn't know how to get right
A hallucination is output from a generative model that's presented as a fact but isn't. A case that doesn't exist. A statistic no study produced. A quote nobody said. What makes it dangerous is that there is no error message, it's wrong while sounding fluent, specific, and formatted exactly like being right.
Hallucinating implies a malfunction, a 'brain fart': suddenly perceiving something false. However, language models just predict plausible text, one token at a time, based on patterns in their training data. When a model writes a real citation it's doing exactly the same thing as when it writes a fake one: producing something shaped like the citations it has seen. Some researchers prefer "confabulation" to "hallucination", it's the clinical term for fluently inventing plausible memories with no intent to deceive. The model is just doing what it was built to do.
That's also why hallucinations aren't a bug that next model version can just fix. Newer models hallucinate less but a system whose core operation is "generate plausible text" will always be able to generate plausible falsehoods. There's a catch too: as the models improve, the surviving fabrications get more convincing.
What they look like in practice
Most of what we see falls into a few standard shapes.
A complete fabrication. A source that simply doesn't exist, an invented case, a fake paper, a standard nobody wrote. That time of thing.
A source mismatch. Everything in the reference is individually real, but the pieces don't belong together. The case name is genuine and the citation points somewhere else. This passes a lazy check ("does this case exist? yes, fine"). The other flagged authority in that same order was exactly this.
A misquoted citation. Source exists, citation is correct, but the source doesn't say what the document claims it says. Somewhere between the actual holding and the argument being built, the model drifted into the holding-shaped sentence that fit best. This is the hardest type to catch since it only falls down when someone reads the source against the claim. The Srinagar court told judges to quote passages verbatim rather than paraphrase, and this is why.
A fill-in-the-blank. These turn up constantly in summaries, where the model fills gaps in the source material with plausible filler.
A yes-man. Ask a model to "summarise the cases supporting our position" and it may manufacture the support, because the prompt assumed support existed. The model is just trying to please and may even lie to do so.
Why "just check everything" keeps not happening
Every professional body's guidance lands on the same instruction: a human needs to verify all AI output against real sources. Correct, obviously. But notice what it asks. Tracing every reference, confirming the match, reading the passage against the claim, done properly, that can take longer than the drafting time the AI saved. People skip verification not because they're careless, but because they're hard up against a deadline and don't have the time.
Hallucinations are therefore a feature of a system built to sound right rather than be right. In professions where a confident falsehood has real consequences, that cost is felt particularly keenly. This is the risk we're looking to mitigate at scale. When people are under time pressure, verification only happens reliably when it's cheap and fast enough to happen automatically.