Guide
Why AI assistants invent article numbers, and how to stop it
Assistants cite articles that do not exist or do not say what they claim. Why it happens, three real examples, and how grounding in the official text fixes it.
Other languages:DeutschEspañolFrançaisPolski
AI assistants invent article numbers because they answer about law from memory: a model stores patterns, not the official text, so it produces a citation that looks right rather than one it has read. The fix is to make the assistant look the provision up in the official source before it answers, and to cite only what came back. That turns every article number into something that was retrieved, quoted and linked, not generated.
Why does a model get citations wrong?
Three reasons, and none of them goes away with a bigger model:
- Memory is not the text. A model learned law from whatever was online when it was trained: commentaries, forum posts, old versions. It reproduces the most likely sentence, and the most likely article number, not the one in force.
- Law has dates. A provision changes; its old wording stays in the training data. Without a date, the model cannot tell 2025 from 2026.
- Look-alike numbers. Acts are full of thresholds, deadlines and rates that differ by one word. A model blends them.
What does that look like in practice?
Take one question: what is the VAT small-business exemption limit in Poland? The answer is 240,000 zł, in art. 113 ust. 1 of the VAT act, since 1 January 2026. In tests of grounded assistants, asking that one question in many phrasings brought three wrong answers, each because the search returned the wrong provision:
| Wrong answer | Where it actually comes from |
|---|---|
| 200,000 zł | art. 145l, a KSeF e-invoicing transitional rule with its own 200,000 zł threshold (and the exemption limit before 2026) |
| 2,000,000 | art. 2 pkt 25, the “small taxpayer” status, counted in euro, a different thing |
| 200,000,000 | art. 145l again, read as if it set the exemption |
Each is a real figure from the right act, attached to the wrong question. The search had returned a neighbouring provision with a similar figure, and the answer quoted it faithfully. That is the lesson: a source alone does not stop wrong citations if it returns the wrong provision, which is why the fourth rule below matters as much as the first three. And it is what makes wrong citations dangerous: they are plausible enough to survive a quick read.
How do you stop it?
- Retrieve before answering. Give the assistant a tool that searches the official consolidated text and make it call the tool for every legal question. Over MCP, an assistant like Claude or ChatGPT does this on its own once the tool is connected (how).
- Cite only what was retrieved. Every article number in the answer must come from a returned provision, with its link. No provision, no citation.
- Ask with a date. “As of 31 December 2025” gets the wording in force then. A source that keeps every version of every provision answers that correctly.
- Test with real questions. Keep a set of questions with checked answers, in everyday and formal wording, and run it whenever the source or the prompt changes. A fix that makes one phrasing right must not make another wrong.
What does VeriActa do about it?
It is built around those four rules. Each result quotes the provision from the official consolidated text, with its citation in the country’s format and a link to the source; it returns the version for the date you ask about; and every change to the search is checked against real questions first, and ships only when every correct answer stays correct. An example conversation shows what that looks like in an assistant.
Frequently asked questions
Will a better model stop inventing citations?
It will invent fewer, but it still answers from memory, with no date and no source. Only retrieval gives each citation something to point at.
Can I check a citation an assistant gave me?
Yes: open the official text and read the article. With a grounded assistant, the answer already links it.
Does this only matter for lawyers?
No. Accountants, payroll teams and anyone relying on an assistant for a threshold, a rate or a deadline hit the same problem, often more, because they ask about numbers.