Guide · Verifying
AI search and fact-checking
AI search answers a question in prose and cites pages it retrieved while answering. It fails in a specific way: the sentence can be wrong while the citation next to it is real, because the model wrote the sentence and attached the source afterwards. Checking means opening the source and finding the claim in it.
How long does it take to learn?
The method is an afternoon. Building the reflex of not believing a fluent paragraph takes a few weeks of catching yourself. It is less a body of knowledge than a habit, and habits are slower than facts.
The fastest path, in order
- 01
Learn the difference between generating and retrieving
A model with no search attached is recalling a compressed impression of its training data. A model with search reads pages first. The two fail differently, and knowing which one you are talking to changes what you check.
- 02
Understand why sources get invented
A fabricated citation is not a bug in a lookup, it is a plausible-looking string produced by the same process that produced the sentence. That is why fabricated references look so right: plausibility is exactly what the machine optimises for.
- 03
Separate the claim from the citation
Read the linked page and find the specific sentence supporting the specific claim. Most verification failures are not fake links, they are real links that do not say what the summary said they said.
- 04
Triangulate on anything that matters
Two independent sources that do not cite each other, or you do not have a fact yet. Aggregators quoting one another look like consensus and are one source wearing four hats.
- 05
Date everything
Ask when the source was published and when the model last knew anything. Most confidently wrong AI answers are correct statements about a world that has moved on, which is the hardest error to spot because nothing about it looks wrong.
- 06
Write down what you could not verify
An explicit unverified line is worth more than a confident sentence you are quietly unsure about. It is also what makes work defensible when someone eventually checks.
Tools worth your time
| Tool | What it is for |
|---|---|
| Perplexity | Search-first assistant that shows its sources inline, built for this workflow. |
| ChatGPT search | Retrieval attached to a strong model, with links you should still open. |
| Claude with web search | Long-context reading of the pages it finds, good for comparing several sources at once. |
| Google Scholar | Where you confirm an academic citation actually exists, in about ten seconds. |
| Semantic Scholar | Paper metadata and citation graphs, useful for checking whether a study is contested. |
| The original page | Not a tool. Still the step almost everyone skips, and the only one that settles anything. |
No detector or checker settles this for you. Tools that claim to verify AI output are themselves models, and running a model over a model to see whether the first one lied is not a chain of custody. The load-bearing step is a human opening the source.
Mistakes that cost people weeks
Treating a citation as proof it was read
The link resolves, the domain is respectable, so it goes in. Then the page turns out to say something adjacent, or the opposite. Open it and find the sentence.
Trusting fluency
We read confidence as competence, and models are fluent by construction. The best-written paragraph in a document is not more likely to be true than the clumsy one, and it feels like it is.
Asking the model whether it was right
It will agree with you either way, because agreeing is what the next token usually is. Self-checking is not verification, it is a second opinion from the same source.