A confident AI answer is not the same as a well-supported answer. Even when it includes an author, date and impressive percentage, the source may support something much narrower than the claim.
You do not need specialist software to start checking. Begin with one question: does the evidence actually support these words?
1. Separate the claim from the confidence
Consider an answer that says a study “proves” a benefit “for everyone.” Those words make several promises: that the research measured the claimed benefit, established what caused it and applies broadly.
Check each promise separately. A source can support one part of a sentence without supporting the rest.
2. Check the quotation—not just the citation
The following authors, study and results are entirely fictional teaching examples, not real research.
Suppose an AI answer says:
“Rowan and Vale (2025) concluded: ‘AI tutors improve memory by 40% for everyone.’”
But the supplied fictional study excerpt actually says:
“On an immediate 20-question quiz, the AI group averaged 14 correct answers and the comparison group averaged 10.”
The excerpt describes 20 adult volunteers after one vocabulary practice session. It says long-term retention was not tested, and the assignment method and participants’ prior vocabulary knowledge were not reported.
This is a misquotation: the sentence presented inside quotation marks does not appear in the supplied excerpt. It also overstates the evidence. An immediate quiz is not a test of long-term memory, and a small group cannot establish benefits for everyone.
Matching the authors and year would not fix either problem. Citation identity and claim support are separate checks.
3. Ask what the percentage measures
The calculation is:
(14 − 10) ÷ 10 × 100 = 40%
That is 40% more correct answers relative to the comparison group’s average. It is not evidence of “40% better memory.”
The quiz scores were 70% and 50%: a difference of 20 percentage points. Both descriptions can be mathematically correct, but they describe different things.
A more careful summary would be:
“In this small fictional study, the AI-assisted group averaged 14 correct answers versus 10 on an immediate quiz. The excerpt does not establish causation, long-term retention or benefits for everyone.”
4. Try a two-minute exercise
Everything in this exercise is fictional.
An AI summary says: “Harbour Council approved a permanent cycle lane after every consultation respondent supported it.”
The supplied records say:
• Of 80 respondents, 50 supported the proposal, 20 opposed it and 10 were unsure.
• The council approved a six-month trial. A permanent scheme requires another decision.
• Responses were voluntary, not a representative survey.
Before reading on, identify two errors and write one corrected sentence.
Answer
“Permanent” is contradicted by the trial decision. “Every respondent” is contradicted by the response counts.
A corrected summary is: “The fictional council approved a six-month cycle-lane trial. Of 80 voluntary consultation respondents, 62.5% supported the proposal.”
Whether most residents supported it is not established. Respondents are not automatically representative of all residents.
A checklist you can reuse
1. Write down the exact claim.
2. Locate the relevant source passage—not just its title.
3. Compare quotations word for word.
4. Check what the numbers measure.
5. Look for missing dates, exceptions and study limitations.
6. Choose a verdict: supported, contradicted or not established.
“Not established” does not mean “false.” It means the available evidence is insufficient. In real-world checks, also verify the original publication’s identity and look for corrections or retractions.
Want more structured practice?
AI Claim Detective: Five Practical Verification Labs is the workbook behind these exercises. The £5 first-edition PDF offer is designed for adult beginners and covers separating claims from opinions, inspecting citations, finding missing context, expressing uncertainty and comparing summaries with primary evidence.
The workbook text includes exercises, worked answers, reusable checklists and a detective-note template. Its exercises are self-contained and fictional: no live AI tool, paid subscription or internet connection is needed to work through them.
See the product listing for confirmed PDF file details and download availability before purchasing. This is introductory practice—not professional certification or a guarantee that you will detect every error.