AI Limitations & Fact-Checking Strategies
Learn to recognise AI limitations, spot hallucinations, and build verification into your study workflow.
🚨 Critical Reality Check
AI tools can be useful learning assistants, but fluency is not evidence. They can confidently present false information, fabricate sources, and make mathematical errors. Understanding these limitations is essential for using AI responsibly in your academic work.
Never submit work containing AI-generated facts, statistics, interpretations or citations without verifying them yourself.
Understanding AI Hallucinations
A "hallucination" occurs when an AI model generates information that sounds plausible but is actually incorrect or entirely fabricated. This is not a bug or malfunction – it is an inherent characteristic of how large language models work. They predict likely-sounding text rather than retrieving verified facts.
🔢 Common Types
- Fabricated academic sources that do not exist
- Incorrect statistics or dates
- Made-up quotes attributed to real people
- False details about historical events
- Invented technical specifications
- Misattributed authorship or publication details
⚠️ Why They Happen
- Models predict plausible text, not truth
- Training data contains errors and inconsistencies
- Models try to please users by providing answers
- Complex questions push models beyond their knowledge
- No built-in fact-checking mechanism
- Statistical patterns can mislead
🎯 When Most Likely
- Obscure or niche topics with less training data
- Recent events after the model's knowledge cut-off
- Requests for specific citations or sources
- Complex mathematical calculations
- Detailed technical specifications
- Questions requiring precise factual knowledge
Fluency Is Not Evidence
A polished answer can still be wrong. Agreement between two AI tools is also not the same as independent corroboration, because models may share training sources, assumptions or failure patterns.
Use AI to generate possible explanations, questions and checks. Use original sources, data, worked calculations, official guidance and your own judgement to decide what is supported.
Conservative
Directly supported by the source or data you have checked. Lowest risk for academic writing.
Ask: What evidence supports this?
Interpretive
A reasonable reading of the evidence, but it depends on assumptions you should name.
Ask: What additional evidence is needed?
Speculative
Useful for brainstorming, but not suitable as a claim unless you can verify it independently.
Ask: What would count against this?
Essential Fact-Checking Strategies for Students
Verification is structural: build these checks into your prompt, notes and editing process rather than leaving them until the final proofread.
Why: AI models can generate plausible-sounding but incorrect numbers.
How:
- Cross-reference with official sources (government statistics, academic databases)
- Check multiple independent sources
- Look for the original data source, not secondary reports
- Be especially careful with comparative statistics
Example: If AI says "60% of students use AI for homework," find the actual study, check the sample size, date, and methodology before citing it.
Why: AI frequently fabricates citations that sound legitimate but do not exist.
How:
- Search for the exact paper title in Google Scholar
- Verify author names and publication years
- Check if the journal or publisher exists
- Access the actual source to confirm it says what the AI claims
- Use university library databases for academic sources
Warning: Never include a citation in your work without verifying the source exists and is relevant. Fabricated references are academic misconduct.
Why: AI can be confidently wrong about technical details, formulas, and specialised knowledge.
How:
- Check formulas and calculations manually or with specialised software
- Verify technical specifications against manufacturer documentation
- Consult authoritative textbooks or academic sources
- Ask subject matter experts (lecturers, tutors) if uncertain
Tip: For mathematical problems, work through the steps yourself. For scientific facts, check recent peer-reviewed papers.
Why: Different models may expose weak spots, alternative interpretations or missing checks. Agreement between models is useful, but it is not proof.
How:
- Ask the same question to more than one tool when the issue matters
- Look for differences, uncertainty and assumptions
- Investigate any discrepancies thoroughly
- Use specialised tools (Perplexity with citations, NotebookLM for your documents)
Strategy: If models disagree, do additional research using academic databases. If they agree, still verify the claim in a source you can cite.
Why: Models cannot know about events after their training data cut-off unless they have web access.
How:
- Know your AI tool's knowledge cut-off date
- Use tools with web search for recent information
- Verify any "current" information with news sources or official websites
- Be cautious with evolving fields (technology, medicine, current events)
Note: Knowledge cut-off dates and web access change regularly. Check the current help page for the tool you are using, especially for recent events, policy, law, medicine, prices or technical specifications.
Recommended Verification Tools
Academic Databases
- Google Scholar: Free, comprehensive academic search
- Web of Science: Citation tracking and verification
- PubMed: Medical and life sciences
- JSTOR: Humanities and social sciences
- IEEE Xplore: Engineering and technology
Fact-Checking Tools
- Perplexity AI: Provides citations with responses
- Consensus: Academic research search with citations
- Semantic Scholar: AI-powered research tool with TLDRs
- Connected Papers: Visual paper exploration
Official Sources
- ONS: UK official statistics
- WHO: Health information
- Gov.uk: UK government data
- Eurostat: European statistics
- UN Data: International statistics
Source-Grounded Tools Help, but They Are Not Magic
Tools such as NotebookLM and retrieval-based chat systems can answer from a set of documents you provide. This is useful because it gives you a narrower source base and often points back to passages.
However, source-grounded does not mean automatically correct. The tool may miss a relevant passage, over-weight a weak match, misread a quotation, or sound confident from only part of the evidence.
Use It For
- Finding candidate passages in readings you have chosen
- Generating questions for closer reading
- Comparing what a small set of sources appears to say
Still Check
- Does the cited passage actually support the answer?
- Are important sources missing from the notebook?
- Has the tool confused summary with analysis?
Understanding Context Windows & Token Limits
The context window (also called context length) is the maximum amount of text an AI model can process at once. This includes your entire conversation history, uploaded documents, and the model's responses.
What Counts Towards the Limit
- All previous messages in the conversation
- All AI responses you have received
- Any uploaded documents or files
- System instructions (invisible to you)
- The new message you are sending
What Happens at the Limit
- Oldest messages may be "forgotten"
- Context from early conversation is lost
- You may need to start a new conversation
- Uploaded documents might be truncated
- Some tools show warnings before reaching limits
What to Check Before Uploading a Large File
Capacity
How much text can this tool process at once, and does it warn you when material is skipped?
Retrieval
Does the tool search the whole file, selected passages, or only a summary of the upload?
Privacy
Will the upload be stored, reviewed, shared, or used for training? Are you using a personal or institution-approved account?
Verification
Can you trace every important answer back to the original passage, page, table or dataset?
Note: Model limits and product settings change regularly, so check the current documentation for the exact tool you are using.
Best Practices for Managing Context
✅ Do These
- Start new conversations for different topics
- Summarise key points periodically in long conversations
- Upload only relevant sections of long documents
- Be concise in your prompts when possible
- Use tools with larger context for document analysis
- Save important responses outside the conversation
❌ Avoid These
- Keeping one conversation going for weeks
- Uploading entire textbooks when you need one chapter
- Assuming the model remembers everything from the start
- Mixing multiple unrelated topics in one conversation
- Ignoring context limit warnings
- Relying on very old context for current questions
🎯 The Golden Rule of AI Verification
If you cannot verify it, do not cite it. If you cannot explain it, you do not understand it.
Your lecturers expect you to understand and be able to defend every claim in your work. AI should enhance your learning and research process, not replace your critical thinking and verification responsibilities.