Andrii Bidochko

• Updated: June 19, 2026
• 6 min read

CiteCheck: Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text

Direct Answer

CiteCheck is a hybrid system that combines scholarly retrieval with a structured large‑language‑model verifier to spot fabricated or corrupted citations in AI‑generated scientific documents. By grounding each reference in real‑world metadata, it prevents the spread of misinformation and safeguards the credibility of automated research writing.

Background: Why This Problem Is Hard

Large language models excel at producing fluent prose, yet they frequently hallucinate citations—inventing papers that look plausible but either do not exist or contain inaccurate bibliographic details. This phenomenon undermines two core expectations of scientific communication: reproducibility and traceability. Traditional plagiarism detectors focus on text similarity, not on the factual existence of a referenced work. Moreover, existing citation‑checking tools rely on static databases or simple string matching, which cannot keep pace with the rapid expansion of scholarly archives or handle subtle metadata drift (e.g., a correct title but a wrong author list).

Current approaches also suffer from a “closed‑loop” limitation: they either query a search engine without any verification logic, or they prompt an LLM to judge a citation without external evidence. Both strategies leave a large error surface, especially when the model’s internal knowledge is outdated or when the citation is deliberately fabricated.

What the Researchers Propose

The authors introduce CiteCheck, a three‑stage pipeline that treats citation verification as a retrieval‑grounded decision problem. The framework consists of:

• Retrieval Engine: Queries external scholarly indexes (e.g., Crossref, Semantic Scholar) to collect candidate publications that could match the cited reference.
• Structured LLM Verifier: A language model receives a formatted comparison between the original citation and each retrieved candidate, then produces a fine‑grained similarity score.
• Label Mapping Module: Translates verifier scores into three human‑readable categories—Exact (perfect match), Minor (small metadata drift), and Major (substantial mismatch or non‑existent work).

This division of labor lets the system exploit the exhaustive coverage of scholarly APIs while retaining the nuanced reasoning capabilities of modern LLMs.

How It Works in Practice

The CiteCheck workflow proceeds as follows:

1. Input Extraction: The system parses a scientific manuscript, isolates each reference string, and normalizes it (e.g., stripping LaTeX commands).
2. Candidate Retrieval: For each normalized citation, a search query is sent to a scholarly API. The API returns a ranked list of potential matches, typically 5–10 entries.
3. Structured Comparison Prompt: The original citation and each candidate are placed into a JSON‑like schema that highlights fields such as title, authors, venue, year, and DOI. This schema is fed to the verifier LLM.
4. Verifier Scoring: The LLM evaluates field‑by‑field similarity, producing a numeric confidence for each candidate and a rationale explaining any discrepancies.
5. Decision Logic: The highest‑scoring candidate’s confidence is mapped to the Exact/Minor/Major label using calibrated thresholds derived from validation data.
6. Output Generation: The final report lists every citation with its assigned label, a confidence score, and a short justification that can be directly presented to authors or downstream agents.

What sets CiteCheck apart is the explicit grounding step: the verifier never operates in a vacuum; it always references an external, up‑to‑date scholarly record. This reduces the chance of “self‑reinforcing” hallucinations where the model simply repeats its own mistaken belief.

Evaluation & Results

To benchmark the system, the authors built a 982‑citation dataset drawn from physics papers, deliberately injecting three corruption types:

• Metadata Drift: Small alterations such as a misspelled author name or an off‑by‑one year.
• Fabricated References: Completely invented titles and DOIs.
• Partial Fabrications: Real titles paired with wrong venues or authors.

The dataset was split into training, validation, and held‑out test partitions. CiteCheck achieved an 88.7 % macro‑F1 score and 88.9 % overall accuracy on the test set, surpassing strong baselines that included GPT‑4, Claude‑3, and Gemini with both zero‑shot and few‑shot prompting. Notably, the retrieval‑grounded approach reduced major‑error rates by more than 30 % compared to pure LLM verification.

These results demonstrate that integrating external scholarly evidence dramatically improves the reliability of citation checks, even when the underlying LLM is already state‑of‑the‑art.

Why This Matters for AI Systems and Agents

For developers building autonomous research assistants, literature‑review bots, or AI‑driven knowledge graphs, citation fidelity is a non‑negotiable quality gate. CiteCheck offers a plug‑and‑play verification layer that can be embedded into any generation pipeline, ensuring that downstream agents never propagate false references.

In practice, this means:

• Higher Trust Scores: Platforms that surface AI‑generated papers can display a “Citation Verified” badge, boosting user confidence.
• Reduced Legal Risk: Mis‑attributed citations can lead to copyright disputes; automated checks mitigate that exposure.
• Improved Training Data: Curated corpora filtered through CiteCheck will contain cleaner metadata, which in turn benefits future model fine‑tuning.

Organizations that already leverage the UBOS platform overview can integrate CiteCheck as a micro‑service within their Workflow automation studio, allowing agents to request citation validation on‑the‑fly.

What Comes Next

While CiteCheck sets a strong baseline, several avenues remain open:

• Domain Expansion: Extending the retrieval layer to cover humanities, social sciences, and pre‑print servers beyond physics.
• Real‑Time Updates: Incorporating streaming metadata feeds so that newly published works are instantly searchable.
• Multilingual Support: Adapting the verifier to handle citations in non‑English scripts and transliteration variations.
• Human‑in‑the‑Loop Interfaces: Building UI components where researchers can manually approve or correct labels, feeding back into the calibration process.

Developers interested in experimenting with voice‑enabled agents can pair CiteCheck with the ElevenLabs AI voice integration to create spoken feedback loops for authors reviewing their drafts. Likewise, the OpenAI ChatGPT integration can serve as the verifier backbone, while the retrieval component remains agnostic to the underlying LLM.

For teams building collaborative research bots on messaging platforms, the Telegram integration on UBOS offers a low‑friction channel to surface citation‑validation results directly in chat, enabling rapid iteration.

Call to Action

Ready to fortify your AI‑generated research pipelines against citation hallucinations? Explore the UBOS homepage for ready‑made modules, or contact our About UBOS team to discuss custom integration. Empower your agents with trustworthy references today.

For the full technical details, see the original CiteCheck arXiv paper.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.