Original title: Improving Reliability and Explainability of Medical Question Answering Through Atomic Fact-Checking in Retrieval-Augmented Large Language Models: Creation and Validation Study
Juraj Vladika, A. Domres, Mai Nguyen, R. Moser +9 · September 2026
Medical AI chatbots can sound smart, but they often hallucinate, meaning they make up false facts. They also fail to show where their facts come from, making them risky for doctors to use. To fix this, researchers built a fact-checking system. Here is how it works: the system takes an AI-generated medical answer and splits it into atomic factsthe smallest pieces of information that cannot be broken down further while still making sense, which are the smallest single units of information. Next, it checks every single unit against trusted medical guidelines. If a fact is FALSE, the system identifies the mistake, fixes it, and links the true information directly to supporting literature. The team found that this system reliably spotted and removed mistakes. It reached an 80% hallucination detection rate and delivered up to a 50% overall answer improvement. It achieved its highest gains on tough, real-world tumor board questions, making AI answers more accurate and easy to trace.
Imagine a student writing a book report. They gather facts from different sources, but sometimes they misremember a detail. To fix this, they check each sentence against the original books. If a sentence is wrong, they rewrite it with the correct information. Here, the AI generates medical answers, and the system checks each tiny claim (atomic fact) against a database of medical guidelines, correcting any mistakes and noting the source.
Doctors and patients are cautious about using AI for medical advice because it sometimes makes up facts. This fact-checking method makes AI answers more reliable by verifying each claim against official guidelines. It also shows exactly where each fact comes from, which satisfies regulators and builds trust.
The abstract reports strong results on specific oncology and neurology datasets, but generalizability to other medical domains is not established. The method depends on the availability of a comprehensive, authoritative knowledge base of medical guidelines, which may not exist for all conditions. The evaluation focuses on open-source models; performance on proprietary models is not addressed. The observed improvement rates (up to 50% answer improvement, 80% hallucination detection) may vary across different questions and models.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Improving Reliability and Explainability of Medical Question Answering Through Atomic Fact-Checking in Retrieval-Augmented Large Language Models: Creation and Validation Study — Juraj Vladika, A. Domres, Mai Nguyen et al. — Journal of Medical Internet Research — doi:10.2196/92090
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.