Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review

This paper is a PRISMA-guided systematic literature review that synthesizes 30 peer-reviewed studies on Retrieval-Augmented Generation (RAG) in healthcare, mapping architectural variants and evaluation practices across three key clinical applications: diagnostic support, EHR summarization, and medical question answering.

Problem Statement

LLMs deployed in healthcare must meet extremely high standards of factual accuracy, since hallucinations can directly harm patient outcomes, and RAG has emerged as a leading mitigation strategy by grounding generation in external knowledge. However, research on RAG's clinical deployment is fragmented across isolated studies with inconsistent architectures and evaluation practices, making it hard for practitioners to identify best practices or unresolved risks.

Key Novelty

A PRISMA-compliant systematic review specifically scoped to RAG in three prevalent clinical use cases (diagnostics, EHR summarization, medical QA), rather than a general LLM or RAG survey
A consolidated taxonomy linking naive, advanced, and modular RAG architectures to their deployment patterns across healthcare applications
Synthesis of clinical-specific evaluation metrics (FactScore, RadGraph-F1, MED-F1) alongside standard NLP metrics, clarifying which metrics best capture factual and clinical validity
Cross-study identification of persistent technical barriers—retrieval noise, domain shift, generation latency, and limited explainability—common to clinical RAG systems

Evaluation Highlights

Comparative analysis of standard NLP metrics (e.g., BLEU/ROUGE/F1-style scores) versus clinical-specific metrics (FactScore, RadGraph-F1, MED-F1) used across the 30 reviewed studies
Methodological rigor assessed via adherence to PRISMA guidelines for study selection, reducing bias in the literature synthesis itself

Signal Assessment

3/10 As a systematic review rather than a novel method or model, the paper's contribution lies in organizing and synthesizing existing knowledge; it is valuable for the field's coherence but does not introduce new algorithms, architectures, or empirical results.

Methodology

Defined research questions and a systematic search strategy across academic databases to identify candidate studies on RAG in clinical settings
Applied PRISMA screening and eligibility criteria to filter the literature down to 30 peer-reviewed studies
Classified included studies by RAG architecture type (naive, advanced, modular) and by clinical application area
Extracted and compared evaluation metrics, reported outcomes, and recurring challenges to synthesize cross-study patterns and gaps

System Components

Naive RAG

Baseline retrieve-then-generate pipeline using single-pass similarity search over an external corpus before LLM generation

Advanced RAG

Enhanced pipelines adding query rewriting, re-ranking, or iterative retrieval steps to improve the relevance of retrieved context

Modular RAG

Composable architecture with interchangeable retrieval, ranking, and generation modules tailored to specific clinical workflows

Clinical Evaluation Metrics

Domain-specific scoring methods (FactScore, RadGraph-F1, MED-F1) designed to measure factual accuracy and clinical relevance beyond generic text-overlap metrics

Challenge Taxonomy

Structured categorization of recurring RAG deployment issues in healthcare: retrieval noise, domain shift, latency, and explainability

Results

Aspect	General-Domain RAG Practice	Healthcare RAG Findings	Implication
Evaluation metrics	BLEU/ROUGE/generic F1	Growing use of FactScore, RadGraph-F1, MED-F1	Clinical-specific metrics better capture factual and medical validity
Architecture complexity	Predominantly naive RAG	Increasing shift to advanced/modular RAG in diagnostics and EHR tasks	More complex pipelines reduce retrieval noise but increase latency
Key challenges	Hallucination, generic retrieval errors	Retrieval noise, domain shift, latency, limited explainability	Clinical deployment requires targeted solutions beyond general RAG fixes

Key Takeaways

Evaluate healthcare RAG systems with clinical-specific metrics (FactScore, RadGraph-F1, MED-F1) in addition to standard NLP metrics to properly assess factual and clinical validity
Match architecture complexity to the task: naive RAG may suffice for simple QA, while diagnostic support and EHR summarization benefit from advanced or modular designs that reduce retrieval noise
Explicitly test and mitigate domain shift when applying RAG models across different clinical data distributions or institutions
Prioritize explainability and latency optimization as key engineering gaps that currently limit real-world clinical adoption of RAG systems

Abstract

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieval to improve factual consistency and reduce hallucinations. Despite growing interest, its use in healthcare remains fragmented. This paper presents a Systematic Literature Review (SLR) following PRISMA guidelines, synthesizing 30 peer-reviewed studies on RAG in clinical domains, focusing on three of its most prevalent and promising applications in diagnostic support, electronic health record (EHR) summarization, and medical question answering. We synthesize the existing architectural variants (naïve, advanced, and modular) and examine their deployment across these applications. Persistent challenges are identified, including retrieval noise (irrelevant or low-quality retrieved information), domain shift (performance degradation when models are applied to data distributions different from their training set), generation latency, and limited explainability. Evaluation strategies are compared using both standard metrics and clinical-specific metrics, FactScore, RadGraph-F1, and MED-F1, which are particularly critical for ensuring factual accuracy, medical validity, and clinical relevance. This synthesis offers a domain-focused perspective to guide researchers, healthcare providers, and policymakers in developing reliable, interpretable, and clinically aligned AI systems, laying the groundwork for future innovation in RAG-based healthcare solutions.