A Systematic Literature Review of Retrieval-Augmented Generation: Techniques, Metrics, and Challenges

This paper presents a PRISMA 2020-compliant systematic literature review of 128 studies (2020–2025) on Retrieval-Augmented Generation, synthesizing empirical evidence on RAG's effectiveness over parametric-only LLMs and mapping the field's architectures, datasets, and evaluation practices.

Problem Statement

Empirical findings on RAG are scattered across heterogeneous tasks, systems, and metrics, making it hard to draw reliable, cumulative conclusions about what actually improves grounding and reduces hallucination. This fragmentation obscures which architectural and evaluation practices are maturing versus which remain unresolved (e.g., efficiency, security), slowing the transition of RAG from research prototypes to dependable production systems.

Key Novelty

A rigorous, PRISMA 2020-governed systematic review of RAG spanning five major databases and 128 citation-filtered studies from Jan 2020–May 2025
Identifies a clear architectural evolution from early DPR+seq2seq baselines to modular, policy-driven RAG with hybrid/structure-aware retrieval, uncertainty-triggered loops, memory, and emerging multimodality
Systematically surfaces under-examined dimensions in the literature—efficiency and security (poisoning, leakage, jailbreaks)—alongside evaluation practice gaps (overlap-heavy EM/F1 dominance)
Translates synthesis findings into concrete research directions: holistic quality-cost-safety benchmarks, budget-aware retrieval/tool-use policies, and provenance-aware pipelines

Evaluation Highlights

128 studies included after PRISMA screening, distributed across knowledge-intensive tasks (35/128, 27.3%), open-domain QA (20/128, 15.6%), software engineering (13/128, 10.2%), and medical domains (11/128, 8.6%)
Descriptive, theme-based synthesis (no meta-analysis) organized around RQ1–RQ4 due to heterogeneity of study designs and metrics across the corpus

Signal Assessment

3/10 As a systematic literature review, the paper contributes rigorous synthesis, taxonomy, and gap analysis rather than a new model or technique, making it a useful field-organizing reference rather than an advance in the state of the art.

Methodology

Formulated RQ1–RQ4 and followed PRISMA 2020 reporting guidelines for transparent, reproducible review conduct
Searched ACM Digital Library, IEEE Xplore, Scopus, ScienceDirect, and DBLP (last searched 13 May 2025) for RAG and retrieval-supported text-generation studies from Jan 2020–May 2025
Applied inclusion/exclusion criteria: citation thresholds (≥15 for 2025, ≥30 for 2024 or earlier), English-language, original contributions, and accessible full text
Conducted single-reviewer screening with independent verification and discussion, brief bias appraisal, and advisory-only LLM assistance
Performed descriptive/thematic synthesis with summary counts and frequencies, forgoing meta-analysis due to methodological heterogeneity

System Components

PRISMA 2020 Review Protocol

Standardized, transparent search-screen-synthesize workflow ensuring methodological rigor and reproducibility of the review

Multi-source Literature Search

Combined querying of ACM Digital Library, IEEE Xplore, Scopus, ScienceDirect, and DBLP with citation-based quality filtering

RQ-Aligned Thematic Taxonomy

Framework categorizing studies by task/domain, architecture, evaluation practice, and identified limitations/gaps

Modular Policy-Driven RAG Pattern

Synthesized architectural trend combining hybrid/structure-aware retrieval, uncertainty-triggered control loops, memory, and multimodal inputs, superseding static DPR+seq2seq pipelines

Evaluation Practice Landscape

Cross-study catalogue of metrics used, from overlap-based EM/F1 to retrieval diagnostics (Recall@k, MRR@k), human judgment, and LLM-as-judge protocols

Results

Domain/Category	Study Count	Share	Observation
Knowledge-intensive tasks	35/128	27.3%	Largest application category in the corpus
Open-domain QA	20/128	15.6%	Traditional RAG benchmark domain
Software engineering	13/128	10.2%	Emerging, non-traditional RAG application
Medical domains	11/128	8.6%	Growing high-stakes deployment area
Core architecture trend	N/A	N/A	Shift from DPR+seq2seq to modular, policy-driven RAG with hybrid retrieval and memory

Key Takeaways

Move beyond EM/F1 overlap metrics toward holistic benchmarks that jointly report quality, cost/latency, and safety, since current evaluation practice is skewed toward accuracy alone
Adopt budget-aware retrieval and tool-use policies to control when and how much to retrieve, balancing answer quality against inference cost in production systems
Treat RAG security risks (data poisoning, information leakage, jailbreaks) as first-class design concerns, not afterthoughts, given their limited current coverage in the literature
Prioritize provenance-aware pipelines that expose uncertainty and traceable evidence to support auditability and trust in deployed RAG systems
Architect for modularity: hybrid/structure-aware retrieval, uncertainty-triggered control loops, memory, and multimodality are converging as the dominant next-generation RAG design pattern

Abstract

Background: Retrieval-augmented generation (RAG) aims to reduce hallucinations and outdated knowledge by grounding LLM outputs in retrieved evidence, but empirical results are scattered across tasks, systems, and metrics, limiting cumulative insight. Objective: We aimed to synthesise empirical evidence on RAG effectiveness versus parametric-only baselines, map datasets/architectures/evaluation practices, and surface limitations and research gaps. Methods: This systematic review was conducted and reported in accordance with PRISMA 2020. We searched the ACM Digital Library, IEEE Xplore, Scopus, ScienceDirect, and DBLP; all sources were last searched on 13 May 2025. This included studies from January 2020–May 2025 that addressed RAG or similar retrieval-supported systems producing text output, met citation thresholds (≥15 for 2025; ≥30 for 2024 or earlier), and offered original contributions; excluded non-English items, irrelevant works, duplicates, and records without accessible full text. Bias was appraised with a brief checklist; screening used one reviewer with an independent check and discussion. LLM suggestions were advisory only; 2025 citation thresholds were adjusted to limit citation-lag. We used a descriptive approach to synthesise the results, organising studies by themes aligned to RQ1–RQ4 and reporting summary counts/frequencies; no meta-analysis was undertaken due to heterogeneity of designs and metrics. Results: We included 128 studies spanning knowledge-intensive tasks (35/128; 27.3%), open-domain QA (20/128; 15.6%), software engineering (13/128; 10.2%), and medical domains (11/128; 8.6%). Methods have shifted from DPR+seq2seq baselines to modular, policy-driven RAG with hybrid/structure-aware retrieval, uncertainty-triggered loops, memory, and emerging multimodality. Evaluation remains overlap-heavy (EM/F1), with increasing use of retrieval diagnostics (e.g., Recall@k, MRR@k), human judgements, and LLM-as-judge protocols. Efficiency and security (poisoning, leakage, jailbreaks) are growing concerns. Discussion: Evidence supports a shift to modular, policy-driven RAG, combining hybrid/structure-aware retrieval, uncertainty-aware control, memory, and multimodality, to improve grounding and efficiency. To advance from prototypes to dependable systems, we recommend: (i) holistic benchmarks pairing quality with cost/latency and safety, (ii) budget-aware retrieval/tool-use policies, and (iii) provenance-aware pipelines that expose uncertainty and deliver traceable evidence. We note the evidence base may be affected by citation-lag from the inclusion thresholds and by English-only, five-library coverage. Funding: Advanced Research and Engineering Centre. Registration: Not registered.