Question-Adaptive Graph Learning for Multi-hop Retrieval Augmented Generation
Problem Statement
Multi-hop questions require RAG systems to identify and combine multiple pieces of evidence, but existing retrievers often fail to capture the complex semantic structure of such questions and are easily misled by irrelevant retrieved content. This bottleneck degrades downstream LLM answer quality, particularly as the number of required reasoning hops increases.
Key Novelty
- Multi-information Level Knowledge Graph (Multi-L KG) that explicitly models information at multiple granularities/levels for richer multi-hop question representation
- Question-Adaptive Graph Neural Network (Quest-GNN) with intra- and inter-level message passing whose aggregation is conditioned on the question itself, enabling noise-robust, multi-granular retrieval
- Two synthesized data generation strategies for pre-training Quest-GNN, improving representation robustness without relying solely on scarce labeled multi-hop data
Evaluation Highlights
- Extensive experiments across multi-hop QA benchmarks showing consistent gains over existing multi-hop RAG retrieval methods
- Up to 33.8% relative improvement on high-hop (more complex) question subsets, indicating the method scales well with reasoning complexity
Signal Assessment
Methodology
- Construct a Multi-L KG from the retrieval corpus that captures information at multiple levels of granularity relevant to multi-hop reasoning
- Encode the input question and use it to condition message passing in Quest-GNN via intra-level (same granularity) and inter-level (cross-granularity) aggregation steps
- Pre-train Quest-GNN using two synthesized data generation strategies to bolster robustness and generalization before task-specific fine-tuning
- Apply the learned question-adaptive graph representations to retrieve relevant multi-hop evidence, which is then fed to an LLM for answer generation
- Evaluate the full pipeline on multi-hop QA benchmarks stratified by hop count, comparing against existing multi-hop RAG baselines
System Components
A knowledge graph structure that represents multiple information granularities (e.g., entity, passage, document levels) to give a more comprehensive representation of multi-hop questions and their supporting evidence.
A GNN that performs representation learning over the Multi-L KG, using the question to guide how information is aggregated at each step.
Two complementary message-passing mechanisms that aggregate information within a single granularity level and across different levels, enabling multi-granular reasoning.
A mechanism that conditions message passing on the question representation, focusing aggregation on question-relevant nodes/edges and filtering out irrelevant noise.
Two data synthesis approaches used to pre-train Quest-GNN, improving robustness of learned representations for downstream multi-hop retrieval.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| High-hop question performance | Existing multi-hop RAG retrievers | Quest-GNN framework | +33.8% (relative improvement) |
| General multi-hop QA benchmarks | Prior graph/RAG-based retrieval methods | Quest-GNN framework | Consistent improvement (exact figures not in abstract) |
Key Takeaways
- Explicitly modeling multiple information granularities in a knowledge graph can help retrievers better handle the compositional structure of multi-hop questions.
- Conditioning GNN message passing on the question is an effective way to suppress retrieval noise, with benefits that grow as hop count (question complexity) increases.
- Synthesized pre-training data can substitute for scarce labeled multi-hop supervision when training graph-based retrieval components.
- The approach could be adopted as a drop-in retrieval enhancement module within existing multi-hop RAG pipelines rather than requiring a full system overhaul.
Abstract
Retrieval-augmented generation (RAG) has demonstrated its ability to enhance Large Language Models (LLMs) by integrating external knowledge sources. However, multi-hop questions, which require the identification of multiple knowledge targets to form a synthesized answer, raise new challenges for RAG systems. Under the multi-hop settings, existing methods often struggle to fully understand the questions with complex semantic structures and are susceptible to irrelevant noise during the retrieval of multiple information targets. To address these limitations, we propose a novel graph representation learning framework for multi-hop question retrieval. We first introduce a Multi-information Level Knowledge Graph (Multi-L KG) to model various information levels for a more comprehensive understanding of multi-hop questions. Based on this, we design a Question-Adaptive Graph Neural Network (Quest-GNN) for representation learning on the Multi-L KG. Quest-GNN employs intra/inter-level message passing mechanisms, and in each message passing the information aggregation is guided by the question, which not only facilitates multi-granular information aggregation but also significantly reduces the impact of noise. To enhance its ability to learn robust representations, we further propose two synthesized data generation strategies for pre-training the Quest-GNN. Extensive experimental results demonstrate the effectiveness of our framework in multi-hop scenarios, especially in high-hop questions the improvement can reach 33.8\%. The code is available at: https://github.com/Jerry2398/QSGNN.