GrepSeek: Training Search Agents for Direct Corpus Interaction
Problem Statement
Conventional retrieval-augmented search agents depend on pre-computed document indices and ranked-list retrievers, which can limit precise evidence composition and flexibility over raw corpus content. Training agents to interact directly with corpora via RL is notoriously unstable due to large action spaces and sparse, long-horizon rewards, and naive shell-based execution over large corpora is computationally impractical at scale.
Key Novelty
- Introduces Direct Corpus Interaction (DCI): a search paradigm where the agent issues executable shell commands against the raw corpus instead of querying a pre-built retrieval index
- Two-stage training pipeline combining a cold-start dataset built via an answer-aware Tutor and answer-blind Planner (producing verified, causally grounded trajectories) with GRPO-based RL refinement for stable policy learning
- A semantics-preserving sharded-parallel execution engine that accelerates shell-based retrieval up to 7.6x while guaranteeing byte-exact equivalence to sequential execution, making DCI computationally viable at corpus scale
Evaluation Highlights
- GrepSeek achieves the strongest overall token-level F1 and Exact Match across seven open-domain QA benchmarks compared to baseline systems
- The sharded-parallel execution engine delivers up to 7.6x speedup over sequential shell execution while preserving exact output equivalence
- Analysis reveals purely lexical DCI underperforms on queries with substantial surface-form variation, identifying a clear boundary condition for the approach
Signal Assessment
Methodology
- Generate a cold-start dataset of causally grounded search trajectories using an answer-aware Tutor (has access to the gold answer) paired with an answer-blind Planner (mimics realistic exploration)
- Initialize the search agent policy via supervised fine-tuning on these verified trajectories
- Refine the policy with Group Relative Policy Optimization (GRPO), letting the agent iteratively issue shell commands against the corpus and learn from task-oriented reward signals
- Deploy a sharded-parallel execution engine to run shell-based retrieval efficiently at scale while preserving byte-exact semantic equivalence with sequential execution
- Evaluate the trained agent across seven open-domain QA benchmarks using token-level F1 and Exact Match
System Components
A compact LLM policy trained to find, filter, and compose evidence by issuing executable shell commands directly against the text corpus, bypassing pre-built retrieval indices
A model with access to the gold answer used to help construct verified, causally grounded search trajectories for cold-start training
A model without access to the answer that generates realistic exploration/search paths, ensuring trajectories reflect genuine search behavior rather than answer leakage
Group Relative Policy Optimization used to refine the cold-started policy through direct, reward-driven interaction with the corpus, improving task-oriented search behavior
Infrastructure that parallelizes shell command execution across corpus shards, achieving up to 7.6x speedup while guaranteeing byte-exact equivalence with sequential execution
Results
| Metric/Benchmark | Baseline | This Paper (GrepSeek) | Delta |
|---|---|---|---|
| Token-level F1 (avg. over 7 open-domain QA benchmarks) | Retriever-based search agent baselines | Highest reported overall F1 | Best-in-class (qualitative) |
| Exact Match (avg. over 7 open-domain QA benchmarks) | Retriever-based search agent baselines | Highest reported overall EM | Best-in-class (qualitative) |
| Shell-based retrieval execution speed | Sequential shell execution | Sharded-parallel execution engine | Up to 7.6x speedup |
| Execution correctness under parallelization | Sequential execution semantics | Sharded-parallel engine output | Byte-exact equivalence preserved |
Key Takeaways
- Direct corpus interaction via shell commands (grep-style) is a practical, competitive alternative or complement to embedding-based retrieval for open-domain QA, especially for precise lexical matching tasks
- Cold-starting agent training with Tutor/Planner-generated causally grounded trajectories helps stabilize otherwise brittle RL training over large action spaces and corpora
- Systems-level engineering (parallel, correctness-preserving execution) is a critical but often underemphasized requirement for making raw-corpus agentic search deployable at scale
- Purely lexical retrieval strategies struggle with queries exhibiting significant surface-form variation, motivating hybrid designs that combine DCI with semantic retrieval
- Compact, efficiently trained agents can perform competitive multi-step evidence gathering directly over raw text without relying on large pre-built indices
Abstract
Large Language Model (LLM) search agents have shown strong promise for knowledge-intensive language tasks through multiple rounds of reasoning and information retrieval. Most existing systems access information using a retriever that takes a keyword or natural language query and returns a ranked list of documents using an index of pre-computed document representations. In this work, we explore a complementary perspective in which the search agent treats the corpus itself as the search environment and finds evidence by issuing executable shell commands. We introduce GrepSeek, an optimized direct corpus interaction (DCI) search agent that trains a compact search agent to find, filter, and compose evidence from large text corpora. To address the instability of learning behavior directly with reinforcement learning on large corpora, we propose a two-stage training pipeline. First, we construct a cold-start dataset using an answer-aware Tutor and answer-blind Planner to generate verified, causally grounded search trajectories. Second, we refine the initialized policy with Group Relative Policy Optimization (GRPO), allowing the agent to improve its task-oriented search behavior through direct interaction with the corpus. To make DCI practical at scale, we further use a semantics-preserving sharded-parallel execution engine that accelerates shell-based retrieval by up to $7.6\times$ while preserving byte-exact equivalence with sequential execution of the shell command. Experiments across seven open-domain question answering benchmarks show that GrepSeek achieves the strongest overall token-level $F_1$ and Exact Match. Our analysis also highlights the limitations of purely lexical interaction on queries with substantial surface-form variation, suggesting DCI as a practical and competitive method for search agents that can complement existing retrieval paradigms in the real world.