Coding Agents are Effective Long-Context Processors
Problem Statement
Even as context windows grow, LLMs process long inputs through opaque attention and degrade sharply as context length increases. Current alternatives such as semantic retrieval (RAG) or further scaling of the context window are either passive similarity lookups or costly and still unreliable. This matters most for corpus-scale tasks reaching trillions of tokens, where neither approach scales gracefully.
Key Novelty
- Reframes long-context processing as an agentic, tool-mediated task: text is placed in a file system and handled with native coding-agent tools (shell commands, grep-like search, scripts) rather than fed through attention.
- Shows that general-purpose frontier coding agents, used without task-specific training, serve as a single interface for long-context reasoning, RAG, and open-domain QA over corpora of up to ~3 trillion tokens.
- Attributes the gains to two factors: native tool proficiency (executable code and terminal commands instead of passive semantic queries) and file-system familiarity (navigating corpora as directory structures).
Evaluation Highlights
- Outperforms published state-of-the-art by 17.3% on average across multiple long-context benchmarks.
- Remains effective at extreme scale, performing open-domain QA over corpora of up to three trillion tokens, well beyond any context window.
Signal Assessment
Methodology
- Convert long contexts or large corpora into a file-system representation (files and directories) that the agent can access.
- Deploy off-the-shelf frontier coding agents with their native toolset (terminal commands, code execution, file navigation) and task instructions.
- Let the agent iteratively explore, search, filter, and compute over the text with executable actions, gathering evidence before producing an answer.
- Evaluate on long-context reasoning, RAG, and open-domain QA benchmarks against published SOTA, and analyze which factors (tool proficiency, file-system familiarity) drive performance.
System Components
Externalizes long text or corpora into files and directories so content can be browsed structurally instead of loaded into the context window.
A frontier LLM coding agent that plans and executes multi-step interactions with the environment; used without fine-tuning.
Terminal commands and executable code (e.g., grep/find-style search, scripts) that allow precise, programmatic retrieval and computation over text, as opposed to embedding-based semantic queries.
Agentic cycle of searching, inspecting, and refining evidence across the corpus before synthesizing a final answer.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Average across benchmarks | Published SOTA | Coding agents | +17.3% avg |
| Long-context reasoning | Long-context LLMs / prior SOTA | Coding agent over file system | Outperforms (specific numbers not in abstract) |
| Retrieval-augmented generation | Semantic-search RAG SOTA | Coding agent with native tools | Outperforms (specific numbers not in abstract) |
| Open-domain QA (up to ~3T-token corpus) | Prior retrieval-based SOTA | Coding agent navigating corpus as directories | Outperforms; scales to trillions of tokens |
Key Takeaways
- Before scaling context windows or building complex RAG pipelines, try dumping documents into a file system and letting a capable coding agent explore them. It may be a strong, low-engineering baseline.
- Programmatic search (regex, grep, scripts) combined with agentic iteration can beat embedding-based semantic retrieval, especially for precise or compositional queries. Hybrid systems should expose such tools to agents.
- Structure matters: organizing corpora into meaningful directory hierarchies uses agents' existing familiarity with file systems. Watch for the cost, latency, and tool-call overhead of agentic exploration compared with single-pass inference.
Abstract
Large Language Models (LLMs) have demonstrated remarkable progress in scaling to access massive contexts. However, the access is via the latent and uninterpretable attention mechanisms, and LLMs fail to effective process long context, exhibiting significant performance degradation as context length increases. In this work, we study whether long-context processing can be externalized from latent attention into explicit, executable interactions, by allowing coding agents to organize text in file systems and manipulate it using its native tools. We evaluate off-the-shelf frontier coding agents as the general interface for tasks that require processing long contexts, including long-context reasoning, retrieval-augmented generation, and open-domain question answering with large-scale corpus contains up to three trillion tokens. Across multiple benchmarks, these agents outperform published state-of-the-art by 17.3% on average. We attribute this efficacy to two key factors: native tool proficiency, which enables agents to leverage executable code and terminal commands rather than passive semantic queries, and file system familiarity, which allows them to navigate massive text corpora as directory structures. These findings suggest that delegating long-context processing to coding agents offers an effective alternative to semantic search or context window scaling, opening new directions for long-context processing in LLMs.