ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering
Problem Statement
Real-world toolsets (API hubs, MCP servers, enterprise tool registries) often contain tools with near-duplicate names and descriptions. This ambiguity confuses LLMs during tool selection. Large toolsets also exceed or crowd the context window, so not every tool can be presented, and simply truncating the list loses relevant tools. Existing tool-retrieval work largely ignores redundancy in the toolset itself.
Key Novelty
- ToolScopeMerger: automatically detects and consolidates semantically redundant tools, framing toolset de-duplication as a preprocessing step for agent tool use.
- Auto-Correction mechanism: an audit loop that verifies proposed tool merges and repairs incorrect ones, reducing the risk of collapsing tools that are functionally distinct.
- ToolScopeRetriever: a context-aware ranking and filtering module that selects a compact, query-relevant subset of tools to fit context budgets without sacrificing selection accuracy.
Evaluation Highlights
- Tool selection accuracy improves by 8.38% to 38.6% over baselines.
- Gains hold across three state-of-the-art LLMs and three open-source tool-use benchmarks, suggesting the approach generalizes across models and datasets.
Signal Assessment
Methodology
- Analyze the toolset offline to identify redundant tools with overlapping names, descriptions and functionality, likely through semantic similarity or clustering.
- Merge the redundant tools into consolidated entries, then run Auto-Correction to audit each merge and fix incorrect consolidations.
- At inference time, use ToolScopeRetriever to score and rank the merged tools against the user query and keep only the top relevant subset.
- Pass the filtered, de-duplicated toolset to the LLM agent for tool selection, then evaluate selection accuracy across multiple LLMs and benchmarks.
System Components
Detects semantically overlapping or redundant tools and consolidates them, which reduces ambiguity and toolset size.
An audit step, likely LLM-driven, that checks each proposed merge for correctness and fixes or reverts erroneous merges.
A query-conditioned ranking and filtering module that compresses the toolset to the most relevant tools so it fits the context window.
The downstream LLM that selects and invokes a tool from the reduced, disambiguated candidate set.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Tool selection accuracy (3 open-source benchmarks) | Raw toolset / standard selection | ToolScope (Merger + Retriever) | +8.38% to +38.6% |
| Generalization across LLMs (3 SOTA models) | Per-model baseline | Consistent improvement | Positive across all models (per abstract) |
| Context efficiency | Full or truncated toolset in prompt | Compressed, query-relevant subset | Toolset fits context limits without accuracy loss (qualitative) |
Key Takeaways
- Audit and de-duplicate your tool registry before deployment. Overlapping tool names and descriptions are a measurable source of selection errors, not just cosmetic clutter.
- Automated merging needs a verification step. An LLM-based audit and correction pass helps avoid merging tools whose functionality only looks similar on the surface.
- For agents with large toolsets (MCP servers, API marketplaces), retrieve query-relevant tools rather than including all of them. This saves context and can improve accuracy.
- Treat merging and retrieval as complementary. De-duplication makes retrieval cleaner, and retrieval makes large deduplicated sets tractable, so the two together likely matter more than either alone.
Abstract
Large language model (LLM) agents rely on external tools to solve complex tasks, but real-world toolsets often contain redundant tools with overlapping names and descriptions, introducing ambiguity and reducing selection accuracy. LLMs also face strict input context limits, preventing efficient consideration of large toolsets. To address these challenges, we propose ToolScope, which includes: (1) ToolScopeMerger with Auto-Correction to automatically audit and fix tool merges, reducing redundancy, and (2) ToolScopeRetriever to rank and select only the most relevant tools for each query, compressing toolsets to fit within context limits without sacrificing accuracy. Evaluations on three state-of-the-art LLMs and three open-source tool-use benchmarks show gains of 8.38% to 38.6% in tool selection accuracy, demonstrating ToolScope's effectiveness in enhancing LLM tool use.