AutoTool: Efficient Tool Selection for Large Language Model Agents
Problem Statement
ReAct-style agents call the LLM at every step to decide which tool to use next. This makes inference cost and latency high and scales poorly for long-horizon or high-volume deployments. Much of that reasoning is redundant when tool sequences are routine, yet current frameworks have no way to reuse statistical regularities from past executions.
Key Novelty
- Identifies and empirically characterizes 'tool usage inertia', the tendency of agent tool invocations to follow predictable sequential patterns, as an exploitable property.
- Proposes a directed tool-transition graph built from historical agent trajectories. Nodes are tools and edges are transition probabilities, which turns tool selection into lightweight graph traversal instead of LLM inference.
- Adds parameter-level information to the graph so that tool inputs, not just tool identity, can be generated or refined with minimal LLM involvement.
Evaluation Highlights
- Reduces inference cost by up to 30% relative to inference-heavy frameworks such as ReAct.
- Keeps task completion rates competitive with the baselines across diverse agent tasks. The abstract gives no exact benchmarks or numbers.
Signal Assessment
Methodology
- Collect historical agent trajectories (sequences of tool calls with their parameters) from prior executions of an LLM agent framework such as ReAct.
- Build a directed graph in which nodes are tools and weighted edges are empirical transition probabilities between consecutive tool calls, capturing tool usage inertia.
- Attach parameter-level information (for example, how outputs of earlier tools map to inputs of later tools) to support tool input generation.
- At inference time, traverse the graph to predict the next tool and its parameters, and fall back to LLM inference when the graph lacks a confident prediction.
- Evaluate on diverse agent tasks, comparing inference cost (LLM calls/tokens) and task completion rate against standard LLM-driven tool selection.
System Components
Empirical study showing that tool call sequences in agent trajectories are highly predictable, which motivates replacing some LLM decisions with statistical predictions.
A directed graph built from historical trajectories, with tools as nodes and transition probabilities as edges, used to predict the next likely tool.
Augments the graph with parameter information (for example, parameter dependencies or flows between tools) to fill in or refine tool arguments without full LLM generation.
An inference-time module that walks the graph to pick tools and parameters efficiently, using the LLM only when needed.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Inference cost (LLM calls/tokens) | ReAct-style per-step LLM tool selection | AutoTool graph-based selection | Up to 30% reduction |
| Task completion rate | ReAct-style baseline performance | Competitive with baseline | Roughly maintained (small or no degradation) |
| Reliance on LLM for tool selection | LLM invoked every step | LLM invoked only selectively | Substantially reduced |
Key Takeaways
- If your agent runs repetitive workflows, log its tool trajectories. A simple transition model over those logs can cut LLM calls and cost with little loss in accuracy.
- Hybrid designs work well: use cheap statistical predictors for routine, high-confidence tool choices and keep the LLM for novel or ambiguous decisions.
- Gains depend on how regular the task distribution is. Expect weaker benefits on highly diverse or open-ended tasks, and plan for cold-start (no history) and distribution shift when deploying.
Abstract
Large Language Model (LLM) agents have emerged as powerful tools for automating complex tasks by leveraging the reasoning and decision-making abilities of LLMs. However, a major bottleneck in current agent frameworks lies in the high inference cost of tool selection, especially in approaches like ReAct that repeatedly invoke the LLM to determine which tool to use at each step. In this work, we propose AutoTool, a novel graph-based framework that bypasses repeated LLM inference by exploiting a key empirical observation: tool usage inertia—the tendency of tool invocations to follow predictable sequential patterns. AutoTool constructs a directed graph from historical agent trajectories, where nodes represent tools and edges capture transition probabilities, effectively modeling the inertia in tool selection. It further integrates parameter-level information to refine tool input generation. By traversing this structured representation, AutoTool efficiently selects tools and their parameters with minimal reliance on LLM inference. Extensive experiments across diverse agent tasks demonstrate that AutoTool reduces inference costs by up to 30% while maintaining competitive task completion rates, offering a practical and scalable enhancement for inference-heavy frameworks. Our work highlights the promise of integrating statistical structure into LLM agent design for greater efficiency without sacrificing performance.