Dynamic Tool Dependency Retrieval for Efficient Function Calling
Problem Statement
On-device LLM agents have limited context and compute, so they rely on retrieval modules to pick a small subset of relevant tools from a large catalog. Existing retrievers run once on the initial query and ignore multi-step tool dependencies and how the task context changes during execution. As a result they often surface irrelevant tools that mislead the agent, lowering accuracy and wasting context and compute.
Key Novelty
- Dynamic retrieval formulation that re-retrieves tools at each step, conditioning on both the initial query and the evolving execution context (prior calls and plan state) rather than on the query alone.
- Explicit modeling of tool dependencies (which tools tend to follow or co-occur with others) learned from function-calling demonstrations, so retrieval adapts as plans unfold.
- A lightweight design suited to on-device deployment, plus a study of strategies for integrating retrieved tools into the agent prompt.
Evaluation Highlights
- Function-calling success rates improve by 23% to 104% (relative) over state-of-the-art static retrievers.
- Evaluated across multiple datasets and LLM backbones on retrieval precision, downstream task accuracy and computational efficiency, with additional analysis of prompt-integration strategies.
Signal Assessment
Methodology
- Collect function-calling demonstrations (query plus sequences of tool calls) and extract tool dependency structure, i.e., which tools follow or condition on others.
- Train a lightweight retriever that scores candidate tools using both the initial user query and the current execution context (previously invoked tools and their outputs or plan state).
- At each agent step, dynamically retrieve a small, context-relevant tool subset and inject it into the LLM prompt using one of several explored integration strategies.
- Benchmark against static state-of-the-art retrievers across datasets and LLM backbones, measuring retrieval precision, end-task success and compute and context efficiency.
System Components
Captures sequential and conditional relationships between tools learned from demonstrations, so likely next tools can be predicted from tools already used.
Lightweight scorer that combines the initial query with the evolving execution context to rank tools at each step.
Re-invokes retrieval as the plan unfolds, updating the candidate tool set rather than fixing it up front.
Mechanisms for inserting retrieved tool descriptions into the LLM prompt, studied for their effect on accuracy and context length.
The agent LLM, potentially a small on-device model, that selects and parameterizes tool calls from the retrieved subset.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Function-calling success rate (across datasets/backbones) | Static SOTA retrievers | DTDR (dynamic) | +23% to +104% relative |
| Retrieval precision | Static query-only retrieval | Context- and dependency-aware retrieval | Improved (qualitative) |
| Computational efficiency / context length | Static retrieval, more irrelevant tools in context | Lightweight dynamic retrieval with fewer irrelevant tools | Reported favorable (qualitative) |
Key Takeaways
- For multi-step agents, retrieve tools per step using execution history rather than once from the query. Static retrieval is a major source of agent error.
- Logged function-calling traces are a valuable supervision signal: mining tool-to-tool dependencies from them is a cheap way to boost retrieval quality.
- On-device and small-model agents benefit most, since precise, compact tool sets reduce both context cost and distraction. How retrieved tools are placed in the prompt also matters and should be tuned.
Abstract
Function calling agents powered by Large Language Models (LLMs) select external tools to automate complex tasks. On-device agents typically use a retrieval module to select relevant tools, improving performance and reducing context length. However, existing retrieval methods rely on static and limited inputs, failing to capture multi-step tool dependencies and evolving task context. This limitation often introduces irrelevant tools that mislead the agent, degrading efficiency and accuracy. We propose Dynamic Tool Dependency Retrieval (DTDR), a lightweight retrieval method that conditions on both the initial query and the evolving execution context. DTDR models tool dependencies from function calling demonstrations, enabling adaptive retrieval as plans unfold. We benchmark DTDR against state-of-the-art retrieval methods across multiple datasets and LLM backbones, evaluating retrieval precision, downstream task accuracy, and computational efficiency. Additionally, we explore strategies to integrate retrieved tools into prompts. Our results show that dynamic tool retrieval improves function calling success rates between $23\%$ and $104\%$ compared to state-of-the-art static retrievers.