Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation

The paper presents a systematic study of how experiential knowledge affects LLM tool-calling performance across acquisition, activation, and internalization stages, culminating in KATE—a framework combining instance-level experiential knowledge, width-expanded parallel reasoning, and RL-based knowledge-aware training.

Problem Statement

LLM-based agents depend on reliable multi-step tool use, but frequently fail due to insufficient tool-related knowledge and ineffective activation of that knowledge during reasoning, limiting their real-world deployment as autonomous agents. Prior work typically addresses isolated aspects (e.g., prompting or fine-tuning alone) rather than treating knowledge acquisition, activation, and internalization as an integrated pipeline. This matters because it offers an empirically-grounded, end-to-end approach for building more robust, knowledge-aware tool-use agents.

Key Novelty

A systematic three-stage study (acquisition, activation, internalization) isolating how experiential knowledge affects tool-use performance
Empirical finding that simple instance-level knowledge outperforms abstract intent-level knowledge, challenging assumptions about knowledge abstraction benefits
Discovery that width-expanded reasoning (parallel sampling with aggregation) activates latent knowledge more effectively than depth-expanded reasoning (longer chain-of-thought)
Evidence that RL-based post-training internalizes tool-use knowledge better than supervised fine-tuning
KATE: a unified framework integrating experiential knowledge, width-expanded inference, and knowledge-aware training

Evaluation Highlights

Evaluated on BFCL-V3 and AppWorld, two established tool-use/agentic benchmarks
Reports consistent and substantial improvements over strong baselines across multiple model scales

Signal Assessment

5/10 The paper delivers valuable, well-validated empirical insights (width vs. depth reasoning, RL vs. SFT for knowledge internalization) and a practical unifying framework, but largely recombines and systematically studies existing techniques rather than introducing a fundamentally new paradigm.

Methodology

Study knowledge acquisition by collecting and comparing instance-level vs. abstract intent-level experiential knowledge for tool use
Study knowledge activation at inference time by comparing reasoning-depth expansion (extended CoT) against reasoning-width expansion (parallel sampling with aggregation)
Study knowledge internalization at training time by comparing SFT and RL post-training on knowledge-augmented data
Integrate best-performing techniques from each stage into the unified KATE framework
Evaluate KATE against strong baselines on BFCL-V3 and AppWorld across different model scales

System Components

Experiential Knowledge Acquisition

Collects instance-level experiential knowledge (e.g., past tool-use trajectories) shown to yield stronger and more reliable gains than abstract intent-level knowledge

Reasoning-Width Expansion

Inference-time mechanism using parallel sampling with aggregation to better activate latent experiential knowledge, outperforming simple reasoning-depth extension

Knowledge-Aware Training

Post-training procedure using knowledge-augmented data, with reinforcement learning shown to internalize knowledge more effectively than supervised fine-tuning

KATE Framework

Unified pipeline integrating experiential knowledge, width-expanded reasoning, and knowledge-aware training into a complete tool-execution system

Results

Benchmark	Baseline	KATE (This Paper)	Delta
BFCL-V3 (tool-calling accuracy)	Strong existing baselines	KATE framework	Consistent, substantial improvement across model scales
AppWorld (agentic task success)	Strong existing baselines	KATE framework	Consistent, substantial improvement across model scales

Key Takeaways

Favor simple, instance-level experiential knowledge over costly abstract knowledge distillation when augmenting tool-use agents
Allocate inference-time compute to parallel sampling with aggregation (width) rather than longer chain-of-thought (depth) to better surface latent tool-use knowledge
When post-training on knowledge-augmented data, prefer reinforcement learning over supervised fine-tuning for more effective knowledge internalization
A staged framework (acquire → activate → internalize) offers a practical template for building more reliable knowledge-augmented tool-calling agents

Abstract

Large language models (LLMs) rely on tool use to act as autonomous agents, yet often fail in multi-step execution due to insufficient tool-related knowledge and ineffective knowledge activation. Therefore, we present a systematic study on how knowledge influences tool-use performance, covering the stages of knowledge acquisition, activation, and internalization. In the knowledge acquisition stage, we acquire and evaluate various forms of experiential knowledge, and our analysis shows that simple instance-level knowledge can already provide strong and reliable gains, while abstract intent-level knowledge offers limited benefits. At inference time, to activate knowledge, we find that prompting LLM to expand the depth of reasoning yields diminishing returns, whereas expanding the width of reasoning by parallel sampling with aggregation more effectively activates latent experiential knowledge. At training time, for knowledge internalization, post-training with knowledge-augmented data further improves performance, with reinforcement learning outperforming supervised fine-tuning. Based on these insights, we propose the Knowledge-Augmented Tool Execution (KATE), a knowledge-augmented tool execution framework that integrates experiential knowledge with reasoning-width-expanded inference and knowledge-aware training. Experiments on BFCL-V3 and AppWorld demonstrate consistent and substantial improvements over strong baselines across model scales. Our Code is available at https://github.com/hypasd-art/KATE.