Unified Tool Integration for LLMs: A Protocol-Agnostic Approach to Function Calling
Problem Statement
Tool-augmented LLM development is fragmented across multiple protocols and vendor-specific function-calling formats. Developers must write JSON schemas by hand, wire up execution loops, and reconcile tools from different sources. This boilerplate slows development, invites schema and implementation mismatches, and often leaves concurrency unexploited when an LLM issues multiple tool calls.
Key Novelty
- A protocol-agnostic tool integration architecture that hides differences between tool sources and protocols behind a single interface
- Automated schema generation that derives function-calling schemas directly from code, eliminating manual JSON schema authoring
- A dual-mode concurrent execution engine, likely combining async I/O-bound and thread or process-based execution, for running multiple tool calls efficiently
- Seamless multi-source tool management that lets local functions and external protocol-based tools coexist in one registry
Evaluation Highlights
- 60-80% reduction in integration code across multiple tool-integration scenarios compared to conventional approaches
- Up to 3.1x performance improvement from optimized concurrent tool execution
- Full compatibility with existing function-calling standards (e.g., OpenAI-style function calling)
Signal Assessment
Methodology
- Analyze the existing tool-integration landscape (vendor function-calling APIs, tool protocols, manual workflows) to identify common abstractions and pain points
- Design a protocol-agnostic abstraction layer with adapters that normalize tools from heterogeneous sources into a unified internal representation
- Automatically generate function-calling schemas from code signatures, type hints and docstrings
- Implement a dual-mode concurrent executor that dispatches LLM-issued tool calls in parallel according to workload characteristics
- Evaluate against conventional integration code on lines-of-code reduction, execution latency or throughput, and standards compatibility
System Components
Unifies different tool protocols and function-calling formats behind a single interface so application code is independent of the tool source
Introspects functions (signatures, types, docs) to produce standard-compliant tool schemas without manual JSON definitions
Runs multiple tool calls concurrently using two execution modes (likely async and thread-based) to optimize latency for I/O- and compute-bound tools
Registry that aggregates and manages tools from local code and remote or protocol-based servers, handling discovery and routing
Ensures generated schemas and call and response formats remain fully compatible with existing LLM function-calling APIs
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Integration code size | Manual schema + execution code (conventional) | Unified framework | 60-80% reduction |
| Tool execution performance | Sequential/non-optimized execution | Dual-mode concurrent execution | Up to 3.1x speedup |
| Function-calling standards compatibility | Protocol-specific implementations | Protocol-agnostic layer | Full compatibility maintained |
| Multi-source tool support | Separate per-protocol handling | Single unified manager | Qualitative: seamless integration |
Key Takeaways
- Generate tool schemas from code (type hints and docstrings) rather than handwriting JSON. This substantially cuts boilerplate and avoids schema drift.
- When LLMs emit multiple tool calls in a turn, execute them concurrently. Speedups of up to 3x are achievable in tool-heavy agent workflows, especially for I/O-bound calls.
- Put an adapter layer between agent logic and tool protocols (MCP, vendor function calling, custom APIs) to reduce lock-in and simplify mixing tool sources.
- Treat the reported gains as engineering benchmarks rather than agent-quality results. Validate against your own workloads, since code-reduction and speedup figures depend heavily on the chosen baselines.
Abstract
The proliferation of tool-augmented Large Language Models (LLMs) has created a fragmented ecosystem where developers must navigate multiple protocols, manual schema definitions, and complex execution workflows. We address this challenge by proposing a unified approach to tool integration that abstracts protocol differences while optimizing execution performance. Our solution demonstrates how protocol-agnostic design principles can significantly reduce development overhead through automated schema generation, dual-mode concurrent execution, and seamless multi-source tool management. Experimental results show 60-80% code reduction across integration scenarios, performance improvements up to 3.1x through optimized concurrency, and full compatibility with existing function calling standards. This work contributes both theoretical insights into tool integration architecture and practical solutions for real-world LLM application development.