Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models
Problem Statement
As agentic AI systems increasingly rely on function/tool calling (e.g., via protocols like MCP) to extend LLM capabilities, they expose a new attack surface beyond traditional prompt injection and jailbreaking. Prior function-calling attacks depend heavily on semantic manipulation of prompts or function descriptions, limiting their generality; this work shows that tool selection itself can be hijacked in a way that is largely domain- and semantics-agnostic, revealing a deeper and more transferable vulnerability in agentic pipelines.
Key Novelty
- Introduces Function Hijacking Attack (FHA), targeting the tool-selection decision process rather than relying on semantic content manipulation used by prior injection/jailbreak-style attacks
- Demonstrates the attack is largely agnostic to context semantics and robust across different function sets, enabling cross-domain applicability
- Shows FHA can be trained into 'universal adversarial functions' that hijack tool selection across multiple distinct queries and payload configurations with a single attacked function
- Provides one of the first systematic evaluations of function-calling attacks explicitly framed around MCP-style agentic architectures
Evaluation Highlights
- Achieved 70%-100% Attack Success Rate (ASR) across 5 different LLMs on the Berkeley Function-Calling Leaderboard (BFCL) dataset
- Effectiveness demonstrated on both instruction-tuned and reasoning model variants, indicating the vulnerability is not mitigated by chain-of-thought reasoning
- Universal adversarial functions shown to generalize across multiple queries and payload configurations, not just a single crafted prompt/function pair
Signal Assessment
Methodology
- Characterize the tool/function selection decision process in agentic, function-calling LLMs to identify manipulable decision surfaces
- Design the Function Hijacking Attack (FHA) to craft an attacker-controlled function definition that biases selection toward it regardless of query semantics or competing function set composition
- Develop a training procedure to optimize 'universal adversarial functions' that transfer across diverse queries and payload configurations rather than being query-specific
- Evaluate attack success rate (ASR) systematically across the BFCL dataset and 5 LLMs (instructed and reasoning variants) to test generality and robustness
- Assess robustness against varying function sets and domains to confirm the context-agnostic nature of the attack
System Components
Core attack that manipulates an agentic model's tool-selection process to force invocation of a specific attacker-chosen function, largely independent of query semantics
A trained/optimized function specification that generalizes across multiple queries and payload configurations, enabling one crafted function to hijack tool selection broadly
The underlying targeted process within the model's function-calling pipeline exploited to bias function choice rather than exploiting semantic relevance
Experimental framework built on the Berkeley Function-Calling Leaderboard dataset used to measure ASR across models, function sets, and domains
Results
| Metric/Benchmark | Existing/Semantic-based Attacks | FHA (This Paper) | Delta |
|---|---|---|---|
| Attack Success Rate on BFCL (5 models) | Domain/semantic-dependent, variable success | 70%-100% ASR | Consistently high across models and settings |
| Generalization across function sets/domains | Limited by semantic alignment | Robust and largely context-agnostic | Broader cross-domain applicability |
| Cross-query/payload transferability | Not established as a capability | Universal adversarial functions transfer across queries and payloads | New attack capability introduced |
| Robustness to model reasoning ability | Not systematically tested | Effective on both instructed and reasoning variants | Vulnerability persists despite reasoning |
Key Takeaways
- Function-calling and MCP-based agentic pipelines have an underexplored attack surface at the tool-selection level, distinct from and potentially more dangerous than prompt-based jailbreaking or injection
- Guardrails and security modules should validate the tool-selection decision logic itself, not just the semantic content of function descriptions or user prompts
- Reasoning-capable models are not inherently robust to function hijacking, so 'reasoning as a safety mechanism' assumptions should be reconsidered for agentic deployments
- Because a single universal adversarial function can compromise many queries, marketplaces and registries for third-party tools/functions (e.g., in MCP ecosystems) need rigorous vetting and runtime monitoring
- Practitioners deploying agentic LLMs should incorporate function-calling-specific red-teaming and ASR benchmarking (e.g., via BFCL-style datasets) into their security evaluation pipelines
Abstract
The growth of agentic AI has drawn significant attention to function calling Large Language Models (LLMs), which are designed to extend the capabilities of AI-powered system by invoking external functions. Injection and jailbreaking attacks have been extensively explored to showcase the vulnerabilities of LLMs to user prompt manipulation. The expanded capabilities of agentic models introduce further vulnerabilities via their function calling interface. Recent work in LLM security showed that function calling can be abused, leading to data tampering and theft, causing disruptive behavior such as endless loops, or causing LLMs to produce harmful content in the style of jailbreaking attacks. This paper introduces a novel function hijacking attack (FHA) that manipulates the tool selection process of agentic models to force the invocation of a specific, attacker-chosen function. While existing attacks focus on semantic preference of the model for function-calling tasks, we show that FHA is largely agnostic to the context semantics and robust to the function sets, making it applicable across diverse domains. We further demonstrate that FHA can be trained to produce universal adversarial functions, enabling a single attacked function to hijack tool selection across multiple queries and payload configurations. We conducted experiments on 5 different models, including instructed and reasoning variants, reaching 70% to 100% ASR over the established BFCL dataset. Our findings further demonstrate the need for strong guardrails and security modules for agentic systems.