Investigating the Impact of Dark Patterns on LLM-Based Web Agents

This paper presents the first systematic study showing that LLM-based generalist web agents, not just human users, are susceptible to dark patterns—deceptive UI designs—achieved through two new tools: LiteAgent, a lightweight agent-execution and logging framework, and TrickyArena, a controlled benchmark of realistic websites with toggleable dark patterns.

Problem Statement

As LLM-based web agents take on autonomous tasks like shopping, subscriptions, and browsing on users' behalf, they inherit exposure to deceptive UI patterns that were originally designed to manipulate human cognition, yet no prior work had measured whether or how these agents fall for such manipulation. This is a critical safety and security gap because agents acting without human oversight could be tricked into costly or privacy-harming decisions (unwanted purchases, subscriptions, data disclosures) at scale, and existing agent evaluations largely ignore adversarial or manipulative web content.

Key Novelty

First empirical investigation of dark pattern susceptibility specifically for LLM-based generalist web agents (as opposed to human users)
LiteAgent: a lightweight, agent-agnostic framework that automates task execution while capturing detailed interaction logs and screen recordings for behavioral analysis
TrickyArena: a controlled, realistic multi-domain (e-commerce, streaming, news) testbed with diverse dark patterns that can be selectively enabled/disabled for causal experimentation
Systematic analysis of how dark pattern intensity (single vs. combined) and implementation details (visual design vs. HTML structure) affect agent susceptibility

Evaluation Highlights

Agents were susceptible to a single dark pattern an average of 41% of the time when it was present
Susceptibility evaluated across 6 popular LLM-based generalist web agents and 3 underlying LLMs, with variation observed across agents/models
Modifying dark pattern UI attributes (visual design or HTML code) and stacking multiple dark patterns simultaneously measurably shifted agent susceptibility

Signal Assessment

6/10 The paper does not introduce a new model or defense mechanism, but it is a first-of-its-kind, rigorously designed measurement study revealing a previously unexplored and practically significant vulnerability in agentic AI, backed by reusable open infrastructure (LiteAgent, TrickyArena).

Methodology

Construct TrickyArena: realistic web applications across e-commerce, streaming, and news domains embedding a diverse taxonomy of dark patterns that can be toggled on/off for controlled comparison
Build LiteAgent: a harness that automatically issues tasks to various LLM-based web agents and records fine-grained logs plus screen recordings of agent-UI interactions
Run each of 6 web agents (across 3 LLMs) on identical tasks with dark patterns enabled vs. disabled to isolate causal impact on agent decisions
Extend experiments to combined/multiple concurrent dark patterns and to variants of a single dark pattern (altered visual styling or HTML markup) to test sensitivity
Analyze logs/recordings to quantify susceptibility rates and characterize failure modes across agents and LLM backbones

System Components

LiteAgent

Lightweight, model/agent-agnostic framework that automatically prompts web agents to execute tasks while capturing comprehensive interaction logs and screen recordings for post-hoc analysis

TrickyArena

Controlled benchmark environment of realistic web apps (e-commerce, streaming, news) with a curated set of dark patterns that can be selectively enabled/disabled to isolate their effect on agent behavior

Dark Pattern Variants

Systematically modified versions of dark patterns (via visual/design changes or underlying HTML code) used to test whether implementation details affect agent susceptibility

Agent/LLM Test Matrix

Evaluation harness covering 6 popular generalist web agents paired with 3 different LLM backbones to assess generality of susceptibility findings

Results

Condition	Metric	Result	Notes
Single dark pattern present	Agent susceptibility rate	41% (average)	Across 6 agents and 3 LLMs
Multiple/combined dark patterns	Susceptibility change	Influenced (increased/varied)	Qualitative finding, magnitude not specified in abstract
Altered UI/HTML attributes of a dark pattern	Susceptibility change	Influenced	Shows sensitivity to implementation, not just presence, of dark pattern

Key Takeaways

LLM-based web agents cannot be assumed immune to human-targeted manipulation—dark patterns fool them roughly 4 in 10 times, a real risk for any deployment where agents transact or share data autonomously
Susceptibility is not fixed: subtle visual or code-level changes to a dark pattern, or stacking several at once, can shift how often agents are tricked, implying adversaries could optimize dark patterns specifically against agents
Robust agent deployment requires layered defenses—agent-side detection/refusal mechanisms plus broader web-level safeguards (e.g., standards or regulation on deceptive design)—rather than relying on the underlying LLM's general safety training
LiteAgent and TrickyArena offer reusable, extensible infrastructure for benchmarking and stress-testing agent robustness against manipulative or adversarial web content in future research

Abstract

As users increasingly turn to large language model (LLM) based web agents to automate online tasks, agents may encounter dark patterns: deceptive user interface designs that manipulate users into making unintended decisions. Although dark patterns primarily target human users, their potentially harmful impacts on LLM-based generalist web agents remain unexplored. In this paper, we present the first study that investigates the impact of dark patterns on the decisionmaking process of LLM-based generalist web agents. To achieve this, we introduce LiteAgent, a lightweight framework that automatically prompts agents to execute tasks while capturing comprehensive logs and screen-recordings of their interactions. We also present TrickyArena, a controlled environment comprising web applications from domains such as e-commerce, streaming services, and news platforms, each containing diverse and realistic dark patterns that can be selectively enabled or disabled. Using LiteAgent and TrickyArena, we conduct multiple experiments to assess the impact of both individual and combined dark patterns on web agent behavior. We evaluate six popular LLM-based generalist web agents across three LLMs and discover that when there is a single dark pattern present, agents are susceptible to it an average of 41 % of the time. We also find that modifying dark pattern UI attributes through visual design changes or HTML code adjustments and introducing multiple dark patterns simultaneously can influence agent susceptibility. This study emphasizes the need for holistic defense mechanisms in web agents, encompassing both agent-specific protections and broader web safety measures.