SoftSkill: Behavioral Compression for Contextual Adaptation

SoftSkill converts a natural-language agent skill file (Markdown) into a short, trainable continuous prefix—initialized from the text and refined by a 'soft delta'—that serves as a latent behavioral prior for a frozen LLM, replacing repeated at-inference re-reading of long textual instructions.

Problem Statement

Agent skills are usually stored as long Markdown files describing answer policies and procedures, which are human-readable but must be re-parsed by a frozen model on every single task instance, incurring token cost and inconsistent behavioral adherence. This work addresses the inefficiency and unreliability of using raw text as the interface for injecting learned behavior into frozen LLMs, proposing a compact latent alternative instead.

Key Novelty

Reframes 'skills' as compact continuous context objects (soft prefixes) initialized from natural-language Markdown rather than as text to be reinterpreted at every inference call
Introduces a frozen-backbone training recipe that tunes only a trainable 'soft delta' on top of the text-initialized prefix via next-token prediction, avoiding any base-model fine-tuning
Provides a direct empirical comparison against SkillOpt (a prior skill-optimization baseline) and no-skill prompting, showing the latent approach both improves accuracy and drastically reduces token overhead
Explicitly tests and reports a negative/boundary result for long-horizon agentic trajectory imitation, delineating where behavioral compression currently fails to generalize

Evaluation Highlights

Length-32 SoftSkill prefix on Qwen3.5-4B improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA
Relative to SkillOpt, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath
Achieves these gains while replacing hundreds to thousands of Markdown skill tokens with only a few dozen virtual tokens
Agentic execution (long-horizon trajectory imitation) shows only partial benefit, identified as a harder boundary case where compression is not yet robust

Signal Assessment

6/10 The paper builds on established soft/prefix-prompt tuning techniques but applies them in a novel and practically compelling way to compress agent skill files into latent priors, showing strong single-round gains and honestly reporting where the idea breaks down for agentic tasks—making it a solid, well-scoped contribution rather than a paradigm shift.

Methodology

Start from an existing natural-language Markdown skill artifact encoding answer policies, evidence-use habits, and task procedures
Initialize a fixed-length continuous context object (e.g., length-32 soft prefix) derived from this textual skill description
Train a lightweight 'soft delta' on top of this initialization using standard next-token prediction loss while keeping the backbone LLM (Qwen3.5-4B) entirely frozen
Deploy the resulting SoftSkill vector as a virtual-token prefix at inference time in place of the original Markdown text, across single-round QA/math/vision-language tasks
Extend the same recipe to multi-step agentic settings via sparse trajectory imitation to probe compression of long-horizon procedural behavior

System Components

Frozen backbone (Qwen3.5-4B)

The base language model remains unchanged throughout training and inference; only the soft skill object is optimized

SoftSkill prefix

A short (length-32) continuous context object that replaces the Markdown skill text as the behavioral interface consumed by the model

Soft delta training

A trainable offset applied to the text-derived initialization, optimized with next-token prediction to sharpen the skill into a latent behavioral prior

Baseline comparators

No-skill prompting and SkillOpt (prior skill-optimization method) used to benchmark accuracy and token efficiency gains

Agentic trajectory imitation module

Extension of the training signal to sparse multi-step trajectories, used to test compression of long-horizon agentic procedures as a boundary case

Results

Benchmark	No-Skill Baseline	SoftSkill	Delta vs No-Skill / SkillOpt
SearchQA	No-skill prompting	SoftSkill prefix (len-32)	+8.3 pts vs no-skill; +5.2 pts vs SkillOpt
LiveMath	No-skill prompting	SoftSkill prefix (len-32)	+42.1 pts vs no-skill; +12.5 pts vs SkillOpt
DocVQA	No-skill prompting	SoftSkill prefix (len-32)	+1.3 pts vs no-skill
Skill token footprint	Hundreds to thousands of Markdown tokens	~32 virtual tokens	Large reduction in inference-time token cost
Agentic long-horizon execution	Trajectory imitation baseline	SoftSkill (boundary case)	Partial signal only; not yet robust for long-horizon procedures

Key Takeaways

Long Markdown skill files can potentially be replaced by a small number of trainable virtual tokens, cutting prompt length/cost while improving accuracy on single-round tasks
Because the base model stays frozen, SoftSkill is a lightweight, deployment-friendly adaptation method that avoids full fine-tuning and can be swapped in/out per skill
Gains are strongest on tasks requiring behavioral/procedural adherence (e.g., large LiveMath improvement) and more modest on tasks already well-handled by the base model (DocVQA)
The approach is not yet reliable for complex multi-step agentic execution, so practitioners should treat long-horizon tool-use/agent trajectories as an open problem rather than assuming latent compression transfers directly
Suggests a broader architectural pattern worth exploring: pairing human-readable skill documentation with a compact learned latent counterpart used at inference time, rather than relying solely on either representation

Abstract

Agent skills are commonly deployed as natural-language Markdown files that encode answer policies, evidence-use habits, and task procedures. These files are readable and portable, but they are consumed indirectly: for each task instance, a frozen language model must translate a long textual artifact into generation-time behavior. This paper asks whether a natural-language skill can instead initialize a compact continuous context object, refined by a trainable soft delta while the base model remains frozen. We propose SoftSkill, a frozen-backbone method that tunes such soft skills with next-token prediction and deploys them as latent behavioral priors at inference time. In our main single-round setting, a length-32 SoftSkill prefix on Qwen3.5-4B improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA. Relative to SkillOpt, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath, while replacing hundreds to thousands of Markdown skill tokens with a few virtual tokens. We further study agentic execution as a harder boundary case, where sparse trajectory imitation provides useful signal but does not yet robustly compress long-horizon procedural behavior. More broadly, the results suggest that some task skills are better treated not as additional Markdown to be reinterpreted at inference time, but as compact latent controls over how a frozen model enters the task.