Qwen3-Coder-Next Technical Report
Problem Statement
Strong coding agents usually rely on large dense or large-active-parameter models, which are expensive to serve and slow in long multi-turn agent loops. Training agents also requires many realistic, verifiable tasks with runnable environments, and these are scarce and hard to build. The paper tests how far training recipes, rather than scale, can push agentic coding capability at low inference cost.
Key Novelty
- A highly sparse MoE coding model (80B total / 3B active) aimed at coding-agent workloads rather than single-shot code completion. It is likely built on the efficiency-oriented Qwen3-Next backbone, though the abstract does not say so.
- Large-scale synthesis of verifiable coding tasks, each paired with an executable environment, so the model can learn directly from execution feedback.
- Environment feedback is used in both mid-training and reinforcement learning, extending agentic training beyond the post-training stage.
- Open release of both base and instruction-tuned weights, which enables downstream agent research and fine-tuning.
Evaluation Highlights
- Competitive performance on agent-centric benchmarks, including SWE-Bench (repository-level issue resolution) and Terminal-Bench (command-line agent tasks), relative to its active parameter count.
- The headline claim is efficiency: strong agentic coding results at roughly 3B active parameters. The abstract frames this as a capability-per-compute advantage rather than absolute state of the art.
Signal Assessment
Methodology
- Start from a sparse MoE architecture with 80B total and about 3B active parameters, chosen for efficient inference in long-horizon agent loops.
- Synthesize a large corpus of verifiable coding tasks (e.g., repository issues and terminal tasks) and pair each with an executable environment and an automatic verifier such as tests.
- Run agentic mid-training on trajectories and interactions grounded in these environments, instilling tool use and multi-turn software-engineering behavior before post-training.
- Apply reinforcement learning with environment-execution feedback as the reward signal to optimize end-to-end agent task success.
- Evaluate on agentic benchmarks (SWE-Bench, Terminal-Bench), then release base and instruct checkpoints.
System Components
A mixture-of-experts model that activates about 3B of its 80B parameters per token, giving the capacity of a large model at the inference cost of a small one.
Generates coding tasks at scale with automatic correctness checks, providing reliable supervision and reward signals for agent training.
Sandboxed, runnable environments (repositories, shells) paired with each task so the agent can act, observe outcomes, and be scored.
An intermediate training stage that exposes the model to environment-grounded agent interactions before final post-training, building agentic priors early.
Optimizes the policy using execution-based rewards (e.g., tests passing) on multi-turn agent tasks.
Public checkpoints for research, fine-tuning, and deployment in real-world coding agents.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| SWE-Bench (agentic issue resolution) | Models with similar or larger active parameter counts | Competitive at ~3B active parameters | Favorable capability per active parameter (numbers not given in abstract) |
| Terminal-Bench (CLI agent tasks) | Models with similar or larger active parameter counts | Competitive at ~3B active parameters | Favorable capability per active parameter (numbers not given in abstract) |
| Inference cost (active params) | Dense or larger-active coding models (tens of billions active) | ~3B active / 80B total | Substantially lower per-token compute |
Key Takeaways
- For coding agents, which generate long multi-turn trajectories, high-sparsity MoE models offer a strong cost/latency trade-off. A 3B-active model can be a practical default for self-hosted agents.
- Investing in verifiable task synthesis with executable environments is a key lever. Execution-grounded data and rewards appear to matter as much as model scale for agentic coding.
- Bringing environment-grounded agentic data into mid-training, not only RL post-training, is a recipe worth replicating when building specialized agent models.
- The released base checkpoint offers a good starting point for domain-specific agent fine-tuning or RL research without the cost of training a large MoE from scratch.
Abstract
We present Qwen3-Coder-Next, an open-weight language model specialized for coding agents. Qwen3-Coder-Next is an 80-billion-parameter model that activates only 3 billion parameters during inference, enabling strong coding capability with efficient inference. In this work, we explore how far strong training recipes can push the capability limits of models with small parameter footprints. To achieve this, we perform agentic training through large-scale synthesis of verifiable coding tasks paired with executable environments, allowing learning directly from environment feedback via mid-training and reinforcement learning. Across agent-centric benchmarks including SWE-Bench and Terminal-Bench, Qwen3-Coder-Next achieves competitive performance relative to its active parameter count. We release both base and instruction-tuned open-weight versions to support research and real-world coding agent development.