← Back to Paper Radar

Structured Reasoning for Large Language Models

Jinyi Han, Zixiang Di, Zishang Jiang, Ying Liao, Jiaqing Liang, Yongqi Wang, Yanghua Xiao
arXiv.org | 2026
Structured Reasoning (SCR) splits LLM chain-of-thought into explicit, separately trainable Generate-Verify-Revise components. It teaches models when to stop reasoning through Dynamic Termination Supervision and progressive two-stage RL, which yields more efficient reasoning and more reliable self-verification.

Problem Statement

Long chain-of-thought reasoning improves accuracy but produces redundant steps, especially needless verification and revision after the model already has the correct answer. The authors attribute this to two causes: reasoning trajectories are unstructured, and training gives no targeted supervision for specific abilities such as self-verification and deciding when to terminate. The result is wasted inference compute and latency with no gain in quality.

Key Novelty

  • A framework (SCR) that turns free-form reasoning traces into explicit, evaluable, trainable components, implemented mainly as a Generate-Verify-Revise paradigm.
  • Dynamic Termination Supervision, which uses structured training data to teach the model when to stop reasoning instead of over-verifying correct answers.
  • A progressive two-stage RL strategy that separates learning signals: Stage 1 optimizes initial generation and self-verification, and Stage 2 optimizes revision, so the objectives for different abilities do not interfere.

Evaluation Highlights

  • Output token length drops by up to 50% compared with existing reasoning paradigms.
  • Reasoning efficiency and self-verification improve substantially across three backbone models. The abstract does not give exact accuracy figures.

Signal Assessment

5/10 This is a solid, practical contribution to the active problem of 'overthinking' in reasoning models, and decoupling RL by ability with explicit termination supervision is a sensible design. However, Generate-Verify-Revise and adaptive-length reasoning are well-explored ideas, so the advance is incremental rather than transformative.

Methodology

  1. Construct structured training data that breaks reasoning trajectories into explicit Generate, Verify, and Revise segments with clear boundaries.
  2. Apply Dynamic Termination Supervision so the model learns to end reasoning once verification confirms a correct answer, and to revise only when needed.
  3. Run Stage 1 RL to optimize initial answer generation and self-verification accuracy.
  4. Run Stage 2 RL to optimize the revision ability, building on the Stage 1 policy so its learning signals do not interfere with earlier skills.
  5. Evaluate on three backbone LLMs against existing reasoning paradigms for accuracy, self-verification quality, and output token length.

System Components

Structured Reasoning (SCR) Framework

Decouples reasoning traces into discrete, explicitly marked components that can each be evaluated and trained separately.

Generate-Verify-Revise Paradigm

The main instantiation of SCR: produce an initial solution, self-verify it, and revise only if verification fails.

Structured Training Data

Supervision data formatted to match the component structure, enabling targeted learning of each reasoning ability.

Dynamic Termination Supervision

Training signal that teaches the model when to stop reasoning, curbing redundant verification and revision after a correct answer.

Progressive Two-Stage RL

Sequential reinforcement learning: Stage 1 targets generation and self-verification, and Stage 2 targets revision, avoiding gradient or reward interference between abilities.

Results

Metric/Benchmark Baseline This Paper Delta
Output token length Existing reasoning paradigms (long CoT) SCR Up to 50% reduction
Reasoning efficiency Unstructured long CoT SCR Substantially improved (qualitative)
Self-verification ability Standard reasoning models SCR Substantially improved (qualitative)
Generalization across backbones N/A 3 backbone models Consistent gains reported

Key Takeaways

  • Explicitly structuring reasoning traces (for example, with generate, verify, and revise segments) makes individual reasoning skills supervisable and can roughly halve inference token cost, which matters directly for serving budgets and latency.
  • Teaching models when to stop is a high-leverage target. Much of the overthinking waste comes from verifying answers that are already correct, so termination-aware supervision is worth adding to reasoning RL pipelines.
  • When training multiple reasoning abilities with RL, staging the objectives (generation and verification first, then revision) can avoid conflicting reward signals. This curriculum-style design transfers to agentic self-correction loops.

Abstract

Large language models (LLMs) achieve strong performance by generating long chains of thought, but longer traces always introduce redundant or ineffective reasoning steps. One typical behavior is that they often perform unnecessary verification and revisions even if they have reached the correct answers. This limitation stems from the unstructured nature of reasoning trajectories and the lack of targeted supervision for critical reasoning abilities. To address this, we propose Structured Reasoning (SCR), a framework that decouples reasoning trajectories into explicit, evaluable, and trainable components. We mainly implement SCR using a Generate-Verify-Revise paradigm. Specifically, we construct structured training data and apply Dynamic Termination Supervision to guide the model in deciding when to terminate reasoning. To avoid interference between learning signals for different reasoning abilities, we adopt a progressive two-stage reinforcement learning strategy: the first stage targets initial generation and self-verification, and the second stage focuses on revision. Extensive experiments on three backbone models show that SCR substantially improves reasoning efficiency and self-verification. Besides, compared with existing reasoning paradigms, it reduces output token length by up to 50%.

Generated automatically from available metadata and abstract on 2026-11-09.