cs.CLNov 11, 2026

Token Commitment as a Control Surface for Steering Masked Diffusion Generation

Authors: Ambroise Aigueperse, Hugo Buurmeijer, Carmen Amo ALonso, Marco Pavone

Organizations: Stanford University

Abstract

Masked diffusion models repeatedly predict provisional tokens before a sampler decides which predictions become permanent. Existing inference-time steering methods primarily modify activations, logits, or denoising trajectories, while treating this commitment policy as a fixed decoding detail. We instead study token commitment as a diffusion-specific control surface and separate it into two decisions: which tokens to commit and how many tokens to commit at each step. For the first, we introduce counterfactual target-aware commitment, which ranks unresolved positions by a mixture of model confidence and leave-one-token-out target contribution. For the second, we introduce calibrated adaptive budgets driven by target progress and confidence. Across language and vision-language settings in the LLaDA family, multiple targets, two activation-steering methods, and an unsteered control, both mechanisms yield empirically monotonic target-quality trade-offs in every configuration we evaluate. Adaptive schedules also recover competitive operating points relative to scorer-free stochastic schedules matched in average denoising cost. Because both mechanisms require only a scalar score evaluated on a provisional completion, they apply unchanged when the target is an automatic proxy for task success rather than an output attribute: NLI entailment for dialogue summarization and CLIPScore for captioning, neither of which corresponds to a single reusable steering direction. We reproduce the behavior of both mechanisms on a second masked-diffusion family, Dream-7B, for one control target and one task-success target. These results characterize commitment--which provisional predictions become irreversible, and how rapidly--as an interpretable control surface for steering masked diffusion generation under the conditions we test.

Figures & tables

Appendix figures & tables42 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. On Trajectory-Aware Training for Masked Diffusion Language Models

    Sep 29, 2026Manuel Madeira, Amitis Shidani, Alice Bizeul +6Masked Diffusion ModelsDiffusion Language Model Inference

  2. The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models

    May 23, 2026Bohang Sun, Max Zhu, Francesco Caso +5Diffusion Language Model DecodingDiffusion Language Models

  3. What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models

    Oct 22, 2026Jinya Sakurai, Xun Xu, Jaehong YoonDiffusion Language Model DecodingDiffusion Language Models