cs.CLOct 13, 2025

Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production

Authors: Alexandre Galashov, Matt Jones, Rosemary Ke, Yuan Cao, Vaishnavh Nagarajan, Michael C. Mozer

Organizations: Google DeepMind

Abstract

Within the landscape of inference-time scaling methods for foundation models, a width-based approach to scaling--which involves the insertion of <pause> tokens in the input stream to delay model responses--offers a unique advantage by increasing model expressivity while remaining highly parallelizable at both training and inference. The existing literature on training models to utilize <pause> tokens relies on the standard cross-entropy objective in which the model output is read out and evaluated only at the final step of a pause sequence. This approach provides no mechanism for the model to regulate its own processing or to signal readiness to respond, treating the additional compute steps as a barrier to skip over rather than a resource to be used adaptively. We propose a supervised loss, Catch Your Breath (CYB), framed as a sequential-decision problem, that trains a model to dynamically and autonomously scale the number of compute steps used for each input token. The model indicates the need for additional compute steps by emitting a special <don't know> output, delaying its response via a pause. The model can abstain multiple times to obtain longer delays. Our experiments demonstrate that when <pause> tokens are included in the input stream, pretraining or fine-tuning with CYB significantly outperforms standard cross-entropy, reducing perplexity and enhancing downstream accuracy with no additional computational or memory cost.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth

    Mar 2, 2026Shixiang Song, He Li, Zitong Wang +6Recurrent Language ModelsInference-Time Adaptation of Language Models

  2. PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Masking

    Mar 2, 2026He Li, Feichen Song, Boyi Zeng +4Adaptive ComputationInference-Time Adaptation of Language Models

  3. Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective

    Sep 3, 2026Jaehyeon Kim, Suhwan Kim, Nakyung Lee +4LLM Fine-TuningFine-Tuning