Constrained Decoding

Follow this topic Ask about this topic

Constrained decoding guides a model toward outputs that obey requirements such as a JSON schema, grammar, API specification, code constraints, or permitted label set. Common approaches use finite automata, parsers, tries, and token masking, while search, sampling, verification, and diffusion-based methods handle more complex or global constraints. These techniques can make outputs reliably valid and improve task-specific measures, but validity alone does not ensure correct meaning or successful tool use. A central tension is how to enforce constraints without distorting the model’s probabilities, hindering reasoning, or creating failures when constraints are incomplete.

Latest papers

67 papers16 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Suffix-Constrained Greedy Search Algorithms for Causal Language Models
Mar 1, 2026Ayoub Hammal, Pierre Zweigenbaum, Caio CorroConstrained Generative Modeling Constrained Decoding
1Université Paris-Saclay, CNRS, LISN · 2INSA Rennes, IRISA, CNRS, Université de Rennes
Vectorizing the Trie: Efficient Constrained Decoding for LLM-based Generative Retrieval on Accelerators
Feb 26, 2026Zhengyang Su, Isay Katsman, Yueqi Wang +10Constrained Decoding Generative Retrieval
Google Mountain View, CA, USA · Yale University New Haven, CT, USA
Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian
Feb 19, 2026Pietro Ferrazzi, Mattia Franzin, Alberto Lavelli +1Small Language Models Clinical NLP
Fondazione Bruno Kessler, Povo, Trento, Italy · University of Padova, Padova, Italy
The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding
Feb 8, 2026Avinash Reddy, Thayne T. Walker, James S. Ide +1Constrained Generative Modeling Constrained Decoding
Department of Computer Science, University of Central Florida, FL, US · Lockheed Martin AI Center, CT, US
Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
Feb 6, 2026Jacqueline He, Jonathan Hayase, Wen-tau Yih +3Constrained Decoding Language Model Decoding
University of Washington · Allen Institute for Artificial Intelligence
Strategies for Span Labeling with Large Language Models
Jan 23, 2026Danil Semin, Ondřej Dušek, Zdeněk KasnerConstrained Decoding Language Model Decoding
Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics, Charles University
Grammar-Aligned Decoding
May 31, 2024Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick +2Constrained Generative Modeling Constrained Decoding
University of Wisconsin-Madison · University of California San Diego

Mar 1, 2026J/K move · Enter open · S save

Suffix-Constrained Greedy Search Algorithms for Causal Language Models

Ayoub Hammal, Pierre Zweigenbaum, Caio Corro

1Université Paris-Saclay, CNRS, LISN · 2INSA Rennes, IRISA, CNRS, Université de Rennes

Constrained Generative Modeling Constrained Decoding LLM Inference Controllable Text Generation Autoregressive Decoding

Open paper AskPDF arXiv

Large language models (LLMs) are powerful tools that have found applications beyond human-machine interfaces and chatbots. Beside free-form generation, there has been an interest in constrained generation, a setting where LLMs are constrained to generate well-formed outputs with respect to the language defined by a formal grammar. Although appealing, this setting may be over restrictive for downstream applications. For example, many LLM tasks require the model to reason freely before generating its final response in a specific format. In this work, we introduce suffix-constrained generation, a constrained generation setting in which only the end of the response is constrained by a grammar, a scenario that is not supported by existing constrained generation methods. We introduce several suffix-constrained generation algorithms that are based on greedy search. We experiment on several datasets, and show that our approach allows to guarantee suffix constraints without having a negative impact on results, and even improving them in many settings.