Grammar-Constrained Decoding

Overview

The field studies how to combine language models with formal grammars so generated sequences follow languages and formats such as JSON, SQL, programming languages, and domain-specific commands. Common approaches use parser- or automaton-state token masks, context-free grammar parsing, and increasingly lookahead, grammar refinement, probability correction, or parallel decoding to improve completion and speed. These constraints support reliable structured generation, executable queries, and policy enforcement, but syntactic validity alone does not guarantee correct meaning or safe behavior. A central tension is that hard masking can distort the model’s conditional distribution or block good outputs when grammars are incomplete, while richer checks and more faithful sampling add computational cost.

Latest papers

CardsList

32 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Attention Meets Reachability: Structural Equivalence and Efficiency in Grammar-Constrained LLM Decoding

    Mar 4, 2026Faruk Alpay, Bilge SenturkGrammar-Constrained Decoding

  2. Grammar-Aligned Decoding

    May 31, 2024Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick +2Constrained Generative ModelingConstrained Decoding