Large language models (LLMs) often generate responses longer than necessary, increasing inference latency and computational cost. While prompting can encourage conciseness, controlling output length without compromising response completeness remains challenging. Aggressive early stopping can reduce generation costs but risks terminating responses prematurely. We introduce MinimalLM, a training-free framework for reliable verbosity control that combines uncertainty-aware, length-dependent stop-token adjustment with structural completeness guardrails. MinimalLM separates the decision of when to stop from whether stopping is structurally appropriate, coordinating response-level instructions, decoding-time intervention, and completion checks without modifying model parameters or requiring auxiliary models. We evaluate MinimalLM across three open-source LLMs and 330 questions, producing 5,940 responses across baseline and ablation configurations. MinimalLM reduces generated tokens by 47.6%, accelerates individual responses by 19-68%, and eliminates structurally truncated outputs in the primary evaluation. Despite substantial token savings, it achieves an average quality score of 8.04/10, compared with 8.42/10 for unmodified models and 6.46/10 for length-matched truncation. Evaluations across diverse subject domains (such as healthcare, programming, law, and science) demonstrate robust efficiency gains, while ablations establish the complementary roles of decoding control and completeness constraints. Our findings highlight that efficient generation requires more than producing fewer tokens: it requires controlling when generation ends while preserving response integrity. MinimalLM offers a lightweight, training-free approach toward this goal.
Figures & tables
Method
Fewer out.
Training-
No cut-off
tokens
free
answers
Speculative decoding
–
✓
n/a
LazyLLM, SlimInfer
–
✓
n/a
Dynamic input pruning
–
minimal
n/a
TokenSkip
✓
–
–
“Be concise” prompt
✓
✓
–
Table 1: Ways to cut generation cost. Only methods that reduce output tokens can cut answers off.
Figure 1: MinimalLM in three stages: an instruction before generation; inside the decoding loop, the stop-token boost with its entropy gate and the completeness gate; and a final check after generation. The base LLM is kept unchanged.
System
TRR
Cut off
Incompl.
Score
Baseline
–
5.7%
14.7%
8.42
Instruction only
45.3%
0.6%
22.0%
8.07
w/o guardrails
32.6%
59.3%
55.2%
7.51
w/o gate
50.8%
17.3%
31.6%
7.83
w/o instruction
19.8%
0.3%
21.7%
8.22
MinimalLM
47.6%
0.0%
23.4%
8.04
Table 2: MinimalLM and its ablations, with 990 answers per configuration (330 questions × 3 models). TRR denotes token reduction ratio; cut off indicates structural truncation detected by the rule; incomplete indicates cut-off or missing information identified by the judge; score is the judge’s rating (1–10). The abbreviation w/o denotes removal of a component.
Figure 2: Example responses from Mistral-7B to a question requesting five examples, displayed in our demonstration interface (intermediate text omitted). Left: without guardrails, the stopping mechanism terminates mid-sentence during the third example (390 tokens). Right: MinimalLM provides all five examples and ends at a complete sentence (421 tokens), compared with 618 tokens for the baseline.
Figure 3: Comparison against direct truncation across 330 questions and three models. Dark curves represent baseline responses truncated to 90–30% of their original length; light curves represent the same responses trimmed to the last complete sentence. Points indicate the configurations in Table 2 . At comparable token reductions, MinimalLM achieves higher judged quality and completeness than direct truncation, although it retains less content from the original baseline response.
ms / token
batched GPU time
Model
base
ours
base
ours
Mistral-7B
30.8
31.0
13.9
13.2
Llama-3.1-8B
32.2
32.5
13.8
10.3
Qwen2.5-3B
28.1
28.2
7.1
3.2
Table 3: Inference and serving costs. Milliseconds per generated token are measured for individual requests using 16 MT-Bench questions. Batched GPU time is reported in minutes for 330 questions processed in fixed batches.
Domain
Tokens
Change
Time (s)
Complete
Health
669 / 407
− 39%
21.1 / 12.9
✓/ ✓
History
645 / 405
− 37%
20.4 / 12.9
✓/ ✓
Economics
520 / 383
− 26%
16.4 / 12.2
✓/ ✓
Programming
784 / 300
− 62%
24.7 / 9.5
✓/ ✓
Science
629 / 330
− 48%
19.9 / 10.5
✓/ ✓
Mathematics
183 / 260
+42%
5.8 / 8.3
✓/ ✓
Table 4: Domain performance on eight categories using Mistral-7B. Values report baseline / MinimalLM for generated tokens, percentage change, latency in seconds, and structural completeness (✓) (see more examples in Appendix D ).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Interface comparison on a health question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 5: Interface comparison on a history question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 6: Interface comparison on an economics question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 7: Interface comparison on a law question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 8: Interface comparison on a science question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 9: Interface comparison on a programming question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 10: Interface comparison on a writing question (Mistral-7B). Left: baseline; right: MinimalLM.
Figure 11: Interface comparison on a mathematics question (Mistral-7B). Left: baseline; right: MinimalLM.
Reinforcement learning (RL) has enhanced the capabilities of large language models (LLMs) through reward-driven training. Nevertheless, this process can introduce excessively long responses, inflating inference latency and computational overhead. Prior length-control approaches typically rely on fixed heuristic reward shaping, which can misalign with the task objective and require brittle tuning. In this work, we propose LACONIC, a reinforcement learning method that enforces a target token budget during training. Specifically, we update policy models using an augmented objective that combines the task reward with a length-based cost. To balance brevity and task performance, the cost scale is adaptively adjusted throughout training. This yields robust length control while preserving task reward. We provide a theoretical guarantee that support the method. Across mathematical reasoning models and datasets, LACONIC preserves or improves pass@1 while reducing output length by over 50%. It maintains out-of-domain performance on general knowledge and multilingual benchmarks with 44% fewer tokens. Moreover, LACONIC integrates into standard RL-tuning with no inference changes and minimal deployment overhead.
Chang Liu, Yiran Zhao, Lawrence Liu +3
University of California, Los Angeles · Salesforce AI Research · National University of Singapore +1
Unlike autoregressive language models, which terminate variable-length generation upon predicting an End-of-Sequence (EoS) token, Diffusion Language Models (DLMs) operate over a fixed maximum-length context window for a predetermined number of denoising steps. However, this process is independent of the required response length, resulting in computational waste for the majority of short responses common in reasoning and chat tasks. To address this problem, we conjecture that the latent prompt representation contains sufficient information to estimate the required output length. We provide empirical evidence for this phenomenon and propose a zero-shot mechanism to dynamically crop the context window before generation begins, leading to fewer diffusion steps and substantial computational savings. We evaluate our approach on four benchmarks with diverse tasks -- GSM8K (reasoning), HumanEval (code generation), IfEval (instruction following), and LongFormQA (question answering) -- revealing massive efficiency gains at minimal performance impact. We report significant reductions in FLOPs across all tasks, with no statistically significant performance degradation, and significant performance improvements in 2 out of 4 tasks.
Vittorio Rossi, Giacomo Cirò, Davide Beltrame +3
Department of Computing Sciences, Bocconi University, Milan, Italy.
Despite the strong performance of Large Language Models (LLMs) on complex instruction-following tasks, precise control of output length remains a persistent challenge. Existing methods primarily attempt to enforce length constraints by externally imposing length signals or optimization objectives, while largely overlooking the underlying limitation: the model's intrinsic deficit in length cognition. To address this, we propose LARFT (Length-Aware Reinforcement Fine-Tuning), a training framework that aligns the model's length cognition with its action. Specifically, LARFT integrates length-oriented reinforcement learning with a hindsight length awareness. By transforming on-policy data into hindsight self-awareness tasks where the model learns to identify the actual length of its own generation, LARFT jointly optimizes the model's internal representation of length information and refines its policy to satisfy length constraints, thereby achieving precise and reliable length instruction following. Extensive experiments across four base models demonstrate that LARFT outperforms existing baselines, achieving an average improvement of +20.92 points across three length instruction following benchmarks with only a marginal decline of -1.45 points on four general capability benchmarks.
Wei Zhang, Lintong Du, Yuanhe Zhang +4
Beijing University of Posts and Telecommunications, Beijing, China · Nanyang Technological University, Singapore, Singapore