cs.CLMar 6, 2026
SaveDiffusion Language Models Are Natively Length-Aware
Organizations: Department of Computing Sciences, Bocconi University, Milan, Italy.
Abstract
Unlike autoregressive language models, which terminate variable-length generation upon predicting an End-of-Sequence (EoS) token, Diffusion Language Models (DLMs) operate over a fixed maximum-length context window for a predetermined number of denoising steps. However, this process is independent of the required response length, resulting in computational waste for the majority of short responses common in reasoning and chat tasks. To address this problem, we conjecture that the latent prompt representation contains sufficient information to estimate the required output length. We provide empirical evidence for this phenomenon and propose a zero-shot mechanism to dynamically crop the context window before generation begins, leading to fewer diffusion steps and substantial computational savings. We evaluate our approach on four benchmarks with diverse tasks -- GSM8K (reasoning), HumanEval (code generation), IfEval (instruction following), and LongFormQA (question answering) -- revealing massive efficiency gains at minimal performance impact. We report significant reductions in FLOPs across all tasks, with no statistically significant performance degradation, and significant performance improvements in 2 out of 4 tasks.
Figures & tables
Figure 1: Predicted Length Distributions. Our SmartCrop ( ) method successfully predicts task-specific output lengths across four benchmark datasets. The abrupt truncations observed in certain distributions correspond to context length constraints (refer to Section 4 for details).
| Benchmark | Max new tokens ( ) | Steps ( ) | Content |
|---|---|---|---|
| GSM8K | 256 | 256 | Math |
| HumanEval | 512 | 512 | Code |
| IfEval | 1280 | 1280 | Instruction |
| LongFormQA | 512 | 64 | QA |
Table 1: Experimental Configurations. Summary of hyperparameters across benchmark datasets. Max new tokens ( ) denotes the number of masked tokens appended to the prompt in the diffusion canvas. Steps ( ) indicates the total number of denoising iterations. For example, with implies a linear schedule where exactly 1 token is unmasked per step. Content describes the task domains represented in each dataset.
| Benchmark | Method | Avg. Processed Length | Metric | FLOPs Saved | Perf. | |
|---|---|---|---|---|---|---|
| IfEval | FC | 87.2 | 1367.2 | - | - | |
| SC-0.5 | ||||||
| SC-0.75 | ||||||
| SC-0.9 | ||||||
| SC-0.95 | ||||||
| SC-0.99 |
Table 2: Main Results. Performance comparison between native diffusion decoding and our proposed dynamic cropping method across four benchmarks using the LLaDA architecture. Benchmark indicates the evaluation suite. Method distinguishes between the native Full Context ( FC ) baseline and our SmartCrop ( SC ) approach at various length-prediction quantiles ( ). represents the average prompt length in tokens (constant per benchmark). Avg. Processed Length denotes the mean number of total tokens processed by the model in each forward pass, comprising both the prompt and the generated sequence. For the FC baseline, this value is constant at , where and represent the prompt length and the maximum sequence length, respectively. In contrast, for SC , the output length is defined by the predicted total length . Metric denotes the task-specific performance score. FLOPs Saved quantifies the reduction in total floating-point operations relative to the FC baseline (e.g., a 98% saving implies our method requires only 2% of the baseline computation). Perf. reports the relative percentage change in Metric compared to the FC baseline. For all metrics marked with , higher is better. We determine statistical significance via pairwise bootstrap hypothesis testing (5000 resamples) between FC and SC within each experimental condition. We compute paired differences on samples matched by document ID for both Metric and FLOPs Saved . We estimate two-sided -values from the resulting bootstrap distributions. Significance levels are denoted as and .
Figure 2: Sensitivity of IfEval Performance to Context Length Perturbations. We analyze the robustness of SmartCrop ( ) by shifting the predicted length by a deviation factor . The blue curve shows the model performance (mean 95% CI) across these varying context lengths. The red line denotes the control baseline, where lengths are sampled from the empirical length distribution of other benchmarks. The green line represents the Full Context baseline performance. While the model is relatively robust to moderate under-estimation (negative ), generation quality degrades as superfluous padding is reintroduced (positive ), eventually converging toward the baseline.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Predicted Length Invariance ( HumanEval ). Left: kernel density estimate of predicted new tokens for . Right: boxplots of the same values. The bulk of the distribution is comparatively stable across , with the main visible difference being a stronger right truncation when , which is expected when the required completion length approaches the canvas limit. Note: causes the predicted length distribution to be heavily truncated and uninformative compared to larger canvases.
Figure 4: Predicted Length Invariance ( IfEval ). Left: kernel density estimate of predicted new tokens for . Right: boxplots of the same values. The central mass of the predicted-length distribution (roughly 50–150 new tokens) is broadly consistent across , while larger canvases primarily increase the range of rare long-length outliers.
Figure 5: Predicted Length Invariance ( LongFormQA ). Left: kernel density estimate of predicted new tokens for . Right: boxplots of the same values. The predicted length is close to invariant across for the typical range of outputs, with only small shifts in the median and dispersion. This supports the claim in the main text that, for LongFormQA , the model’s inferred length prior is largely insensitive to the particular (potentially conservative) initial canvas size used for the first forward pass.
Figure 6: Predicted Length Invariance ( GSM8K ). Left: kernel density estimate of predicted new tokens for . Right: boxplots of the same predicted token counts. We observe a pronounced dependence of the predicted length distribution on (most visible when setting at 256 or 4096), indicating that the per-position EoS probabilities used by SmartCrop are not strictly invariant to the amount of masked padding presented to the model at the first denoising step. The curve also shows a hard truncation consistent with the finite canvas constraint.
Figure 7: Performance Gain vs. Length ( GSM8K ). Per-instance change in Exact Match (grey; ) plotted against generated length. The blue curve reports the mean delta within length bins.
Figure 8: Performance Gain vs. Length ( HumanEval ). Per-instance change in Pass@1 (grey; ) plotted against generated length. The binned average (blue) is non-monotonic and fluctuates across bins, which is consistent with either weak dependence on length or limited sample counts per bin.
Figure 9: Performance Gain vs. Length ( IfEval ). Per-instance change in prompt-level Strict Accuracy (grey; ) plotted against generated length. The binned average (blue) is positive for shorter generations and approaches zero for longer ones, indicating that the net gains from SmartCrop at this operating point are concentrated on prompts that produce relatively short outputs. The negative dip around intermediate lengths should be interpreted with the corresponding bin sample size in mind.
Figure 10: Performance Gain vs. Length ( LongFormQA ). Per-instance change in the evaluation score (grey; continuous delta) plotted against generated length. The binned mean delta (blue) is positive across the typical length range and does not exhibit a clear monotonic trend.