cs.CLOct 19, 2026
SaveOn the Fragility of Tasks under Composition: Implicit Budgeting in Language Models
Organizations: TU Munich · MIT CSAIL
Abstract
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation. This degradation cannot be fully explained by the accumulation of independent errors. Across both open-weight models and the frontier model Claude Opus 5, our findings instead point to an implicit thinking budget underlying this effect. In particular, under composition, models use substantially fewer tokens per task than in isolation, even when the full sequence remains well within the context window. Tasks that require longer solution traces in isolation are consequently more under composition, i.e., models make more errors on prompts containing these tasks. In isolation, non-fragile tasks receive more reasoning than they need and waste compute. However, by grouping tasks according to their fragility, the implicit budgeting can be manipulated to reduce token usage by % across a range of math and linguistic tasks while also preserving accuracy.
Figures & tables
Figure 1: Composition reduces every task’s reasoning by the same factor. When solved in isolation (top), each task receives more reasoning than it requires. When composed into a single prompt (bottom), the reasoning allocated to each task shrinks proportionally (yellow). Tasks with sufficient slack absorb this reduction and are still solved correctly, whereas the task that requires long reasoning falls below its requirement and thus fails.
Figure 2: Accuracy degradation on confidently solved GPQA-Diamond tasks. Left: accuracy drop per model at . Right: accuracy drop as a function of composition size . All tasks were solved in three out of three isolated attempts, yet accuracy drops once they are composed.
Figure 3: Context usage across datasets. Most responses use under 25% of the context window; in most datasets, under 1% hit the limit.
Figure 4: Token ratio as a function of composition size , where is the number of thinking tokens spent on task within a composition of size and the number spent in isolation. Per-task token counts in compositions are obtained by locating task tags in the reasoning trace. Tasks are binned by the quantile of their isolated trace length into Low ( ), Medium ( – ), and High ( ). On log-log axes, the ratio decreases approximately linearly in , indicating that LLMs shrink the relative token usage of each task under composition by roughly , with no systematic dependence on . Results are from Qwen3 4B Thinking.
Figure 5: Accuracy drop as a function of the relative thinking token reduction for Qwen3 4B Thinking on MMLU-Redux (left) and MATH-500 (right) below the dashed line, the y-axis is logarithmic for presentation purposes. Tasks are binned by the quantile of their isolated trace length into Low ( ), Medium ( – ), and High ( ). As the reduction through composition (comp.) grows, accuracy drops, and tasks with high reasoning requirements degrade substantially more than low-effort tasks. Forcing a fixed budget externally (forced) generally causes stronger degradation, but the degradation follows the same patterns (more details in Appendix A.1.3 ).
| Isolated, | Random batching, | Fragility-aware (measured) | Fragility-aware (predictor) | |||||
| Acc. | Tok. | Acc. | Tok. | Acc. | Tok. | Acc. | Tok. | |
| GPQA-Diamond | ||||||||
| Qwen3 4B T. | 66.7% | 7663 | 56.4% | 2470 | 66.8% | 5200 | 67.3% | 7534 |
| Qwen3 8B | 58.7% | 5973 | 53.4% | 1950 | 60.0% | 3520 | 56.9% | 5106 |
| Qwen3 14B | 61.5% | 4597 | 51.8% | 1500 | 57.9% | 2111 | 61.3% | 3665 |
| Qwen3.8 27B | 83.2% | 4303 | 79.2% | 2740 | 84.0% | 4102 | 85.3% | 4324 |
Table 1: Task accuracy (Acc.) and output tokens (Tok., includes thinking and answer) per task under different batching policies. For each policy, the left column reports accuracy and the right column tokens, with the change to solving in isolation ( ) in parentheses, given as an absolute difference in percentage points for accuracy and as a relative change in % for tokens. The Average row gives the mean change over the models shown. Standard fixed-size random batching causes substantial accuracy degradation, which fragility-aware batching largely mitigates. With a model-specific linear predictor, part of these token savings carries over to unseen tasks while still avoiding compositional degradation.
Figure 6: Effects of implicit budgeting across three post-training stages of OLMo 3 7B on GPQA-Diamond. Left: Token ratio as a function of composition size . Right: Accuracy drop as a function of the token ratio.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Accuracy drop as a function of composition size for all datasets and models. All tasks were solved in three out of three isolated attempts, yet accuracy drops once they are composed, on every dataset except OneWordQA and StrategyQA, whose tasks require little reasoning. The R1-distilled models degrade the most, and the extent of degradation varies strongly between datasets.
Figure 8: Token ratio on Qwen3 4B Thinking. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 9: Token ratio on Qwen3 8B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 10: Token ratio on Qwen3 14B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 11: Token ratio on Qwen3.8 27B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 12: Token ratio on R1 Llama 8B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 13: Token ratio on R1 Qwen 7B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 14: Token ratio on R1 Qwen3 8B. Power-law fits through , one per dataset: median slope (range ), mean .
Figure 15: Accuracy drop versus relative thinking token reduction for Qwen3 4B Thinking.
Figure 16: Accuracy drop versus relative thinking token reduction for Qwen3 8B.
Figure 17: Accuracy drop versus relative thinking token reduction for Qwen3 14B.
Figure 18: Accuracy drop versus relative thinking token reduction for Qwen3.8 27B.
Figure 19: Accuracy drop versus relative thinking token reduction for R1 Llama 8B.
Figure 20: Accuracy drop versus relative thinking token reduction for R1 Qwen 7B.
Figure 21: Accuracy drop versus relative thinking token reduction for R1 Qwen3 8B.
Figure 22: Positional dependence of thinking budget allocation and compositional degradation on MATH-500 at composition size . The upper panels show thinking tokens (yellow-red-purple) and the lower panels show the accuracy drop (yellow-green-purple). The last position in each composition holds the dummy task, which we exclude from the analysis because our method cannot reliably determine its token usage (Appendix A.2.1 ).
Figure 23: Disattenuated Pearson correlation of accuracy degradation across different conditions, i.e. model and composition size. Each pair of conditions (a cell in the heatmap) averages over datasets and, on the left, over models, on the right, over composition sizes. Left: The correlation is substantially larger when comparing accuracy degradation within the same model across different composition sizes . Right: By contrast, the correlation between different models is low, especially across model families, suggesting different architectures and training recipes lead to different tasks being fragile.
| Isolated, | Random batching, | Fragility-aware (measured) | Fragility-aware (predictor) | |||||
| Acc. | Tok. | Acc. | Tok. | Acc. | Tok. | Acc. | Tok. | |
| GPQA-Diamond | ||||||||
| Qwen3 4B T. | 66.7% | 7663 | 56.4% | 2470 | 66.8% | 5200 | 67.3% | 7534 |
| Qwen3 8B | 58.7% | 5973 | 53.4% | 1950 | 60.0% | 3520 | 56.9% | 5106 |
| Qwen3 14B | 61.5% | 4597 | 51.8% | 1500 | 57.9% | 2111 | 61.3% | 3665 |
| Qwen3.8 27B | 83.2% | 4303 | 79.2% | 2740 | 84.0% | 4102 | 85.3% | 4324 |
Table 2: Task accuracy (Acc.) and output tokens (Tok., includes thinking and answer) per task under different batching policies for all models and datasets. The columns report absolute values. All other columns report absolute values with the change with respect to in parentheses, as an absolute difference in percentage points for accuracy and as a relative change in % for tokens. Within each policy, the left column is accuracy and the right column is tokens. The Average row reports the mean over all models shown, eight on GPQA-Diamond and seven on the other datasets: absolute values for and the mean change for all other policies. On OneWordQA, the Average leaves out R1 Qwen3 8B (grey), whose answers in composition do not follow our result contract, which leads to increased parsing errors.
| Response at largest | Responses (%) | ||||
|---|---|---|---|---|---|
| Dataset | tokens | of budget | hit limit | unparseable | |
| GSM8K | 20 | 2.8k | 3% | 0.0 [0] | 0.2 [2] |
| MATH-500 | 20 | 19.6k | 29% | 0.9 [10] | 2.4 [32] |
| AIME | 10 | 57.6k | 35% | 4.8 [39] | 10.7 [47] |
| Integrals | 20 | 7.4k | 17% | 0.0 [0] | 1.4 [15] |
| MMLU-Redux † | 20 | 1.6k | 4% | 0.2 [2] | 3.9 [44] |
Table 3: Context limits and parsing failures in the composed sets. Each response has a token budget equal to the context window minus the prompt, capped at the model’s maximum generation length; a response that exhausts it is truncated. Response tokens are the mean at the largest . Truncation and parsing-failure rates are pooled over all models and , with the worst model and in brackets. † Most unparseable responses come from R1 Qwen3 8B, which often breaks the answering contract under composition.
More in this topic
Compositional Reasoning
Compositional reasoning studies how models combine facts, concepts, rules, and skills to solve tasks that require multiple structured steps.
See all 39 papers