CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization

Deep Shah; Sanket Badhe; Nehal Kathrotia; Priyanka Tiwari

cs.CLApr 8, 2026

CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization

Authors: Deep Shah, Sanket Badhe, Nehal Kathrotia, Priyanka Tiwari

Organizations: Google LLC · Purdue University

Abstract

Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation. Existing automatic prompt optimization(APO) frameworks target task accuracy exclusively at the expense of generating long reasoning traces. We propose Cost-Regularized Optimization of Prompts (CROP), an APO method that introduces regularization on response length by generating textual feedback in addition to standard accuracy feedback. This forces the optimization process to produce prompts that elicit concise responses containing only critical information and reasoning. We evaluate our approach on complex reasoning datasets, specifically GSM8K, LogiQA and BIG-Bench Hard. We achieved an 80.6% reduction in token consumption while maintaining competitive accuracy, seeing only a nominal decline in performance. This presents a pragmatic solution for deploying token-efficient and cost-effective agentic AI systems in production pipelines.

Figures & tables

Figure 1: Overview of CROP optimization pipeline. A meta-optimizer updates the system prompt by aggregating textual gradients for both task accuracy (

g_{task}

) and response brevity (

g_{reg}

), systematically enforcing token-efficient reasoning.

Figure 2: Performance and efficiency comparison across three datasets using the Gemini 2.0 Flash model.

Method	Core Instruction
No Reasoning	Output ONLY the final answer without reasoning or thinking.
Chain-of-Thought	Think step by step.
CROP (Ours)	[…] concisely, showing the combined essential calculations leading directly to the final numerical answer. Omit intermediate labels. Focus on chained calculations.

Table 1: Comparison of core system instructions across prompting baselines for the GSM8K dataset. Standard prefix formulations and output formatting constraints have been omitted to highlight the distinct reasoning directives.

Panel of Table 2 — Table 2: Reasoning comparison on the GSM8K dataset.

Model	Dataset	CoT	w/o Reasoning	CROP	TextGrad	Be Concise	Only Number	Use Abbrev
Gemini 2 Flash	GSM8K	95.3 (198.4)	51.2 (9.2)	93.4 (50.0)	95.5 (185.5)	96.2 (120.8)	91.4 (113.0)	95.1 (145.7)
	Count Object	96.0 (101.2)	66.4 (9.4)	94.8 (19.6)	90.5 (24.5)	96.0 (59.0)	92.5 (55.0)	95.0 (92.0)
	LogiQA	65.2 (540.5)	60.4 (5.1)	64.2 (181.0)	66.4 (663.0)	65.1 (301.0)	61.9 (423.0)	65.3 (422.0)
Qwen2.5 7B	GSM8K	90.2 (123.4)	54.9 (3.4)	87.8 (21.3)	89.8 (101.2)	90.4 (82.3)	85.1 (31.1)	88.2 (72.4)
	Count Object	92.1 (56.3)	67.8 (3.8)	90.1 (12.1)	88.8 (32.7)	92.7 (46.6)	81.5 (16.2)	90.2 (39.7)
	LogiQA	61.2 (287.0)	54.0 (3.4)	59.3 (84.2)	60.1 (212.3)	61.1 (152.3)	50.8 (32.8)	54.7 (212.0)

Table 3: Model Performance Across Prompting Methods. Cells display Accuracy (%) with Average Token Count in parentheses.

Meta-Optimizer Model	Discovered System Prompt (Core Instructions)
Gemini 2.0 Flash (Lower Capacity)	Solve the problem. Only provide the answer and do not provide any explanations.
Gemini 3.1 Pro (High Capacity)	Solve the problem with extreme conciseness. Do not use conversational filler, restate the question, or write full sentences. Simply provide the final sum.

Table 4: Comparison of optimized system prompts generated by different meta-optimizers for the BBH Object Counting task. While the lower-capacity model (Gemini 2.0 Flash) defaults to a blunt truncation directive, the high-capacity model (Gemini 3.1 Pro) successfully synthesizes multi-faceted constraints targeting specific vectors of verbosity.

Optimization Setting	Discovered System Prompt (Core Instructions)
Low Capacity Optimizer (Gemini 2.0 Flash, $B=128$ )	Solve the problem. Only provide the answer and do not provide any explanations.
Stochastic Optimization (Gemini 3.1 Pro, $B=1$ )	Solve the problem concisely. Go straight to the answer without restating the problem or adding redundant explanatory sentences.
Optimal Configuration (Gemini 3.1 Pro, $B=128$ )	Solve the problem with extreme conciseness. Do not use conversational filler, restate the question, or write full sentences. Simply provide the final sum.

Table 5: Qualitative ablation of discovered system prompts under different optimization constraints for the BBH dataset. Only the optimal configuration (high capacity, large batch) successfully balances extreme conciseness with the preservation of reasoning.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 370 papers

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
Feb 22, 2026Yinhan He, Yaochen Zhu, Mingjia Shi +5Efficient Language Model Reasoning Language Model Post-Training
University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA
Efficient Reasoning via Constrained Optimization in Latent Space
Sep 28, 2026Zhinan Hou, Xingchen Li, Keyou YouOverthinking in Language Models Efficient Language Model Reasoning
Tsinghua University
On the Token Value Inequality in Efficient Reasoning
Sep 27, 2026Runjia Zeng, Hang Hua, Yiyang Liu +5RL for Language Model Reasoning Efficient Language Model Reasoning
Rochester Institute of Technology · Purdue University · MIT-IBM Watson AI Lab +3

Feb 22, 2026J/K move · Enter open · S save

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning

Yinhan He, Yaochen Zhu, Mingjia Shi, Wendy Zheng+4

University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA

Efficient Language Model Reasoning Language Model Post-Training RL for Language Model Reasoning

Open paper AskPDF arXiv

Large language models increasingly rely on long chains of thought to improve accuracy, yet such gains come with substantial inference-time costs. We revisit token-efficient post-training and argue that existing sequence-level reward-shaping methods offer limited control over how reasoning effort is allocated across tokens. To bridge the gap, we propose IAPO, an information-theoretic post-training framework that assigns token-wise advantages based on each token's conditional mutual information (MI) with the final answer. This yields an explicit, principled mechanism for identifying informative reasoning steps and suppressing low-utility exploration. We provide a theoretical analysis showing that our IAPO can induce monotonic reductions in reasoning verbosity without harming correctness. Empirically, IAPO consistently improves reasoning accuracy while reducing reasoning length by up to 36%, outperforming existing token-efficient RL methods across various reasoning datasets. Extensive empirical evaluations demonstrate that information-aware advantage shaping is a powerful and general direction for token-efficient post-training. The code is available at https://github.com/YinhanHe123/IAPO.

Abstract

Efficient Language Model Reasoning

Explore similar work

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning

Efficient Reasoning via Constrained Optimization in Latent Space

On the Token Value Inequality in Efficient Reasoning