Same Task, Different Statement: When and How Reformulation Helps LLM Problem Solving
Authors: Wentao Zhang, Xiaoyu Zhang, Chengwei Liu, Yutao Hu, Tao Zhang, Junsheng Wu, Hongping Gan, Yang Liu
Organizations: Northwestern Polytechnical University · CCDS, Nanyang Technological University · Nankai University · Huazhong University of Science and Technology
When people face a hard problem, they often restate it in a clearer way before solving it. We study whether the same habit helps large language models by rewriting the statement of a problem. A legal rewrite must keep the meaning, the constraints, and the correct answer unchanged. We test three kinds of rewrites. Decomposition makes hidden subgoals explicit. Typed Reformulation casts the task into a more suitable solving frame. Concept Reduction removes surface detail and exposes the task structure. We evaluate them on eight math, logic, and code benchmarks with eight models, all under the same chain-of-thought solving prompt. The effect is large but depends on the problem type. An oracle that picks the best rewrite after the fact raises accuracy by +27.6% on average. Yet no single rewrite helps every problem, and a mismatched rewrite can hurt. Still, the effect follows clear patterns. Each kind of rewrite fits certain problem types. A simple fixed rule learned from one set of problems transfers part of the gain to new problems. It gains +11.0% on the problems it chooses to rewrite, while blind rewriting loses 5.6% on a similar share of problems. Rewriting is therefore not a universal way to solve problems. The question is when and how to rewrite.
Figures & tables
Figure 1: A rewrite can unlock a solution the original statement misses. Left: a human solver first rewrites a logic story as symbols and rules, and only then solves it. Right: Concept Reduction applies the same habit to an LLM input. The reformulated statement flips GPT-4o-mini from a wrong valid to a correct invalid , while the task and its correct answer are unchanged.
Figure 2: Task-preserving statement reformulation. Rewrites map (x,g) to (x′,g′) under shared legality constraints, using three controlled families: Decomposition, Typed Reformulation, and Concept Reduction. The evaluation target a is held out until evaluation.
Figure 3: Study design. From eight public benchmarks we form a typed pool and sample a Train Set and a disjoint Test Set. Candidate reformulations pass a task-preservation audit. Evaluation uses eight LLMs with a shared zero-shot CoT prompt.
Source Type
Typing Pool
Train Set
Held-out Test
Search
489
129
144
Classification
411
125
138
Computation
3,980
167
197
Explanation
498
129
144
Causality
190
110
80
Optimization
205
112
93
Table 1: Source-type distribution after 3-run consistency classification and log-weighted stratified sampling. Types are operational labels for the stem’s main demand: Search (locate/filter), Classification (choose among options), Computation (calculate), Explanation (account for a fact), Causality (judge cause), Optimization (best choice under constraints), Planning (schedule under constraints), Design (construct an artifact), Verification (check correctness).
Slice
n
Agreement
κ
All
240
95.0%
0.72
C
80
91.2%
0.34
D
80
100.0%
1.00
TR
80
93.8%
0.82
Table 2: Dual-annotator legality labels on 240 Train Set candidates.
Concept Reduction
Decomposition
Typed Reformulation
MS-Best
Oracle Selection
Model
Orig
C-Best
C-Oracle
D-Best
D-Oracle
TR-Best
TR-Oracle
Acc
Δ
Sel.
Δ
GPT-4o-mini
43.7
55.1
58.0
63.0
67.3
58.5
60.0
63.0
+19.3
80.7
+37.0
GPT-5-mini
63.4
72.9
77.9
64.4
73.1
67.1
80.5
72.9
+9.5
86.4
+23.0
Qwen3-Flash
58.4
72.8
76.4
63.2
69.0
60.5
76.6
72.8
+14.4
85.4
+27.0
Qwen3-Plus
62.8
72.7
76.3
64.0
70.6
63.6
79.0
72.7
+9.9
84.4
+21.6
DeepSeek-V4-Flash
59.5
71.0
75.4
60.8
68.3
61.5
76.9
71.0
+11.5
84.6
+25.1
Table 3: RQ1 main results on the Train Set. (Metrics follow Section 3.5 . Sel. is Oracle Selection. Both Δ columns are vs. Orig. Best and Oracle Selection are retrospective ceilings.)
Setting
Acc (%)
Orig
43.7
pass@ 3
50.3
pass@ 6
54.8
Paraphrase oracle
60.3
Family ceiling
76.0
Oracle Selection ( Table 3 )
80.7
Table 4: RQ1 matched-budget controls on GPT-4o-mini. (Under pass@ k , an item is correct if any of k independent Orig solves is correct. The paraphrase oracle uses the original statement and five free paraphrases, while the family ceiling uses the original statement and all stored C/D/TR reformulations.)
Figure 4: RQ1 accuracy (%) on the Train Set under Orig, MS-Best, and Oracle Selection. Weaker models, including open ones such as Llama-3.1-8B, already approach or pass the GPT-5-mini Orig baseline under MS-Best. Oracle Selection lifts every model well above that line, so the presentation ceiling far exceeds a strong closed model’s original accuracy. The dashed line marks GPT-5-mini Orig (63.4%) as a reference.
Figure 5: RQ2 pooled ΔAcc (%) by source type and family. (Rows are source types, and columns are families. ⋆ marks Rec ( Section 5.2 ). Red-toned cells indicate harm.)
Full-set Acc (%)
Δ Acc on acted-upon
Model
Orig
Blind
Random
Active
Blind
Random
Active
GPT-4o-mini
58.5
58.6
61.2
60.2
− 2.4
+10.9
+9.5
GPT-5-mini
68.8
65.8
67.7
68.9
− 15.4
+0.5
+2.0
Qwen3-Flash
64.9
63.3
63.8
66.3
− 10.6
+1.0
+7.0
Qwen3-Plus
69.0
69.4
68.3
70.5
− 0.5
− 2.1
+8.0
DeepSeek-V4-Flash
66.7
65.9
62.5
68.0
− 1.9
− 0.5
+6.5
Table 5: RQ3 on the Test Set. (Left block: full-set accuracy (%). Right block: Acc(method) − Acc(Orig) only on problems that method reformulates (acted-upon; Blind n=208 , 19.5%; Random n=193 ; Active n=199 , 18.7%.)
Data contamination undermines the reliable evaluation of large language models (LLMs) on mathematical problem solving. While rewriting-based evaluation mitigates memorization, existing methods lack guarantees of problem validity and answer correctness. We propose Proof-Verified Benchmark Rewriting (RePro), the first framework to integrate Lean-oriented neural automated theorem provers (ATPs) into benchmark rewriting, which rewrites problems and regenerates answers with correctness ensured by Lean-verified proofs. Experiments on GSM8K and MATH show that RePro's retained rewritten instances achieve 100% well-definedness, feasibility, and answer correctness, while existing methods still produce invalid or incorrect instances. Moreover, several models exhibit accuracy drops on proof-verified rewritten benchmarks, suggesting that their performance is sensitive to surface-level and structural variations and may partly reflect memorization effects. Our source code and data are available at https://github.com/AI4Engi/RePro.
Xiyuan Zhou, Zhuoqi Li, Xinlei Wang +6
Nanyang Technological University · The Chinese University of Hong Kong, Shenzhen · INSAIT, Sofia University “St. Kliment Ohridski” +2
Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential, which can be a barrier for non-expert users. This work addresses the challenge by introducing a Task-Aware Prompt Rewriter (TAPR), a model that reformulates user prompts into task-optimized prompts with the explicit goal of improving downstream LLM performance. We train TAPR using reinforcement learning with Group Relative Policy Optimization (GRPO), where rewards are derived from LLM-as-judge evaluations of both the reformulated prompt and the corresponding task output. Experimental results on diverse tasks, such as question answering, summarization, and arithmetic reasoning, show that our method yields consistent gains over base models in prompt rewriting ability. Fine-tuning Phi-4-mini-instruct (as the base model for TAPR) produces prompts that contain clearer and more instructive language, leading to higher accuracy on established benchmarks such as Natural Questions and GSM8K. Our code is available at: https://github.com/OliverSavolainen/task-specific-prompt-rewriter
Oliver Savolainen, Emanuele Bastianelli, Hosein Azarbonyad
University of Amsterdam, Amsterdam, The Netherlands · 2Elsevier, Amsterdam, The Netherlands
Prompt-only, single-step LLM query rewriting, where a rewrite is generated from the query alone without retrieval feedback, is commonly used in production RAG pipelines, but its effect on dense retrieval is poorly understood. We present a systematic empirical study across three BEIR benchmarks, two dense retrievers, and multiple training configurations, and find strongly domain-dependent behavior: rewriting degrades nDCG@10 by 9.0 percent on FiQA, improves it by 5.1 percent on TREC-COVID, and has no significant effect on SciFact. We identify a consistent mechanism: degradations co-occur with reduced lexical alignment between rewritten queries and relevant documents, as rewriting replaces domain-specific terms in already well-matched queries. In contrast, improvements arise when rewriting shifts queries toward corpus-preferred terminology and resolves inconsistent nomenclature. Lexical substitution occurs in 95 percent of rewrites across all outcome groups, showing that effectiveness depends on the direction of substitution rather than substitution itself. We also study selective rewriting and find that simple feature-based gating can reduce worst-case regressions but does not reliably outperform never rewriting, with even oracle selection offering only modest gains. Overall, these results show that prompt-only rewriting can be harmful in well-optimized verticals and suggest that domain-adaptive post-training is a safer strategy when supervision or implicit feedback is available.