How carefully and unambiguously a question is phrased has a profound impact on the quality of the response, for Language Models (LMs) as well as people. While model capabilities continue to advance, the interplay between grounding context and query formulation remains under-explored. This work investigates how the quality of background grounding information in a model's context window affects accuracy. We find that combining well-grounded dynamic context construction (i.e, RAG) with query rewriting reduces question ambiguity, resulting in significant accuracy gains. Given a user question with associated answer-free grounding context, rewriting the question to reduce ambiguity produces benchmark improvements without changing the answer itself, even compared to prepending that context before the question. Using gpt-oss-20b to rewrite a subset of Humanity's Last Exam using answer-free grounding context improves gpt-5-mini accuracy from 0.14 to 0.37. We demonstrate that this accuracy improvement cannot be fully recovered just through prompting at inference time; rather, distinct rewriting and answering phases are required. Code and data are available at https://github.com/mmajurski/lm-rewrite-uplift
Figures & tables
Figure 1: When RAG systems surface relevant information, LM performance can be enhanced by rewriting the initial query using context —added information that, without providing the answer, gives relevant background knowledge and direction.
Figure 2: Rewriting questions using answer-free grounding context yields significant accuracy improvement (distance above line) over the original questions, as evaluated on a subset of Humanity’s Last Exam (HLE). See Figure 4 for a complete legend.
Figure 3: The quality of context presented to an LM has an impact on question-answering performance. Accuracy is highest when RAG systems surface context containing the answer ( cyan ), but when the question is presented without context ( orange ) or the surfaced information does not contain the answer ( green ), performance (without query rewriting) suffers. The act of interpretation during question rewriting ( pink ) produces an accuracy improvement beyond just prepending the Question + AnswerFree Context information used to rewrite before the question; despite not including the AFC when asking an LM the rewritten question.
Dataset
Release Date
Public Datasets
Humanity’s Last Exam [ 18 ]
2025
Squadv2 [ 19 ]
2018
HotpotQA [ 32 ]
2018
TrivaQA-web [ 11 ]
2017
NaturalQuestionsShort [ 13 ]
2019
Table 1: Evaluation Datasets and Publication Dates
Model
Knowledge-Cutoff
Public-Release
gpt-5
Sep 2024
Aug 2025
gpt-5-mini
May 2024
Aug 2025
gpt-5-nano
May 2024
Aug 2025
gpt-oss-20b
Jun 2024
Aug 2025
gpt-oss-120b
Jun 2024
Aug 2025
gemma-3-1b-it
Aug 2024
Mar 2025
Table 2: Evaluation Models Knowledge Cutoff
Figure 4: Performance of various LMs with and without answer-containing context and answer-free context. The x -axis shows the original question benchmark accuracy while the y -axis shows the rewritten question benchmark accuracy; distance above the line conveys improvement. Colors represent the different models tested ( table 2 ) and shapes represent the dataset ( table 1 ). (top left) Answers are present in the context (baseline) : Benchmark performance significantly improves with the addition of context that contains the answer. (top right) Answers not present in the context : Simply providing relevant answer free context without question rewriting does not improve performance. (bottom left) Questions rewritten using answer-free context (our approach) : Improvement in accuracy caused by rewriting the question only using AFC where during benchmarking only the rewritten question is presented to the LM ( AFC is withheld).
Figure 5: Per-dataset per-model difference in benchmark accuracy between the rewritten questions and the original questions with associated answer-free context. The violin plot distribution highlights the range of accuracy deltas over all datasets for each model evaluated. Benchmark accuracy improved by an average of 0.1346.
Figure 6: Per-dataset difference in benchmark accuracy between the rewritten questions without AFC and the original questions with AFC . The violin plot distribution highlights the range of accuracy deltas over all models for each dataset evaluated. Benchmark accuracy improved by an average of 0.1346.
Figure 7: Per-dataset improvement in benchmark accuracy from Rewrite_Q+AFC compared to the Orig_Q+AFC during benchmark evaluation. Benchmark accuracy improved by an average of 0.0875.
Figure 8: Accuracy improvement on the x -axis vs. improvement in cosine similarity between the question and context for rewritten questions on the y -axis. x -axis: delta benchmark accuracy Rewrite_Q − Orig_Q ; larger values indicate that the rewritten question improved benchmark accuracy. y -axis: increase in cosine similarity between the question and context due to rewriting; larger values indicate better alignment between Rewrite_Q and AFC compared to Orig_Q and AFC . The fact that most points are in the upper right quadrant demonstrates that improvements in embedding alignment between the question and grounding context correlate with improved benchmark accuracy.
Figure 9: Per-dataset improvement in benchmark accuracy from performing an in situ rewrite of the question using answer-free context during benchmark evaluation. This combines the rewrite-then-answer method into a single operation. The prior accuracy improvement disappears, highlighting the impact of task separation between the rewrite and answer phases.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Per-dataset per-model difference in benchmark accuracy between the rewritten question and the original question. The violin plot distribution highlights the range of accuracy deltas over all datasets for each model evaluated. Benchmark accuracy improved by an average of 0.1303.
Figure 11: Per-dataset per-model difference in benchmark accuracy between the rewritten question and the original question. The violin plot distribution highlights the range of accuracy deltas over all datasets for each model evaluated. Benchmark accuracy improved by an average of 0.1303.
Figure 12: Per-model difference in benchmark accuracy between the rewritten questions and the original questions with associated answer-free context. The violin plot distribution highlights the range of accuracy deltas over all datasets for each model evaluated. Benchmark accuracy improved by an average of 0.1346.
Figure 13: Per-dataset per-model difference in benchmark accuracy between the rewritten questions and the original questions with associated answer-free context. The violin plot distribution highlights the range of accuracy deltas over all models for each dataset evaluated. Benchmark accuracy improved by an average of 0.1346.
Figure 14: Per-dataset improvement in benchmark accuracy from Rewrite_Q with AFC compared to the Orig_Q with AFC during benchmark evaluation. Benchmark accuracy improved by an average of 0.0875.
User queries are often underspecified and may admit multiple valid interpretations. Rather than silently making assumptions about the user's intent, a helpful assistant should surface such ambiguity by asking a clarifying question. Doing so requires two abilities: recognizing that a query is ambiguous, and acting on that recognition by seeking clarification instead of answering directly. To study these abilities, we evaluate models on ambiguous, unambiguous, and disambiguated questions in three settings: standard question answering, explicit ambiguity judgment, and behavioral analysis, where a judge model classifies responses as direct answers, refusals, or clarifying questions. We find a clear gap between recognition and behavior: models often identify ambiguity when explicitly asked to judge it, yet in the QA setting they overwhelmingly default to direct answers. Retrieved context further widens this gap by improving answerability while making models even less likely to ask clarifying questions.
Long-context language models can increasingly process entire documents, making full-context prompting a practical alternative to passage retrieval. However, greater capacity determines how much information a model can receive, but not which information best supports a query. This distinction motivates rethinking retrieval: when the full document already fits, can selecting and organizing its evidence improve answering? The challenge is to retain supporting facts and relations while keeping the context focused on the question. To reconcile these goals, we introduce \ours, a training-free framework that uses semantic abstraction to guide selection while retaining original text for answering. \ours constructs a reusable semantic hierarchy from the full document in a single generation, linking node descriptions to source spans. Given a query, structural--semantic scoring and fine-grained lexical attribution nominate shared source nodes; their union with ancestor context preserves structural relations and complete local text. Across four frontier answer models, \ours improves average QA-F1 by 1.72 points over full-context prompting on three LongBench tasks, surpassing the evaluated baselines on the four-model average while reducing answer-context length by 58.55%. Our code is available at the repository.
Large language models consistently fail the "car wash problem," a viral reasoning benchmark requiring implicit physical constraint inference. We present a variable isolation study (n=20 per condition, 6 conditions, 120 total trials) examining which prompt architecture layers in a production system enable correct reasoning. Using Claude 3.5 Sonnet with controlled hyperparameters (temperature 0.7, top_p 1.0), we find that the STAR (Situation-Task-Action-Result) reasoning framework alone raises accuracy from 0% to 85% (p=0.001, Fisher's exact test, odds ratio 13.22). Adding user profile context via vector database retrieval provides a further 10 percentage point gain, while RAG context contributes an additional 5 percentage points, achieving 100% accuracy in the full-stack condition. These results suggest that structured reasoning scaffolds -- specifically, forced goal articulation before inference -- matter substantially more than context injection for implicit constraint reasoning tasks.