Temperature-Dependent Performance of Prompting Strategies in Extended Reasoning Large Language Models

Mousa Salah; Amgad Muneer

cs.CLMar 18, 2026

Temperature-Dependent Performance of Prompting Strategies in Extended Reasoning Large Language Models

Authors: Mousa Salah, Amgad Muneer

Organizations: Department of Computer Engineering, Gujarat Technological University, India · The University of Texas MD Anderson Cancer Center, Texas, USA

Abstract

Extended reasoning models represent a transformative shift in Large Language Model (LLM) capabilities by enabling explicit test-time computation for complex problem solving. However, the optimal configuration of sampling temperature and prompting strategy for these systems remains largely underexplored. We systematically evaluate chain-of-thought and zero-shot prompting across four temperature settings (0.0, 0.4, 0.7, and 1.0) using Grok-4.1 with extended reasoning on 39 mathematical problems from AMO-Bench, a challenging International Mathematical Olympiad-level benchmark. We find that zero-shot prompting achieves peak performance at moderate temperatures, reaching 59% accuracy at T=0.4 and T=0.7, while chain-of-thought prompting performs best at the temperature extremes. Most notably, the benefit of extended reasoning increases from 6x at T=0.0 to 14.3x at T=1.0. These results suggest that temperature should be optimized jointly with prompting strategy, challenging the common practice of using T=0 for reasoning tasks.

Figures & tables

Figure 1: Overview of the evaluation methodology. The framework consists of: (Left) Dataset & Input Configuration using the AMO-Bench dataset and Grok-4.1; (Center) Experimental Design featuring a complete factorial evaluation across temperatures and reasoning modes ( $2\times 2\times 4\times 39=624$ tests); and (Right) *Answer Processing & Grading, highlighting a hybrid automated and manual verification process that increased reported results from 22–24% to 29–32%.

Temperature	Correct / Total	Accuracy (%)
0.0 (Deterministic)	49 / 156	31.4%
0.4 (Moderate)	50 / 156	32.1%
0.7 (Moderate)	50 / 156	32.1%
1.0 (High)	46 / 156	29.5%

Table 1: Overall Accuracy by Temperature across all prompting and reasoning configurations.

Temp	No Reasoning	With Reasoning	Multiplier
0.0	9.0% (7/78)	53.8% (42/78)	6.0 $\times$
0.4	7.7% (6/78)	56.4% (44/78)	7.3 $\times$
0.7	9.0% (7/78)	55.1% (43/78)	6.1 $\times$
1.0	3.8% (3/78)	55.1% (43/78)	14.3 $\times$

Table 2: Reasoning Mode Impact by Temperature

Figure 2: The Reasoning Amplification Effect. This bar chart illustrates how the performance benefit of internal reasoning chains scales with sampling randomness. While extended reasoning provides a 6.0

\times

improvement at

T=0.0

, the multiplier surges to 14.3

\times

at the high-diversity limit (

T=1.0

). This suggests that internal reasoning mechanisms act as critical logical guardrails, effectively harnessing rather than degrading under temperature-induced exploration.

Temperature ( $T$ )	Zero-Shot	Chain-of-Thought	$\Delta$ (pp)
0.0 (Deterministic)	53.8% (21/39)	53.8% (21/39)	0.0
0.4 (Moderate)	59.0% (23/39)	53.8% (21/39)	+5.2
0.7 (Moderate)	59.0% (23/39)	51.3% (20/39)	+7.7
1.0 (High)	51.3% (20/39)	59.0% (23/39)	-7.7

Table 3: Performance with Extended Reasoning Enabled. Peak values for each strategy are bolded.

Figure 3: The interaction between prompting strategy and temperature. Note the strategy reversal: Zero-shot achieves peak efficiency at moderate temperatures (

T=0.4

–

0.7

), while CoT performs best at the high-diversity extreme (

T=1.0

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 429 papers

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve
Aug 4, 2026Denys Pushkin, Albert Q. Jiang, Aryo Lotfi +2LLM Prompting CoT Reasoning
EPFL · Apple · Mistral AI
A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting
Oct 21, 2026Ibrahim Khan, Thai Van Nguyen, Yi Xia +5LLM Prompting Adaptive Reasoning in Language Models
College of Information Science and Technology, Ritsumeikan University, Japan · Graduate School of Information Science and Technology, Ritsumeikan University, Japan · Independent Researcher, and an alumnus of the Graduate School of Information Science and Technology, Ritsumeikan University, Japan +1
Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models
Apr 14, 2026Xudong Wang, Chaoning Zhang, Chenghao Li +10Automatic Prompt Optimization LLM Prompting
Kyung Hee University · University of Electronic Science and Technology of China · Nota Inc. +1

Aug 4, 2026J/K move · Enter open · S save

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve

Denys Pushkin, Albert Q. Jiang, Aryo Lotfi, Colin Sandon+1

EPFL · Apple · Mistral AI

LLM Prompting CoT Reasoning Few-Shot Prompting LLM Mathematical Reasoning

Open paper AskPDF arXiv

Chain-of-Thought (CoT) prompting remains the standard baseline for evaluating models' reasoning abilities. Originally, this technique was introduced to elicit step-by-step reasoning from large language models (LLMs), which would otherwise tend to directly output the final answer. However, many modern LLMs produce CoT-style responses natively when presented with reasoning tasks, which made us revisit the effectiveness of standard CoT prompting. We evaluate several modern mid-sized language models on a math problem-solving task and find that models specialized for reasoning achieve better performance in a simple zero-shot setting than when using few-shot CoT examples - significantly surpassing officially reported results at no additional cost (e.g., from

\sim

77% to

\sim

84% for Mathstral on GSM8K). For the tested general-purpose model, a zero-shot CoT prompt is also sufficient to outperform a few-shot CoT baseline. We attribute this to a `guidance-distraction' tradeoff: standard CoT prompting also demands style adaptation, formatting compliance, and potentially undesired contextualization, which can distract models from the core reasoning task. Our findings suggest that using standard CoT prompting increasingly acts as a source of distraction as models grow stronger.

Abstract

LLM Prompting

Explore similar work

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve

A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting

Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models