MIRAGE: Multi-Perspective Creative Language Model Reasoning with Reinforcement Learning Guidance

Arash Lagzian; Srinivas Anumasa; Dianbo Liu

cs.CLSep 18, 2026

MIRAGE: Multi-Perspective Creative Language Model Reasoning with Reinforcement Learning Guidance

Authors: Arash Lagzian, Srinivas Anumasa, Dianbo Liu

Organizations: National University of Singapore.

Abstract

Recent advances in Large Language Models (LLMs) have revolutionized artificial intelligence and how human interact with AIs. Despite impressive advancements, LLMs struggle with complex mathematical, scientific, and logical tasks. Inspired by human cognitive flexibility - our ability to dynamically switch mental perspectives - we propose MIRAGE (Multi-perspective Inference-time Reasoning via Agent-Guided Exploration), a novel inference-time creative thinking framework. MIRAGE includes a Selector that prioritizes effective conceptual perspectives (e.g., algebraic, probabilistic) and a Reasoner that sequentially solves tasks until a confident solution emerges, otherwise aggregating multiple perspectives. Tested on GSM8K, MATH500, MMLU-Pro, and Game-of-24 benchmarks, MIRAGE consistently outperforms methods like Chain-of-Thought and diverse prompting ensembles, significantly boosting accuracy with minimal inference overhead, providing a scalable solution for practical applications.

Figures & tables

Figure 1: Overview of our multi-perspective reasoning framework. (a) Standard reasoning uses a single forward pass through the Reasoner model. (b) Our method proceeds in four stages: (1) the Selector ranks conceptual perspectives, (2) the Reasoner model solves each perspective, (3) if confidence exceeds the threshold, the answer is returned, and (4) otherwise, answers are aggregated. These stages are illustrated with numbered circles.

Figure 2: Training curve of the average number of selected perspectives $k$ during REINFORCE training. The dashed blue line shows the 100‐step EMA of $k$ ; the solid orange line is the best‐fit linear trend (slope $\approx-2.02\times 10^{-4}$ ), highlighting a slight but consistent downward drift. Under penalty $\lambda=0.05$ , the selector stabilizes at around eight perspectives per query.

Selector	Avg. $k$	Accuracy (%)	Inference cost
Random-8	8	63.1	$1.0\times$
Full-20	20	74.5	$2.5\times$
RL (ours)	8.1	74.0	$1.0\times$

Table 1: Held-out comparison of selector policies.

Figure 3: Final perspective-Selection Distribution. Normalized usage rates of each conceptual perspective after REINFORCE training ( $\lambda=0.05$ ). The height of each bar is the fraction of queries the selector directed to that perspective; numerical labels show percentage frequency. Note the clear peak at Algebraic and Probabilistic perspective, followed by a long-tail distribution across the remaining 18 perspective.

Base Model	Simple Prompt	CoT	DIPPER (n=3)	DIPPER (n=5)	MIRAGE (avg. $N$ )
DeepSeek-v3	90.83%	89.69%	91.81%	93.48%	89.99% (2.32)
ChatGPT-4o	79.23%	90.67%	82.26%	81.80%	92.04% (1.10)
Claude 3.7-sonnet	71.49%	85.52%	77.56%	77.41%	96.21% (1.07)
Gemini 2.0 Flash 001	91.81%	93.78%	87.87%	88.48%	95.53% (1.02)
Qwen2.5-7B	67.00%	79.80%	79.21%	83.09%	89.01% (1.97)

Table 2: Results on GSM8K. MIRAGE outperforms others with minimal queried perspectives.

Base Model	Simple Prompt	CoT	DIPPER (n=3)	DIPPER (n=5)	MIRAGE (avg. $N$ )
DeepSeek-v3	51.20%	80.00%	83.20%	84.20%	82.40% (1.89)
ChatGPT-4o	31.60%	65.00%	75.60%	75.60%	70.60% (1.30)
Claude 3.7-sonnet	41.40%	64.40%	77.20%	75.80%	77.20% (1.61)
Gemini 2.0 Flash 001	48.20%	80.60%	73.80%	80.20%	84.40% (1.45)

Table 3: Results on MATH500. MIRAGE shows strong gains over baseline methods.

Base Model	CoT	DIPPER (n=3)	DIPPER (n=5)	MIRAGE (avg. $N$ )
DeepSeek-v3	57.25%	52.00%	54.00%	71.00% (4.07)
ChatGPT-4o	37.25%	36.25%	36.25%	61.00% (3.58)
Claude 3.7-sonnet	46.25%	48.25%	48.00%	71.25% (3.19)
Gemini 2.0 Flash 001	55.78%	56.25%	55.20%	83.75% (2.14)

Table 4: Results on MMLU-Pro. MIRAGE demonstrates robustness in general reasoning tasks.

Base Model	CoT	DIPPER (n=3)	DIPPER (n=5)	MIRAGE (avg. $N$ )
DeepSeek-v3	66.40%	71.60%	75.60%	91.20% (1.88)
ChatGPT-4o	62.00%	82.40%	85.60%	76.40% (1.01)
Claude 3.7-sonnet	92.80%	96.40%	96.00%	96.40% (1.36)
Gemini 2.0 Flash 001	90.40%	95.60%	97.60%	99.20% (1.13)

Table 5: Results on Game-of-24. MIRAGE shows exceptional gains and efficiency.

Figure 4: Accuracy vs. computational cost across four benchmarks (GSM8K, MATH500, MMLU-Pro, and Game-of-24). Each point corresponds to a (base model, prompting method) pair. Marker shape denotes the model, while color encodes the prompting strategy. Our method consistently achieves high accuracy with minimal computational cost (Top-Left is better).

Figure 5: Unified accuracy comparison on the GSM8K dataset using the Qwen2.5-7B model. Individual reasoning perspectives (light blue) vary significantly in accuracy. Aggregation across all perspectives (dark blue) significantly surpasses any single perspective. Our Selector-driven approach (orange) outperforms all baseline methods (patterned bars), achieving the highest accuracy with efficient use of computational resources ( $k<2$ ).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

r=\mathbb{I}[\hat{a}=a^{*}]-\lambda\cdot\frac{k}{|\mathcal{S}|}

Appendix

Algorithm 2 REINFORCE Training of the Selector (with Fixed Reasoner)

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

$\nabla$ -Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space
Mar 5, 2026Peihao Wang, Ruisi Cai, Zhen Wang +4LLM Reasoning Test-Time Optimization
The University of Texas at Austin · UC San Diego · TTIC +1
Enhancing reasoning accuracy in large language models during inference time
Mar 22, 2026Vinay Sharma, Manish JainLLM Inference LLM Reasoning
Universe of Thoughts: A Computational Framework for Creative Reasoning in Large Language Models
Nov 25, 2025Yuto Suzuki, Farnoush Banaei-KashaniComputational Creativity Computational Cognitive Modeling
Department of Computer Science and Engineering University of Colorado Denver Denver, CO, USA

Mar 5, 2026J/K move · Enter open · S save

$\nabla$ -Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space

Peihao Wang, Ruisi Cai, Zhen Wang, Hongyuan Mei+3

The University of Texas at Austin · UC San Diego · TTIC+1

LLM Reasoning Test-Time Optimization Efficient Language Model Reasoning Inference-Time Language Model Alignment

Open paper AskPDF arXiv

Scaling inference-time compute for Large Language Models (LLMs) has unlocked unprecedented reasoning capabilities. However, existing inference-time scaling methods typically rely on inefficient and suboptimal discrete search algorithms or trial-and-error prompting to improve the online policy. In this paper, we propose

\nabla

-Reasoner, an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop to refine the policy on the fly. Our core component, Differentiable Textual Optimization (DTO), leverages gradient signals from both the LLM's likelihood and a reward model to refine textual representations.

\nabla

-Reasoner further incorporates rejection sampling and acceleration design to robustify and speed up decoding. Theoretically, we show that performing inference-time gradient descent in the sample space to maximize reward is dual to aligning an LLM policy via KL-regularized reinforcement learning. Empirically,

\nabla

-Reasoner achieves over 20% accuracy improvement on a challenging mathematical reasoning benchmark, while reducing number of model calls by approximately 10-40% compared to strong baselines. Overall, our work introduces a paradigm shift from zeroth-order search to first-order optimization at test time, offering a cost-effective path to amplify LLM reasoning.

Abstract

LLM Reasoning

Explore similar work

∇\nabla∇-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space

Enhancing reasoning accuracy in large language models during inference time

Universe of Thoughts: A Computational Framework for Creative Reasoning in Large Language Models

$\nabla$ -Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space