MIRA: Reinforcing Multimodal Reasoning via Deceptive Contextual Augmentation

Zhihan Yin; Jianxin Liang; Yifeng Yao; Nonghai Zhang; and Bingyue Peng; Chunguang Qie; Huishuai Zhang; Dongyan Zhao

cs.CVNov 2, 2026

MIRA: Reinforcing Multimodal Reasoning via Deceptive Contextual Augmentation

Authors: Zhihan Yin, Jianxin Liang, Yifeng Yao, Nonghai Zhang, and Bingyue Peng, Chunguang Qie, Huishuai Zhang, Dongyan Zhao

Organizations: Peking University · ByteDance · Wangxuan Institute of Computer Technology · National Engineering Research Center of New Electronic Publishing Technologies

Abstract

While Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced multimodal reasoning, existing frameworks suffer from a critical "perception-reasoning gap." Due to an over-reliance on seemingly relevant visual tokens, even minor perceptual perturbations can propagate through the reasoning process, leading to compounding hallucinations. To address this issue, we propose MIRA, a training framework that improves robustness to erroneous visual contexts while enhancing logical reasoning ability. By injecting filtered, deceptive visual contexts during training, we construct challenging "hard traps" that stress-test the model's robustness to misleading information. Furthermore, we introduce a group-wise reflection-triggering mechanism that enables the model to autonomously detect and rectify inconsistencies between visual cues and logical constraints, which is ultimately internalized as an intrinsic policy behavior. Importantly, MIRA does not rely on human annotations. Experimental results across multiple reasoning and perception benchmarks show that MIRA yields an average 7% improvement on in-domain benchmarks, with consistent improvements on vision-centric benchmarks. These findings highlight that active error recovery is essential for robust multimodal reasoning. Comprehensive ablation studies and analyses further provide insights into why MIRA is effective.

Figures & tables

Figure 1: Overview of MIRA. MIRA consists of two parts. The top illustrates the construction of deceptive visual context using the policy model and an expert model. The bottom shows the training process; compared to GRPO, the key changes are modified inputs and the use of intra-group triggers.

General-purpose Models
	Math Benchmarks				General / Visual Benchmarks			Avg.
	MathVista	MathVerse	MathVision	WeMath	MMMUpro	HalluBench	FREAK	Avg.
GPT-4o	62.6	50.4	31.6	67.6	53.2	73.5	33.4	53.2
InternVL2.5-8B	59.6	34.8	19.2	56.3	33.2	67.5	29.7	42.9
InternVL3-8B	71.6	47.7	29.5	62.9	41.9	67.5	29.7	50.1
Qwen2.5-VL-7B	68.2	46.7	24.3	61.4	38.5	65.7	30.0	47.8

Table 1: Evaluation results of different reasoning methods. All results are reproduced using our evaluation scripts. Detailed evaluation settings are provided in Appendix D . Results of Vision-R1 on MathVision and WeMath are omitted due to potential training data contamination. Best (green) and second-best (blue) results among open-source multimodal reasoning methods are highlighted. Averages are computed over available benchmarks, excluding missing entries.

Backbone	Method	MathVista	MathVerse	MathVision	WeMath
Qwen2-VL-7B	GRPO	62.0	35.6	18.9	48.1
Qwen2-VL-7B	MIRA	63.5	36.5	19.1	49.5
Qwen3-VL-4B	GRPO	77.1	59.6	45.4	75.6
Qwen3-VL-4B	MIRA	77.8	61.6	47.0	78.7
Qwen3-VL-8B	GRPO	78.0	66.1	51.1	79.1
Qwen3-VL-8B	MIRA	79.9	66.1	52.9	81.2

Table 2: Results on additional backbones (%). Each MIRA/GRPO pair uses the same source data and training configuration.

Expert Model Ablation (Expert Type)
Configuration	MathVista	MathVerse	MathVision	WeMath	Avg.
GPT 5.4	75.2	53.8	29.0	70.7	57.18
GPT 5.4-mini	74.3	52.5	29.7	69.6	56.53
GPT 5.1-mini	74.2	53.1	29.0	69.0	56.33
Data Ratio Sensitivity (Augmentation Ratio)
0 (GRPO ∗ )	71.5	51.0	28.4	68.0	54.7

Table 3: Comparison across benchmarks with different expert models and data ratio settings. Data ratio sensitivity experiment is conducted with 14K dataset. Refer Appendix D.3 for details.

Setting	MathVista	MathVerse	WeMath	MathVision	Avg.
MIRA (full)	75.2	53.8	70.7	29.0	57.2
w/o correction reward	73.8	53.2	70.7	29.0	56.7
+ full trigger	73.6	52.6	68.8	28.4	55.8
- remove trigger	70.9	51.8	68.8	29.0	55.1
- remove context (GRPO)	72.9	51.5	68.8	29.3	55.6

Table 4: Ablation study on reward design and trigger mechanisms.

Figure 3: Comparison between vanilla GRPO and MIRA. Left: training accuracy comparison; Middle: ratio of groups that the variance is 0; Right: accuracy comparison between normal samples and augmented samples. the reported result of MIRA is trained using GPT-5.4 as the expert model.

Figure 4: Training case of MIRA. Notice that in this case no triggers are inserted. The model autonomously corrects both erroneous context and reasoning errors. Incorrect and correct parts are highlighted in red and green, respectively.

(a) Controlled inputs		Recall	SRR	Aug. acc.	Normal acc.
Base		27.42	36.33	19.48	43.51
MIRA		42.96	59.54	50.66	68.69
(b) Benchmark	Method	Accuracy	RR	SRR	FCR
MathVista	GRPO	72.9	4.1	85.4	0.30 (3/1000)
MathVista	MIRA	75.2	5.4	66.7	0.20 (2/1000)
MathVerse	GRPO	51.5	8.9	57.8	0.30 (12/3940)

Table 6: Quantitative reflection results on controlled and real inputs (%).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Baseline	Venue	Data Scale
R1-Onevision [ Yang et al., 2025b ]	ICCV’2025	R1-Onevision-155K
Vision-R1 [ Huang et al., 2026 ]	ICLR’2026	200K SFT + 10K RL
MM-Eureka [ Meng et al., 2025 ]	-	MMK12 (15.6K)
PAPO [ Wang et al., 2026b ]	ICLR’2026	ViRL-39K
Perception-R1 [ Xiao et al., 2026 ]	ICLR’2026	Geometry3K
SophiaVL-R1 [ Fan et al., 2026 ]	ICLR’2026	SophiaVL-130K

Appendix

Table 7: Summary of baselines, including venue, data scale, and data source.

Figure 5: The statistics of training data.

Training Configuration		RL Configuration
Training epochs	2	Rollout number	8
Gradient accumulation steps	4	KL Coefficient	0.004
Effective batch size	128	Rollout Engine	vLLM
Sequence length cutoff	5,000	Temperature	1.0
Optimizer	AdamW	Trigger Configuration
Learning rate	$1\times 10^{-6}$	Rollouts with trigger $K$	4

Appendix

Table 8: Training hyperparameters we used for main results.

Metric	No trigger	Seven triggers (mean $\pm$ std.)
Reflection rate	42.2	$90.36\pm 3.22$
SRR	39.8	$\boldsymbol{56.64\pm 1.07}$
Accuracy	42.2	$\boldsymbol{54.73\pm 1.18}$

Appendix

Table 9: Trigger wording sensitivity on 500 augmented examples (%). Variability is across seven English phrasings.

Figure 6: The pre-defined triggers set of intra-group trigger.

Figure 7: The prompt we used for expert modification in data construction.

Figure 8: The system prompt we used for training.

Figure 9: The system prompt we used for evaluation.

Reasoning for VLMs

See all 85 papers

IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
May 18, 2026Chenghao Li, Fusheng Hao, Xikai Zhang +5Multimodal Grounding Object Hallucination in VLMs
Hangzhou International Innovation Institute, Beihang University · Shenzhen Institute of Advanced Integration Technology, Shenzhen · Kuaishou Technology +1
PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
Jun 7, 2026Qiming Li, Tianlun Li, Xiaolong Cheng +5Token-Level Credit Assignment VLM Reasoning
Amap CV Lab, Alibaba Group · Peking University
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
May 21, 2026Changyuan Tian, Zhicong Lu, Huaxing Liu +7Multimodal Large Language Models Multimodal Reasoning
AMAP, Alibaba Group · University of Chinese Academy of Sciences · Tsinghua University +1

May 18, 2026J/K move · Enter open · S save

IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning

Chenghao Li, Fusheng Hao, Xikai Zhang, Likang Xiao+4

Hangzhou International Innovation Institute, Beihang University · Shenzhen Institute of Advanced Integration Technology, Shenzhen · Kuaishou Technology+1

Multimodal Grounding Object Hallucination in VLMs Multimodal Reasoning RL for VLMs

Open paper AskPDF arXiv

Multimodal large language models via reinforcement learning (RL) have demonstrated remarkable capabilities in complex visual reasoning tasks, yet they remain limited in long-horizon multimodal scenarios, often suffering from visual hallucination and logical error. Current methods typically pre-encode high-dimensional visual scenes into discrete textual proxies to facilitate downstream reasoning. As the reasoning chain unfolds, however, the inherent information asymmetry between text and visual scenes tends to erode visual grounding, resulting in misguided reasoning and erroneous outputs. To address this issue, we introduce IVR-R1 (Iterative Visual-grounded Reasoning), a novel RL training framework that facilitates dynamic visual re-alignment that actively rectifies reasoning trajectories to guide policy optimization. Specifically, by leveraging a reward-driven screening mechanism to identify flawed rollouts, IVR-R1 executes a fine-grained, step-level error attribution within the multimodal context. By iteratively cross-referencing intermediate reasoning states against pristine visual priors, a Re-Reasoning Loop enables automated trajectory rectification, effectively synthesizing expert-level demonstrations that serve as high-fidelity reasoning templates for the policy model. Our experiments across diverse multimodal benchmarks demonstrate that IVR-R1 consistently outperforms existing reinforcement learning methods, establishing a superior paradigm for maintaining logical and visual consistency in complex multimodal reasoning.

Abstract

Reasoning for VLMs

Explore similar work

IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention