The Image Still Has a Say: Final-Answer Repair for Multimodal Chain-of-Thought

Yulan Pan; Xiaoyu Xu; Zhihong Shen

cs.CLOct 13, 2026

The Image Still Has a Say: Final-Answer Repair for Multimodal Chain-of-Thought

Authors: Yulan Pan, Xiaoyu Xu, Zhihong Shen

Organizations: Fuzhou University

Abstract

Although chain-of-thought prompting has improved multimodal reasoning, it is still difficult to produce correct final answers, as the reasoning can drift away from the image. Besides, the trace is already fixed when the answer is generated, leaving no room for self-correction. Considering the image is still available at the final-answer step, we think it is reasonable to fix the answer there without rewriting the reasoning. As such, the problem becomes to correct the final answer using the image under fixed reasoning. Unlike methods that improve trace generation through training, resampling, or decoding-time grounding, we repair the final-answer readout while keeping the reasoning trace fixed. The motivation is based on our finding that a wrong option can become prematurely committed in the reasoning while the correct option still has stronger image support in middle-to-deep layers. Thus, we propose Text-to-Visual Evidence Reallocation (TVER), a training-free method for final-answer repair. Specifically, TVER selects answer-related text and question-matched image patches, and moves bounded attention from text to patches. Furthermore, the patches are chosen in a middle-to-deep layer window based on question relevance. Our method improves answers across benchmarks and backbones, with gains in both multiple-choice and open-answer evaluations.

Figures & tables

Figure 1: The problem and our main idea. Prior training or decoding-time methods change the reasoning trace before the final answer is read. TVER keeps the trace fixed and changes only the final-answer readout by reallocating attention from answer-related text in the fixed context to selected visual evidence.

Figure 2: Motivating diagnostic for final-answer repair. The wrong option becomes stable before the answer line, while the correct option keeps larger visual lift and stronger image-to-answer flow in middle-to-deep layers.

Figure 3: Overview of TVER . The image, question, options, and CoT prefix are fixed. TVER selects a text-side source and a visual evidence target from the unmodified readout, then reruns only the final-answer line with capped attention reallocation.

Method	Visual Grounding		General VQA		Reasoning
	TreeBench $\uparrow$	V*Bench $\uparrow$	MathVista $\uparrow$	MMStar $\uparrow$	RH-Reason $\uparrow$
Qwen2.5-VL-7B	37.0	76.4	68.2	63.9	39.7
+ VCD	40.5	78.0	69.0	65.0	41.3
+ PAI	38.3	77.0	68.5	64.4	42.3
+ VAR	39.5	78.5	68.9	64.8	41.3
+ VISTA	41.0	79.1	69.4	65.4	42.0

Table 1: Performance comparison under the fixed-prefix final-answer protocol. All rows use the same answer extraction or judge, and TVER uses one fixed setting across all model-task cells. Highlighted rows are TVER ; underlines mark the strongest non- TVER result in each model-task group; arrows show gains over CoT, computed from unrounded scores.

Figure 4: Composability of VISTA and TVER on Qwen2.5-VL-7B and InternVL3-8B. The x-axis lists the five benchmarks, and the y-axis shows accuracy gains over CoT in percentage points. VISTA and VISTA+ TVER share the same VISTA-generated trace and differ only at the final-answer readout; the combination gives higher gains than either method alone in all ten settings.

Part	Setting	Acc.	$\Delta$
Base	CoT	50.6	–
Control	re-answer	51.0	+0.4
Control	answer-text downweight	51.6	+1.0
Control	release-only	52.0	+1.4
Source	recent-only	51.8	+1.2
Source	answer text	52.6	+2.0

Table 2: Module ablations on matched MCQ rows. Source rows fix the target as VE-25% ; target rows fix the source as answer text+recent. Answer-text downweight scales source attention without adding it to image tokens; release-only removes the released mass from the source and renormalizes the row; task-only ranks image tokens by question–option relevance without the broad-attention penalty. Highlighted rows are the settings used by TVER , and

\Delta

is the accuracy gain over CoT.

Method	Compute	Acc.	$\Delta$
CoT	$1.0\times$	37.0	–
VCD	$\sim 1.2\times$	40.5	+3.5
SC ( $k=2$ )	$\sim 2.0\times$	40.0	+3.0
SC ( $k=4$ )	$\sim 4.0\times$	40.7	+3.7
VISTA steering	$\sim 1.0\times$	41.0	+4.0
TVER	$\bm{\sim 1.3\times}$	41.7	+4.7

Table 3: Compute controls on TreeBench. Cost is relative to one CoT and counts extra scoring, attention collection, or decoding;

\Delta

is over CoT.

Figure 5: Qualitative example on a TreeBench perspective question. The panels show the input, the original CoT, the selected text-side source and visual target, and the final score shift from A to B after TVER .

Factor	Setting	Acc.	$\Delta$
Base	CoT	50.6	–
Target	VE-10%	53.1	+2.5
Target	VE-25%	53.8	+3.2
Target	VE-50%	53.4	+2.8
Selector	all-layer avg.	53.4	+2.8
Selector	middle avg.	53.5	+2.9

Table 4: Setting robustness on Qwen2.5-VL-7B tuning rows. Highlighted rows are final settings;

\Delta

is over CoT.

Multimodal CoT Reasoning

Multimodal chain-of-thought reasoning studies how AI models combine visual, audio, and textual evidence through intermediate reasoning to solve complex tasks.

See all 143 papers

ETCHR: Editing To Clarify and Harness Reasoning
May 22, 2026Beichen Zhang, Yuhong Liu, Jinsong Li +3Tool Use in VLMs Multimodal Reasoning
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University +2
When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
Aug 5, 2026Yongxin Wang, Ruizhe Zhou, Yueling Tang +4Multimodal QA Multimodal Reasoning
Mohamed bin Zayed University of Artificial Intelligence · Sun Yat-sen University · Transsion +1
Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning
Jul 24, 2026Hao Yang, Jin Wang, Xuejie ZhangAttention-Guided Distillation VLM Distillation
School of Information Science and Engineering, Yunnan University, Kunming, China

May 22, 2026J/K move · Enter open · S save

ETCHR: Editing To Clarify and Harness Reasoning

Beichen Zhang, Yuhong Liu, Jinsong Li, Yuhang Zang+2

The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University+2

Tool Use in VLMs Multimodal Reasoning Multimodal CoT Reasoning Visually Grounded Reasoning Multimodal Foundation Models Image Editing+1

Open paper AskPDF arXiv

Multimodal Large Language Models have advanced visual reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ''think with images'' paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as reasoning assistants with two complementary gaps: a language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness Reasoning), a question-conditioned, reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps: Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.

Abstract

Multimodal CoT Reasoning

Explore similar work

ETCHR: Editing To Clarify and Harness Reasoning

When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning

Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning