Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning

Hengyuan Xu; Wei Cheng; Yumeng Ji; Xuanyang Zhang; Xianfang Zeng; Gang Yu; Xingjun Ma

cs.CVAug 27, 2026

Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning

Authors: Hengyuan Xu, Wei Cheng, Yumeng Ji, Xuanyang Zhang, Xianfang Zeng, Gang Yu, Xingjun Ma

Organizations: Fudan University · StepFun · Shanghai Jiaotong Univeristy

Abstract

Explicit visual intermediates can help multimodal large language models (MLLMs) externalize spatial evidence and updated visual states, but their utility depends on whether an image editor can faithfully realize the required transformation. We introduce Aphanta, an automated task-discovery and closed-loop diagnostic framework for the MLLM -> image editor -> MLLM pipeline. Aphanta evaluates three conditions---direct reasoning, reasoning with an editor-generated intermediate, and reasoning with an idealized reference intermediate---to separate potential visual headroom from the practical utility of current editors. Across 20 candidate tasks and multiple editor--MLLM combinations, we find that utility is strongly task-conditioned. Gains concentrate in visual cue injection, grounding, and counterfactual state realization, whereas intermediates requiring symbol-sensitive construction or structural extrapolation are substantially less reliable. On the selected positive-task subset, our consolidated Qwen pipeline improves the mean task score from 0.343 to 0.445 ( $+10.2$ points; $+29.7\%$ relative), while the full study also retains filtered and unsuccessful tasks to expose the boundary. These results position image editing as a specialized visual workspace rather than a universal reasoning mechanism, and establish Aphanta as a reusable protocol for measuring task--representation alignment, editor realization, and downstream pipeline utility.

Figures & tables

Figure 2: The Aphanta Pipeline. A four-phase loop proposes and screens tasks, performs preliminary A/B/C diagnosis, constructs and reviews data, and trains and evaluates editor-in-the-loop pipelines. Results feed back into task proposal, producing a task-conditioned utility map rather than a success-only collection.

Figure 3: Showcases of Tasks Aphanta Found. The colors represent four different categories: \highlighttext whitecolorLowLevelLow-level Recognition, \highlighttext whitecolorGroundingGrounding, \highlighttext whitecolorEditingCounterfactual, and \highlighttext whitecolorLogicLogic.

ID	Task	Status	Topic	Evaluation			ID	Task	Status	Topic	Evaluation
ID	Task	Status	Topic	Direct	+Edit	+Ref.	ID	Task	Status	Topic	Direct	+Edit	+Ref.
1	Where Is My Mirror	\passbadge PASS	Grounding	0.70	0.78	0.77	11	Plane Geometry Aux. Line	\failbadge FAIL	Logic	0.41	0.43	0.47
2	Circuit Diagram Parsing	\failbadge FAIL	Logic	0.70	0.60	0.88	12	Dense Dot Counting	\passbadge PASS	Low Level	0.05	0.23	0.52
3	Auto. Driving Assistant	\passbadge PASS	Grounding	0.42	0.57	0.45	13	Tangram	\passbadge PASS	Low Level	0.31	0.51	0.61
4	RPM: Synthetic	\passbadge PASS	Logic	0.18	0.30	0.39	14	Gear Rotation Reasoning	\failbadge FAIL	Logic	0.92	0.42	0.83
5	RPM: Real	\passbadge PASS	Logic	0.61	0.79	0.71	15	Industrial Defect Inspect	\passbadge PASS	Grounding	0.70	0.82	0.88

Table 1: Complete 20-Task Audit. A/B/C denote Direct, Actual Edit, and Reference. Italicized scores are preliminary values for tasks without final-stage evaluation; “status” records whether a task produced a retained practical pipeline, not whether visual assistance is possible in principle.

(a) Complete within-family pipelines
Pipeline	Direct	Edit	Ref.	$\Delta$ (pt.)
Qwen3-VL + Qwen-Image-Edit	0.343	0.445	0.558	+10.2
Seed-2.0 + Seedream-4.5	0.505	0.540	0.650	+3.5
GPT-5 + GPT-Image-1.5	0.475	0.425	0.635	-5.0
Gemini-3 + Nano Banana 2	0.580	0.625	0.650	+4.5

Table 3: Consolidated Evaluation on the Selected Positive-Task Subset. Left: complete within-family MLLM–editor pipelines. Right: editors compared with Qwen3-VL fixed.

\Delta

is the absolute change in mean task score, reported in points; the Qwen pipeline’s

+10.2

points correspond to

+29.7\%

relative to its direct score. Results characterize this selected subset rather than a general editor leaderboard.

Figure 5: Relative Edit Gain by Task Operation and Model/Editor Combination. Large percentages can arise from small direct baselines; Table 3 reports the corresponding absolute aggregate deltas.

Figure 6: Positive Cases from the Qwen3-VL and Qwen-Image-Edit Pipeline. Each row compares direct reasoning, the actual edited intermediate, and the programmatically constructed reference condition.

Figure 7: Reliable Visual-State Realization. A counterfactual task in which editing realizes the queried visual state before the final MLLM decision.

Figure 8: Plausibility Can Exceed Task Fidelity. A visually coherent numbering edit that omits a task-critical item.

Figure 9: Structured Extrapolation Is Less Reliable. Representative realization errors in circuit interpretation and geometry construction.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10: Joint Data Construction Diagnostics. Panel (a) visualizes the balanced training export produced from the catalog-level training manifest: source rows are shown before resampling, and every pipeline is resampled to 10k rows. Panel (b) visualizes the completed-pipeline evaluation manifest. All lite splits sample 20 rows per pipeline; the full split uses 200 rows when available, except the real Raven split (P8.1), which contributes 28 standard test examples.

Panel of Figure 11 — Figure 11: Unified Inference and Artifact Flow. The same normalized evaluation row feeds Direct, Actual Edit, and Reference branches. The Actual Edit branch first plans short edit instructions, then calls an editor backend and re-answers from the edited helper image(s). The evaluator writes both metric-level summaries and per-sample traces, which makes planner failures, editor failures, and answer errors auditable after a run.

Panel of Figure 12 — Figure 12: Diagnostic Views over the 20-Task Audit. Panel (a) plots the reference gain against the actual edit gain for tasks with both quantities available; filled markers denote retained practical pipelines, and open markers denote stopped or unsuccessful tasks. Tasks 10 and 19 lack an actual-edit score and are omitted from the scatter but included in panel (b). Panel (b) summarizes retained versus stopped/unsuccessful outcomes by dominant operation.

Template family	Representative tasks	Typical instruction form	Planner role
Fixed spatial annotation	Mirror localization, traffic signs, industrial defects, dense difference	Draw red boxes around the relevant region; highlight anomalous regions; mark the only differing tile.	Cue grounding
Parameterized local edit	Clock reasoning, object deletion, dense counting	Rotate the minute hand by a specified angle; remove {object} ; mark each {target} with a red dot and index.	State update
Constructive final state	Synthetic and real RPM, tangram	Fill the blank matrix cell with the final pattern; generate the final tangram board after the described steps.	Visual construction
Isolation and close-up	Zoom-in/remake and blackout variants	Keep only the target object and black out the rest; crop the edited subject into a square close-up.	Evidence filtering

Appendix

Table 4: Edit-Instruction Template Families Used by the Unified Planner. Templates are distilled from successful task packets and injected as in-context examples for planning; concrete object names, directions, angles, and target categories are filled from each sample.

ID	Task	Status	Operation	Metric	Train $N$	Preliminary diagnosis			Final evaluation
ID	Task	Status	Operation	Metric	Train $N$	Direct	+Edit	+Ref.	Direct	+Edit	+Ref.
1	Where Is My Mirror	\passbadge PASS	Grounding	IOU	3,818	0.53	0.53	0.50	0.70	0.78	0.77
2	Circuit Diagram Parsing	\failbadge FAIL	Structured	ACC	30,155	0.70	0.60	0.88	–	–	–
3	Auto. Driving Assistant	\passbadge PASS	Grounding	ACC	2,303	0.45	0.43	0.63	0.42	0.57	0.45
4	RPM: Synthetic	\passbadge PASS	Structured	ACC	10,000	0.24	0.18	0.37	0.18	0.30	0.39
5	RPM: Real	\passbadge PASS	Structured	ACC	2,100	0.51	0.43	0.60	0.61	0.79	0.71

Appendix

Table 5: Detailed Task Audit. Preliminary and final-stage scores are reported separately. Reference intermediates are programmatically constructed diagnostic targets.

(a) BabyVision
Reasoner	Direct	+Edit	Edit $\Delta$
Qwen	0.1958	0.1623	-0.0335
GPT	0.1804	0.1546	-0.0258
Gemini	0.3015	0.2319	-0.0696

Appendix

Table 6: Transfer to BabyVision and MIRA. Edit

\Delta

and Ref.

\Delta

are absolute accuracy changes from direct reasoning; MIRA additionally provides constructed reference intermediates.

Multimodal Reasoning

Multimodal reasoning studies how AI systems combine text with images, audio, video, and other signals to solve evidence-dependent tasks.

See all 512 papers

ETCHR: Editing To Clarify and Harness Reasoning
May 22, 2026Beichen Zhang, Yuhong Liu, Jinsong Li +3Tool Use in VLMs Multimodal Reasoning
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University +2
RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward
Feb 19, 2026Qiucheng Wu, Jing Shi, Simon Jenni +4Text-Guided Image Editing RL for Image Editing
Adobe Research · UC, Santa Barbara
VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
Jun 5, 2026Didi Zhu, Changrui Chen, Stefanos Zafeiriou +1VLM Evaluation Multimodal Grounding
Imperial College London

May 22, 2026J/K move · Enter open · S save

ETCHR: Editing To Clarify and Harness Reasoning

Beichen Zhang, Yuhong Liu, Jinsong Li, Yuhang Zang+2

The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University+2

Tool Use in VLMs Multimodal Reasoning Multimodal CoT Reasoning Visually Grounded Reasoning Multimodal Foundation Models Image Editing+1

Open paper AskPDF arXiv

Multimodal Large Language Models have advanced visual reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ''think with images'' paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as reasoning assistants with two complementary gaps: a language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness Reasoning), a question-conditioned, reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps: Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.

Abstract

Multimodal Reasoning

Explore similar work

ETCHR: Editing To Clarify and Harness Reasoning

RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward

VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?