UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?

Zimo Wen; Boxiu Li; Wanbo Zhang; Junxiang Lei; Xiaoyu Chen; Yijia Fan; Qi Zhang; Yujiang Wang

cs.CVMar 3, 2026

UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?

Authors: Zimo Wen, Boxiu Li, Wanbo Zhang, Junxiang Lei, Xiaoyu Chen, Yijia Fan, Qi Zhang, Yujiang Wang,

Organizations: Microsoft Research Asia · Shanghai Jiao Tong University · Fudan University · University of Oxford ·

Abstract

Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear. Existing benchmarks lack a systematic exploration of the specific tasks where generation facilitates understanding. To this end, we introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks, requiring varying degrees of implicit or explicit visual transformations. Extensive evaluation of over 30 models reveals three core findings: 1) Unified models generally underperform their base Vision-Language Models (VLMs), and Generate-then-Answer (GtA) inference typically degrades performance relative to direct inference. 2) Consistent enhancements emerge in spatial intelligence, visual illusions, or multi-round reasoning subtasks, where enhanced spatial and shape perception, as well as multi-step intermediate image states, prove beneficial. 3) Tasks with similar reasoning structures and models sharing architectures exhibit correlated behaviors, suggesting that generation-understanding coupling induces class-consistent inductive biases over tasks, pretraining data, and model architectures. These findings highlight the necessity for more diverse training data and novel paradigms to fully unlock the potential of unified multimodal modeling.

Figures & tables

Figure 1: Model Performance Radar Chart

Benchmark	# Understanding Samples	# Subtasks	Synergy
MME-Unify	1964	3	✗
Uni-MMMU	524	4	✓
ROVER	404	6	✓
RealUnify	400	4	✓
UniG2U (ours)	3000	30	✓

Table 1: Comparison of unified multimodal benchmarks.

Figure 2: Taxonomy of unified multimodal models (UMMs). All models annotated in the figure are benchmarked in this work.

Figure 3: Task taxonomy overview of UniG2U.

Category	Subtasks	Data Sources	# Samples
Real-world Applications	Attentional Focusing; Visual Shortest-Path	VSP [ 13 ] , RealUnify [ 29 ]	200
Geometry Reasoning	Planar Geometry; Solid Geometry	Geometry3K [ 21 ] , AuxSolidMath-Easy [ 7 ]	200
Physics Reasoning	Mechanics; Optics	PhyX [ 48 ]	200
Puzzles and Games	Mental Reconstruction; Mental Tracking; Visual Tracking; Maze; Jigsaw; Sliding	Uni-MMMU [ 54 ] , RealUnify [ 29 ] , BabyVision [ 3 ]	537
Chart & Table Reasoning	Chart Question Answering	ChartQA [ 22 ]	100
Spatial Intelligence	MSR; Attribute (Measurement); Attribute (Approximation); Motion (Camera); Motion (Object)	MMSI-Bench [ 42 ]	500

Table 2: Overview of the UniG2U dataset. The benchmark contains 3,000 samples organized into 7 reasoning categories and 30 fine-grained subtasks, thoughtfully curated from multiple established multimodal reasoning datasets.

Figure 4: Task selection and coverage in UniG2U.

Figure 5: Overview of our two evaluation prompt formats: Direct and Generate-then-Answer (GtA).

Figure 6: Rubric weights for GtA intermediate evaluation. RA measures the adherence and fidelity of the generated intermediate image w.r.t. the generation prompt, while AL measures the logical consistency between the final answer, the generated image, and the original question.

Model	Date	Backbone	Modality	Image Paradigm	Training	i2i	Unify	Text Paradigm
Emu3	2024.09	LLaMA-2	image, audio	diffusion	UP+ST+AT	no (OOD)	end-to-end	AR
X-Omni	2025.07	Qwen-2.5	image	AR	UP+ST+AT	no (OOD)	decoupled	AR
UniWorld-V1	2025.06	Qwen-2.5-VL	image	diffusion	UP+ST	yes	decoupled	AR
MIO	2025.10	Yi-6B-Base	image, audio	diffusion	UP+ST	yes	decoupled	AR
Bagel	2025.05	Qwen-2.5	image, video	Flow	UP+ST	yes	end-to-end	AR
OmniGen2	2025.06	Qwen-2.5-VL	image	Flow	UP+ST	yes	decoupled	AR

Table 3: Comparison of Unified Multimodal Models. indicates frozen backbone, indicates unfrozen (trainable) backbone. The Backbone column represents the corresponding Base VLM used for G2U evaluations.

Figure 7: Representative case studies from UniG2U-Bench

Model	Real-world	Geometry	Physics	Puzzles	Chart	Spatial	Perception	Overall
	Apps	Reasoning		& Games		Intel.	Reasoning	(G2U Gain $\Delta$ )
qwen2.5-vl-7b*	40.25	23.00	34.00	25.73	86.00	23.40	39.42	34.45 (+ $0.00$ )
OmniGen2(direct)	41.25	14.00	41.00	20.91	86.00	23.80	35.62	31.99 (- $2.46$ )
OmniGen2(GtA)	41.25	13.00	41.50	21.91	79.00	26.40	34.52	31.87 (- $2.58$ )
UniWorld-V1(GtA)	24.00	19.00	34.50	24.72	80.00	24.80	39.35	32.96 (- $1.49$ )
OneCAT-3b(direct)	39.50	17.00	32.00	22.05	76.68	25.20	34.55	31.15 (- $3.30$ )

Table 4: Main results on UniG2U across categories. All values are reported as percentages (

\times 100

). The best result within each model group is bolded . Baseline models (marked with

*

) are highlighted with gray background.

\Delta

denotes the performance gap to the respective baseline.

Model	Illusion		Spatial Intelligence			Overall ( $\Delta$ )
Model	icon_shape	in_shape	MSR	Attr. (Meas.)	Motion (Cam.)	Overall ( $\Delta$ )
llava_onevision*	6	18	17	26	20	17.40 (+ $0.00$ )
X-Omni(direct)	5	11	20	31	24	18.20 (+ $0.80$ )
bagel(direct)	2	18	26	31	26	20.60 (+ $3.20$ )
bagel(GtA)	2	17	25	36	24	20.80 (+ $3.40$ )
TokenFlow-XL(direct)	12	13	16	24	19	16.80 (- $0.60$ )

Table 5: Excerpt of Absolute Results on Selected Illusion and Spatial Subtasks. Baseline model is highlighted in gray. See Appendix C for the comprehensive table encompassing all model blocks.

Model	MSR	Maze	Sliding
	(Dir / GtA)	(Dir / GtA)	(Dir / GtA)
OmniGen2	0.220 / 0.290	0.043 / 0.046	0.000 / 0.001
OneCAT-3B	0.320 / 0.390	0.084 / 0.074	0.112 / 0.031
Ovis-U1	0.120 / 0.270	0.134 / 0.003	0.334 / 0.007
MIO	0.210 / 0.320	0.000 / 0.000	0.000 / 0.000
Bagel	0.260 / 0.250	0.021 / 0.281	0.009 / 0.195

Table 6: Direct vs. GtA accuracy on generation-friendly subtasks. GtA shows consistent improvements in multi-step spatial reasoning regimes. Results are formatted as (Direct / GtA).

Figure 8: GtA vs. Direct (Category-level). Each point represents a (model, category) pair comparing Direct and GtA accuracy.

Figure 9: GtA vs. Direct (Overall). Overall accuracy comparison for models supporting both inference protocols.

Model	Real-world	Geometry &	Physics	Puzzles	Chart &	Spatial	Perception	Overall
	Apps	Geometry		& Games	Table	Intel.	Reasoning	(RA / AL)
Bagel	2.25/3.17	3.05/4.52	2.04/2.97	2.66/2.86	1.96/4.66	2.86/3.54	3.22/3.90	2.86/3.61
UAE	1.78/2.79	1.56/2.17	1.98/3.10	2.10/2.83	1.07/3.74	2.14/2.83	2.94/3.45	2.36/3.09
OmniGen2	1.51/2.90	1.26/1.56	1.49/2.31	1.88/2.40	1.34/2.97	1.79/2.70	2.13/3.22	1.86/2.79
GPT-4o + GPT-image	2.43/3.90	4.69/4.72	2.04/2.97	2.70/2.75	3.38/4.71	4.35/4.34	4.26/4.63	3.73/4.09
Ovis-U1	1.67/2.52	1.80/3.23	1.85/2.85	1.45/2.18	1.24/3.37	2.26/3.65	2.33/3.46	2.01/3.14

Table 7: Intermediate-image alignment across task categories. Each cell reports the RA / AL scores (out of 5.0). Higher RA indicates stronger adherence to the intended visual reasoning scaffold; higher AL indicates tighter logical consistency between the final answer and the generated artifact.

Panel of Figure 10 — Figure 10: Spearman correlation heatmaps of G2U gains ( $\Delta$ ). Rather than raw accuracy, these matrices visualize the correlation of the relative offset ( $\Delta$ ) to isolate generation-induced effects. (a) Task-level: Evaluates how different subtasks align in terms of G2U gains. The subtasks along the axes are roughly ordered from perception-oriented (left/top) to reasoning-oriented (right/bottom). Perception tasks and reasoning tasks each form distinctly positive internal clusters, yet they exhibit strong negative correlations with one another, highlighting a fundamental cognitive trade-off. (b) Model-level (Base VLM): Models sharing the same foundational base weights exhibit highly similar G2U behavior, forming distinct high-correlation blocks. (c) Model-level (Architecture): Grouping models purely by their generative paradigm (e.g., Diffusion vs. Autoregressive) yields weaker correlation patterns, indicating that inherited foundational representations govern G2U effects more strongly than the chosen generative architecture.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Real-world	Geometry	Physics	Puzzles	Chart	Spatial	Perception	Overall
	Apps	Reasoning		& Games		Intel.	Reasoning
UniPic2(direct)	36.50	23.50	36.00	24.48	67.00	24.40	39.43	33.68
UniWorld-V1(direct)	35.50	22.00	33.00	25.62	65.00	27.40	36.66	32.79
STAR-7B(direct)	32.50	19.00	30.50	26.25	86.00	25.40	38.56	33.50
Uni-Video(direct)	30.00	20.50	29.50	27.16	81.00	26.00	39.67	33.93

Appendix

Table 8: Direct (Non-CoT) results of frozen-backbone models on UniG2U. All values are reported as percentages (

\times 100

). The Overall column represents the average accuracy across all task categories.

Model	Real	Geometry	Physics	Puzzle	Chart	Spatial	Perception	Overall
UAE (Direct)	36.75	10.50	30.50	19.06	85.00	25.20	36.37	30.94
UAE (GtA)	23.00	7.50	32.50	20.33	71.00	22.80	34.68	28.61 (-2.33)
X-Omni (Direct)	21.50	9.50	29.50	26.24	83.00	27.00	35.31	30.86
X-Omni (GtA)	16.50	4.00	29.00	23.31	22.00	20.20	19.72	19.87 (-10.99)
TokenFlow-XL (Direct)	19.00	10.50	46.00	14.97	43.00	21.80	38.88	29.15
TokenFlow-XL (GtA)	14.50	9.50	26.50	10.06	38.00	21.80	34.28	24.50 (-4.65)

Appendix

Table 9: Sheet7 results (values

\times 100

). Blue numbers indicate the overall drop relative to the corresponding Direct setting.

Panel of Figure 11 — Figure 11: Examples of erroneous generation from edit-untrained models.

Model	Illusion		Spatial Intelligence			Overall ( $\Delta$ )
Model	icon_shape	in_shape	MSR	Attr. (Meas.)	Motion (Cam.)	Overall ( $\Delta$ )
qwen2.5-vl-7b*	5	10	25	20	29	17.80 (+ $0.00$ )
OmniGen2(direct)	6	8	22	33	12	16.20 (- $1.60$ )
OmniGen2(GtA)	6	11	29	33	18	19.40 (+ $1.60$ )
UniWorld-V1(GtA)	4	10	27	22	21	16.80 (- $1.00$ )
OneCAT-3b(direct)	2	8	32	30	24	19.20 (+ $1.40$ )

Appendix

Table 10: Absolute results on selected Illusion and Spatial subtasks. All values are reported as percentages (

\times 100

). Baseline models (marked with

*

) are highlighted with gray background. Overall is computed as the average over the 5 shown subtasks, and the value in parentheses denotes the gap to the corresponding baseline within each model block. The best Overall within each block is bolded .

Panel of Figure 12 — Figure 12: Capability failure. The model produces geometrically inconsistent or physically invalid diagrams, preventing correct downstream reasoning.

Panel of Figure 13 — Figure 13: Capability failure in physics reasoning. Generated diagrams fail to preserve quantitative/structural relations required for correct integration- or geometry-based computation.

Panel of Figure 14 — Figure 14: Surface-relevant but non-utility generation. Outputs are related to the prompt but provide no additional constraints or computational benefit.

Panel of Figure 15 — Figure 15: Surface-triggered generation. The model draws a “coin” due to lexical cues, while the task requires kinematic computation; the generated visuals provide little to no reasoning utility.

Panel of Figure 16 — Figure 16: Irrelevant generation. Outputs are misaligned with the task: chart generations fail to encode structured data, and factual QA triggers unnecessary visual generation.

Panel of Figure 17 — Figure 17: Irrelevant generation in textual knowledge tasks. The model produces visual content despite the task being purely factual, indicating generation–task misalignment.

Panel of Figure 18 — Figure 18: Jigsaw task input images

Figure 19: Generated image: Completion with Candidate 0

Figure 20: Generated image: Completion with Candidate 1

Figure 21: Initial maze state (blue dot at start, green frame marks goal)

Panel of Figure 22 — Figure 22: Example step images (blue dot moves through the maze)

Panel of Figure 23 — Figure 23: Sliding puzzle input: initial and goal states

Panel of Figure 24 — Figure 24: Example step-by-step puzzle solving sequence

Model	Perception Reasoning												Real-world Apps		Chart
Model	icon_scene	icon_shape	in_scene	in_shape	logo_scene	logo_shape	algorithmic	analogical	deductive	inductive	spatial	Fine Discrim.	Attn. Focus.	VSP Reason.	ChartQA
qwen2.5-vl-7b*	98	5	98	10	94	20	31	24	46	26	28	11	44	36.5	86
OmniGen2(direct)	89	6	88	8	89	18	33	20	39	18	24	11	50	32.5	86
OmniGen2(GtA)	80	6	81	11	78	23	28	27	39	25	22	9.82	49	33.5	79
UniWorld-V1(GtA)	99	4	99	10	95	19	36	22	47	21	34	6.75	14	34	80
OneCAT-3b(direct)	85	2	84	8	89	13	30	29	38	20	22	10.02	46	33	76.68

Appendix

Table 11: Main results on UniG2U across categories (Part 1/2). All values are reported as percentages (

\times 100

). Baseline models (marked with

*

) are highlighted with gray background.

Model	Geometry Reasoning		Spatial Intel.					Puzzles & Games						Physics
Model	Geometry	3d Geometry	MSR	Attr. (Meas.)	Attr. (Appr.)	Motion (Cam.)	Motion (Obj.)	Mental Recon.	Mental Track.	Visual Track.	maze	jigsaw	sliding	light	mechanism
qwen2.5-vl-7b*	46	0	25	20	10	29	33	36	25	9.64	14.12	52	5.61	34	34
OmniGen2(direct)	28	0	22	33	19	12	33	31	23	7.23	4.31	48	0	42	40
OmniGen2(GtA)	26	0	29	33	17	18	35	31	23	6.02	4.6	54	0.13	41	42
UniWorld-V1(GtA)	38	0	27	22	16	21	38	39	24	6.02	14.43	47	6.14	33	36
OneCAT-3b(direct)	34	0	32	30	27	24	13	24	25	6.02	8.4	50	11.18	41	23

Appendix

Table 12: Main results on UniG2U across categories (Part 2/2). Continuing from Part 1 with the remaining columns.

Model	Perception Reasoning												Real-world Apps		Chart
Model	icon_scene	icon_shape	in_scene	in_shape	logo_scene	logo_shape	algorithmic	analogical	deductive	inductive	spatial	Fine Discrim.	Attn. Focus.	VSP Reason.	ChartQA
qwen2.5-vl-7b*	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
OmniGen2(direct)	-9	1	-10	-2	-5	-2	2	-4	-7	-8	-4	0	6	-4	0
OmniGen2(GtA)	-18	1	-17	1	-16	3	-3	3	-7	-1	-6	-1.18	5	-3	-7
UniWorld-V1(GtA)	1	-1	1	0	1	-1	5	-2	1	-5	6	-4.25	-30	-2.5	-6
OneCAT-3b(direct)	-13	-3	-14	-2	-5	-7	-1	5	-8	-6	-6	-0.98	2	-3.5	-9.32

Appendix

Table 13: Delta results on UniG2U across categories (Sheet 3, Part 1/2). Baseline models (marked with

*

) are highlighted with gray background, and missing values are denoted by ‘-‘.

Model	Geometry Reasoning		Spatial Intel.					Puzzles & Games						Physics
Model	Geometry	3d Geometry	MSR	Attr. (Meas.)	Attr. (Appr.)	Motion (Cam.)	Motion (Obj.)	Mental Recon.	Mental Track.	Visual Track.	maze	jigsaw	sliding	light	mechanism
qwen2.5-vl-7b*	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
OmniGen2(direct)	-18	0	-3	13	9	-17	0	-5	-2	-2.41	-9.81	-4	-5.61	8	6
OmniGen2(GtA)	-20	0	4	13	7	-11	2	-5	-2	-3.62	-9.52	2	-5.48	7	8
UniWorld-V1(GtA)	-8	0	2	2	6	-8	5	3	-1	-3.62	0.31	-5	0.53	-1	2
OneCAT-3b(direct)	-12	0	7	10	17	-5	-20	-12	0	-3.62	-5.72	-2	5.57	7	-11

Appendix

Table 14: Delta results on UniG2U across categories (Sheet 3, Part 2/2). Continuing from Part 1 with the remaining columns.

Unified Multimodal Models

Unified multimodal models combine multiple data types in shared systems for cross-modal understanding, generation, reasoning, and interaction.

See all 290 papers

Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?
Feb 27, 2026Hongbo Jiang, Jie Li, Yunhang Shen +4Unified Multimodal Models VLM Reasoning
Tencent Youtu Lab, Shanghai, China · Xiamen University, Media Analytics and Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen, China.
Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation
Jun 25, 2026Jinyu Liu, Xincheng Shuai, Henghui Ding +1Unified Multimodal Models Multimodal Foundation Models
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
Feb 9, 2026Cheng Yang, Chufan Shi, Bo Shui +7Unified Multimodal Models Image Generation
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1

Feb 27, 2026J/K move · Enter open · S save

Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?

Hongbo Jiang, Jie Li, Yunhang Shen, Pingyang Dai+3

Tencent Youtu Lab, Shanghai, China · Xiamen University, Media Analytics and Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen, China.

Unified Multimodal Models VLM Reasoning Multimodal Model Evaluation Multimodal Generation

Open paper AskPDF arXiv

Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture. However, existing evaluations typically assess these capabilities separately, overlooking semantic equivalence, i.e., the ability to manifest consistent reasoning results regardless of the output modality. In this work, we investigate whether current U-MLLMs satisfy this premise. We observe that while models demonstrate robust textual reasoning, they fail to maintain semantic equivalence when required to render the same results in the image modality. To rigorously diagnose this discrepancy, we introduce VGUBench, a framework to decouple reasoning logic from generation fidelity. VGUBench comprises three diagnostic tasks: (1)Textual Generative Understanding, establishing a baseline for reasoning accuracy in textual response; (2)Visual Generative Understanding, evaluating the ability to generate visual responses that represent the correct answer; and (3)a Visual Rendering control task, which assesses the ability to directly render explicit visual descriptions into images without complex reasoning. Our evaluation reveals a significant disparity: despite strong performance in textual understanding and visual rendering, U-MLLMs exhibit a marked performance collapse when required to generate visual answers to questions. Furthermore, we find a negligible correlation between visual answering performance and basic rendering quality. These results suggest that the failure stems not from insufficient generation fidelity, but from a breakdown in cross-modal semantic alignment. We provide diagnostic insights to address this challenge in future Unified Generation and Understanding Models.

Abstract

Unified Multimodal Models

Explore similar work

Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?

Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models