Unified multimodal models (UMMs) jointly support visual understanding and image generation, yet information flow between these two capabilities remains largely asymmetric: understanding representations routinely condition generation, whereas generative representations are rarely reused for multimodal reasoning. In this work, we investigate whether intermediate representations from a UMM's native diffusion branch can enhance spatial reasoning without compromising general multimodal understanding. We propose DiFuse, which selectively incorporates intermediate diffusion representations into the understanding pathway of UMMs. Through systematic layer-wise analysis, we identify which diffusion representations are most informative and where they can be integrated with minimal interference to semantic processing. Guided by these findings, DiFuse spatially aligns diffusion features with visual tokens and injects them into selected multimodal decoder layers via residual fusion. Extensive experiments on UniWorld-V1 show that, DiFuse consistently improves spatial reasoning over a matched SFT baseline while maintaining broadly comparable performance on general multimodal benchmarks. Further analyses show that these gains depend critically on the choice of diffusion representations, injection positions, and the preservation of spatial correspondence between diffusion features and visual tokens. Results on OmniGen2 further demonstrate the transferability of DiFuse across unified multimodal architectures. Overall, our findings show that native diffusion representations can provide complementary spatial cues for multimodal reasoning when selectively integrated in a structure-preserving manner.
Figures & tables
Figure 1: Overview of DiFuse and its impact on unified multimodal models. (a) DiFuse establishes a generation-to-understanding pathway by reusing intermediate representations from the native Diffusion Transformer (DiT) branch and integrating them into the visual understanding pathway of a unified multimodal model. (b) General–spatial capability landscape. Models are positioned by their average general-understanding (Avg-G) and spatial-reasoning (Avg-S) scores. Marker shapes distinguish general-purpose MLLMs, UMMs, spatially enhanced models, and DiFuse. (c) Gains relative to Qwen2.5-VL-7B. We compare four Qwen2.5-VL-7B-based models against the same baseline. Blue and green bars show changes in Avg-G and Avg-S, respectively. UniWorld-V1 + DiFuse improves Avg-S by 2.60 points while leaving Avg-G nearly unchanged (+0.02 points).
Figure 2: Paradigms for generation-assisted and structure-enhanced multimodal understanding. Existing methods use diffusion objectives, explicit visual states, external diffusion features, or dedicated geometry branches, whereas DiFuse directly reuses spatially aligned representations from the UMM’s native DiT branch.
Figure 3: DiFuse architecture. (a) Overall pipeline. Given an input image and a question, the UMM’s understanding branch processes the visual and text tokens, while its native generation branch receives the same image and a fixed generation prompt “Generate the depth map of this image.” DiFuse extracts an intermediate spatial representation from the frozen DiT at a selected layer and denoising timestep, without decoding a final image. (b) DiFuse module. The extracted feature is spatially pooled and compressed with PCA, then normalized and projected through a convolutional adapter. Bilinear resizing aligns the resulting feature map with the visual-token grid. DiFuse adds the aligned features to visual-token states at selected decoder layers through residual connections; text-token states receive no direct residual update.
Summary
General Multimodal Understanding
Spatial Reasoning
Backbone
Variant
Avg-G
Avg-S
MME norm
MMB
MMStar
GQA
RWQA
VSR
EmbSpatial
CVB-3D
3DSR-C
UniWorld-V1 [-1pt] (Qwen2.5-VL-7B)
Base
71.92
71.08
82.35
83.93
62.74
60.92
69.67
87.32
69.97
81.08
45.94
+ SFT
71.51 (-0.41)
71.88 (+0.80)
78.23
84.88
63.76
60.90
69.80
87.81
70.99
81.92
46.79
+ DiFuse
71.94 (+0.02)
73.68 (+2.60)
80.82
84.54
62.97
61.70
69.67
89.36
72.39
84.58
48.38
OmniGen2 [-1pt] (Qwen2.5-VL-3B)
Base
66.04
65.01
76.55
77.49
56.72
60.22
59.22
80.77
61.46
74.08
43.73
+ SFT
67.69 (+1.65)
65.37 (+0.36)
76.71
78.26
58.00
61.17
64.31
82.41
62.20
74.75
42.12
Table 1: Controlled comparison of SFT and DiFuse on unified multimodal models. For each backbone, SFT and DiFuse use identical instruction-tuning data and optimization settings. Avg-G averages MME norm , MMB, MMStar, GQA, and RWQA; Avg-S averages VSR, EmbSpatial, CVB-3D, and 3DSR-C. Numbers in parentheses denote absolute changes over the corresponding base UMM. Bold numbers indicate the best result within each backbone family.
Summary
General Multimodal Understanding
Spatial Reasoning
Method
Params.
Avg-G
Avg-S
MME norm
MMB
MMStar
GQA
RWQA
VSR
EmbSpatial
CVB-3D
3DSR-C
General-purpose VLMs
InternVL3
8B
74.24
72.41
85.18
85.74
68.65
62.73
68.89
87.07
74.45
84.00
44.11
LLaVA-1.5
7B
56.51
47.16
66.89
64.00
33.76
61.93
55.95
51.47
45.77
57.00
34.38
Llama-3.2-Vision-Instruct
11B
52.37
52.77
62.46
72.77
48.44
21.70
56.47
72.59
51.76
64.58
22.14
Idefics2
8B
59.46
62.50
63.99
75.26
47.34
51.22
59.48
78.56
53.49
71.00
46.93
Table 2: Contextual comparison with representative general-purpose VLMs, unified multimodal models (UMMs), and spatial-reasoning-enhanced models. All models are evaluated using the same benchmark splits and scoring implementation under deterministic decoding. Avg-G and Avg-S follow the definitions in Table 1 . Bold numbers in the summary columns indicate the best result within each model category. The shaded row denotes our DiFuse-enhanced UMM.
Summary
General Multimodal Understanding
Spatial Reasoning
Source layer
Avg-G
Avg-S
MME norm
MMB
MMStar
GQA
RWQA
VSR
EmbSpatial
CVB-3D
3DSR-C
g17
71.44
72.14
79.00
83.76
62.61
61.11
70.72
88.30
71.18
81.58
47.51
g18
71.89
73.08
81.31
83.51
64.03
60.92
69.67
89.28
72.12
82.83
48.09
g28
71.38
73.18
79.95
83.93
62.39
61.23
69.41
88.87
72.17
83.67
48.03
g53
71.74
73.06
79.95
84.36
62.46
60.92
70.98
89.12
71.95
83.00
48.19
Table 3: Diffusion source-layer screening. All candidates are evaluated under the same reduced screening configuration with all other settings fixed. These values are not directly comparable to the final DiFuse results. Bold numbers indicate the best result in each column, and the shaded row denotes the selected source layer.
Summary
General Multimodal Understanding
Spatial Reasoning
Factor
Variant
Avg-G
Avg-S
MME norm
MMB
MMStar
GQA
RWQA
VSR
EmbSpatial
CVB-3D
3DSR-C
Diffusion timestep
t
50
71.52
72.98
79.84
83.76
62.87
61.48
69.67
88.95
71.76
83.58
47.64
t
100 (ours)
71.94
73.68
80.82
84.54
62.97
61.70
69.67
89.36
72.39
84.58
48.38
t
200
71.59
73.05
80.33
83.93
62.53
61.32
69.80
89.77
72.39
81.50
48.54
t
300
71.50
73.52
80.29
84.19
62.83
61.43
68.76
89.12
72.31
83.75
48.90
Table 4: Ablation of core DiFuse design choices on UniWorld-V1. Each block varies one factor while fixing the others; shaded entries denote the final configuration.
Summary
General Multimodal Understanding
Spatial Reasoning
Condition
Variant
Avg-G
Avg-S
MME norm
MMB
MMStar
GQA
RWQA
VSR
EmbSpatial
CVB-3D
3DSR-C
Correspondence
Aligned features (ours)
71.94
73.68
80.82
84.54
62.97
61.70
69.67
89.36
72.39
84.58
48.38
Correspondence
Image-shuffled features
71.33
72.60
80.25
84.19
62.52
61.46
68.24
88.87
71.65
82.83
47.04
Correspondence
Token-shuffled features
71.80
71.84
81.59
83.93
62.41
61.14
69.93
87.81
70.80
81.42
47.33
Correspondence
Random features
71.91
72.80
80.36
84.45
64.73
60.74
69.28
88.95
72.01
82.00
48.23
Table 5: Effect of spatial correspondence in DiFuse. Image and token shuffling disrupt the corresponding image-level and token-level spatial alignment, while random features replace the diffusion representation. The shaded row denotes the final configuration.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
UniWorld-V1
OmniGen2
Understanding backbone
Qwen2.5-VL-7B
Qwen2.5-VL-3B
Decoder hidden dimension
3,584
2,048
Decoder layers
28
36
DiT source layer
18
6
Target timestep
100
900
Cached feature dimension
1,024
1,024
Appendix
Table 6: Backbone-specific DiFuse configurations. DiT layer denotes the source layer in the native generation branch; injection layers denote layers in the multimodal decoder.
Setting
Value
Training examples
296,401
Training epochs
3
Optimization steps
6,948
Hardware
8× NVIDIA A100-80GB
Per-device batch size
2
Gradient accumulation
8
Appendix
Table 7: Training configuration for the UniWorld-V1 experiments. The effective batch size includes gradient accumulation over all GPUs.
Backbone
LoRA Params.
Fusion Params.
LoRA Size
Fusion Size
UniWorld-V1
323.0M
207.7M
646.0 MB
415.3 MB
OmniGen2
239.5M
157.3M
957.9 MB
314.6 MB
Appendix
Table 8: Trainable parameter and checkpoint storage costs for the final DiFuse configurations. The reported values correspond to the main UniWorld-V1 experiment and the OmniGen2 generalization experiment. Parameter counts may vary across ablation settings that change the number of injection layers or the feature dimension.
EmbSpatial
CV-Bench-3D
Backbone
Variant
AI2-THOR
MP3D
ScanNet
Overall
Depth
Distance
Overall
Base
68.93
71.61
69.42
69.97
83.33
78.83
81.08
UniWorld-V1
+ SFT
70.86
72.11
70.00
70.99
83.17
80.67
81.92
+ DiFuse
72.64
73.36
71.17
72.39
85.33
83.83
84.58
Base
55.45
65.11
64.00
61.46
76.50
71.67
74.08
OmniGen2
+ SFT
56.74
64.61
65.42
62.20
79.50
70.00
74.75
Appendix
Table 9: Category-wise spatial reasoning results on EmbSpatial and CV-Bench-3D (%). EmbSpatial is divided into AI2-THOR, Matterport3D, and ScanNet. CV-Bench-3D reports Depth Order and Relative Distance subtasks. The Overall column denotes the official aggregate accuracy.
Source layers
Inject.
λ
Avg-G
Avg-S
VSR
EmbSpatial
CVB-3D
3DSR-C
Single-layer g18
0 – 3
0.5
71.89
73.08
89.28
72.12
82.83
48.09
g18+g53
0 – 3
0.5
71.90
73.04
88.87
71.40
83.75
48.15
g18+g28
0 – 3
0.5
71.63
73.23
89.12
71.59
84.08
48.13
Single-layer g18
0 – 7
0.5
72.00
72.08
88.30
69.81
81.17
49.06
g18+g28
0 – 7
0.5
71.51
73.40
89.12
72.34
83.75
48.38
g18+g53
0 – 7
0.5
72.02
73.00
88.63
72.25
83.17
47.97
Appendix
Table 10: Exploratory anchor-residual multi-layer fusion. All multi-layer variants use g18 as the anchor and add either g28 or g53 through a learnable bounded residual. Rows are grouped by decoder injection range and main fusion coefficient λ . Shaded rows denote the matched single-layer references, with the darker row indicating the final DiFuse configuration. Bold numbers indicate the best Avg-G and Avg-S within each group.
Figure 4: Diffusion responses across timesteps at fixed layer g18 . Response maps are shown at t∈{50,100,200,300} for three representative images. Spatial organization is retained across timesteps, while the response granularity varies.
Figure 5: Diffusion responses across candidate layers at fixed timestep t=100 . Response maps are shown for the vital-layer candidates identified following Stable Flow ( Avrahami et al., 2025 ) . The spatial responses vary substantially across diffusion depth despite using the same image and timestep.
Figure 6: Qualitative comparison between SFT and DiFuse. Examples cover spatial relations, depth and 3D reasoning, and general and fine-grained visual understanding. Compared with the matched SFT baseline, DiFuse corrects several errors involving spatial relations and depth reasoning while largely preserving predictions on general visual understanding examples. Red and green responses denote incorrect and correct predictions, respectively.
Multimodal models increasingly think with different modalities such as images, 3D point clouds, and robot states, not just text. Yet each modality is still encoded into its own representation space, creating a modality-switching gap whenever reasoning moves from one modality to another. In this paper, we introduce Uni-LaDiR (Unified Latent Diffusion Reasoner), a framework that unifies different modalities into a shared latent space for multimodal reasoning. A unified encoder maps teacher reasoning steps from different modalities into latent thought tokens in a shared space, trained to extract the information needed for later reasoning steps and the final output. A diffusion reasoner, trained jointly with the encoder, generates these tokens at inference without teacher reasoning steps. Across eleven vision-language model (VLM) benchmarks and two vision-language-action (VLA) suites, Uni-LaDiR achieves relative gains over the strongest baselines of 7.3% on four mathematical and logical VLM benchmarks and 6.1% on RLBench manipulation tasks. Controlled comparisons show increasing gains as more teacher modalities are unified. These results suggest that unification improves multimodal reasoning by weaving it into a single thread, where the model predicts successive thoughts in a common representation space.
Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang +3
University of California, San Diego · Meta · University of Pennsylvania
Unified multimodal models (UMMs) aim to handle perception and generation in a single model. Yet existing UMMs still rely on a frozen, separately pretrained VAE for image generation, imposing a structural bottleneck. Naively removing it introduces a quality gap, as the model must learn both high-level structure and low-level details from raw pixels. In this paper, we propose Representation Forcing (RF), a technique that closes this gap by making representation prediction a native capability of the model. Concretely, RF forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels; these tokens then stay in context to guide pixel diffusion within the same backbone. By turning representations from perception outputs into generation targets, RF eliminates the need for any external generative latent space. We find that RF benefits both understanding and generation. On image generation, our pixel-space model with RF matches state-of-the-art VAE-based unified models. On image understanding, pixel-space RF generally outperforms its VAE-based variant. Together, these results offer an effective step toward end-to-end, bottleneck-free UMMs.
Yuqing Wang, Zhijie Lin, Ceyuan Yang +10
University of Hong Kong · ByteDance Seed · The Chinese University of Hong Kong +2
Unified Multimodal Models (UMMs) have emerged as a critical direction for general-purpose multimodal intelligence, integrating understanding and generation into a single framework. However, existing UMMs face prominent challenges: (1) the inherent learning conflicts between visual understanding and generation tasks, leading to suboptimal modeling in both tasks; (2) different understanding and generation visual spaces impeding scalability; (3) over-reliance on task-specific data that neglects the duality of text-image understanding and generation. To address these challenges, we propose UniDDT, which leverages a Noisy ViT encoder along with an LLM to unify semantic encoding for visual generation and understanding tasks, while employing a separate diffusion decoder to decouple diffusion decoding from text decoding. With this Noisy ViT encoder, UniDDT is able to leverage the latent space as a unified visual representation, enabling seamless compatibility between understanding and generation tasks. Thus, the scalability within the generation tasks and the semantic expressiveness within understanding tasks can be balanced. Also, we construct dual data structures from the same image-text pairs, fostering interdependence between the generation and understanding data to exploit their inherent duality. Extensive experiments demonstrate that UniDDT achieves effective unification of multimodal understanding and generation with enhanced semantic consistency and scalability. For visual generation tasks, our UniDDT achieves 0.87 GenEval score and 86.9 DPG overall score. For multimodal understanding tasks, our UniDDT achieves 1699.5 score on MME benchmark and 76.5 overall score on SEEDbench.
Shuai Wang, Liang Li, Yang Chen +3
Nanjing University · ByteDance Seed · University of Hong Kong