Diffusion/flow models have emerged as the de facto paradigms for image generation, giving rise to a diverse family of modern generators with increasingly heterogeneous architectures and representations. Despite their remarkable generation capabilities, the cross-model alignment of latent representations and the consistency of their underlying denoising dynamics remain underexplored. In this work, we uncover an intriguing cross-model compatibility: independently trained generators exhibit similar latent spatial organization, while directly handing off intermediate states between unaligned models can still preserve coarse visual content. Motivated by these observations, we propose InstantFusion, a lightweight shared latent interface that connects the denoising trajectories of heterogeneous image generators. Specifically, to enable flexible cross-model state translation, model-specific Latent AutoEncoders (LAEs) are introduced to project intermediate states from different generators into a shared latent space and decode them into the latent space of a target generator. This shared space is constrained by two complementary objectives: aligning intermediate states derived from the same image at matched noise levels and synchronizing their corresponding local denoising velocities. Once trained, InstantFusion allows denoising to seamlessly transition across frozen generators without requiring a dedicated mapping for every model pair. The versatility of InstantFusion is demonstrated across acceleration, multi-preference composition, cross-architecture on-policy distillation, and cross-model composition, achieving competitive performance across these diverse settings.
Figures & tables
Figure 1: Overview of InstantFusion. During training, model-specific LAEs map the intermediate states of frozen generators into a shared representation space, where both latent states and denoising dynamics are aligned. During inference, a sender state is translated through the shared space into the receiver’s latent space, and an adaptive criterion determines when the receiver takes over denoising.
Figure 2: Cross-model compatibility. (a) Intermediate states from Qwen-Image, SD3, and FLUX.1 exhibit similar spatial self-similarity patterns. (b) Direct handoff between related models preserves generation quality, while cross-family handoff retains the main subject and coarse layout. (c) These observations motivate a lightweight interface for connecting heterogeneous generators.
Figure 3: Architecture of the LAE. (a) σ denotes the scalar noise level associated with denoising step tk . It is spatially broadcast and concatenated with the intermediate state xtkA . Convolutional layers and time-conditioned residual blocks then map the input to its encoded representation ztkA , with a skip connection preserving the original state. (b) Details of the time-conditioned resblock: Each residual block embeds σ and predicts scale and shift parameters that modulate group-normalized features, conditioning the transformation on the current denoising stage.
Figure 4: Effect of denoising-dynamics alignment. Training without Ldyn establishes correspondence between intermediate states x , but leaves their associated denoising velocities v poorly aligned. Adding Ldyn improves velocity alignment. Each line in the UMAP visualization connects the v feature points produced by different models when conditioned on the same prompt.
Figure 5: Training dynamics of the model-specific LAEs. The total loss and the three component losses ( Lrec , Lalign , and Ldyn ) decrease steadily and converge during training.
Figure 6: Qualitative comparison of Qwen-Image, FLUX.1, and SD3 outputs after 1, 5, and 10 repeated latent round trips. The original outputs before any round trip serve as baselines.
Figure 7: Qualitative results of cross-model latent transfer among heterogeneous generators.
Table 1: Evaluation of the shared latent interface. (a) Reconstruction fidelity across repeated latent round-trips. (b) Quantitative generation performance under cross-architecture on-policy distillation. (c) Reconstruction fidelity degradation upon removing the reconstruction loss Lrec .
Models
New Model
GenEval-v2
CVTG-2K
Clip
OneIG-Bench
Alignment
Text
Diversity
Style
Reasoning
Avg.
▼ Stronger Model
Qwen-Image [ 34 ]
–
41.73
0.6699
25.46
0.8304
0.9138
0.2091
0.2230
0.2486
0.4849
▼ Generation Baselines
SD3 [ 9 ]
–
22.14
0.3682
28.65
0.7934
0.4161
0.2327
0.3596
0.2770
0.4157
FLUX.1 [ 4 ]
–
22.86
0.5180
27.75
0.7894
0.5736
0.2136
0.3663
0.2653
0.4416
Table 2: Comparison of base models and cross-model combinations, demonstrating that guidance from a small set of high-quality generators can improve the generation quality of weaker models.
Models
New model
Time ↓
GenEval -v2 ↑
Clip ↑
Aesthetic ↑
OneIG-Bench ↑
Alignment
Text
Diversity
Style
Reasoning
Avg.
▼ Generation Baselines
Qwen-Image [ 34 ]
–
18.84
41.73
25.46
5.9797
0.8304
0.9138
0.2091
0.2230
0.2486
0.4849
Qwen-Few-Step
–
0.893
32.94
27.48
6.1430
0.8804
0.9752
0.0885
0.3234
0.3069
0.5148
FLUX.2-Klein [ 4 ]
–
0.516
25.43
27.71
5.9017
0.5718
0.1863
0.3930
0.4059
0.2415
0.3597
Z-Image-Turbo [ 6 ]
–
3.521
39.21
25.02
4.9892
0.8658
0.9788
0.2193
0.4573
0.3137
0.5669
Table 3: Quantitative comparison of generation quality and inference efficiency. Performance benchmarks across baselines, existing acceleration techniques, and InstantFusion. Furthermore, InstantFusion exhibits orthogonal compatibility when integrated with training-free acceleration methods like DiCache, yielding cumulative speedups without compromising generation fidelity.
Models
New Model
GenEval-v2
HPSv2
HPSv3
Anime
Concept
Paint.
Photo
Avg.
Anime
Concept
Paint.
Photo
Avg.
▼ Reward-Specialized GRPO Baselines
GRPO baseline (HPSv2)
–
28.83
32.21
31.67
31.91
28.95
31.18
12.14
12.88
13.30
9.94
12.07
GRPO baseline (HPSv3)
–
26.53
31.42
30.57
30.93
27.63
30.14
12.31
13.06
13.48
9.73
12.15
GRPO baseline (GenEval-v2)
–
35.89
28.55
27.28
27.47
25.47
27.19
10.98
11.30
11.71
7.69
10.42
▼ Direct V Fusion
Table 4: Evaluation results of reward-specialized GRPO models and model combination strategies.
Table 12
Figure 8: Evolution of generated images over training steps during cross-model OPD.
Figure 9: Comparison between the predicted clean images at adaptively selected handoff steps and the corresponding fully denoised outputs, demonstrating preservation of global structure.
Rollout Schedule
GenEval-v2
HPSv2
Anime
Concept
Paint.
Photo
Avg.
SD3 → FLUX.1 → Qwen
19.99
27.04
25.41
25.46
24.69
25.65
SD3 → Qwen → FLUX.1
24.93
24.35
22.80
22.72
21.91
22.95
FLUX.1 → SD3 → Qwen
25.22
27.86
25.41
25.30
24.87
25.86
FLUX.1 → Qwen → SD3
27.96
25.68
22.85
22.96
22.68
23.54
Qwen → SD3 → FLUX.1
30.76
26.60
24.14
24.23
24.06
24.76
Table 7: Robustness of the shared latent interface across multi-stage rollout schedules. We evaluate different execution orders across three heterogeneous generators (SD3, FLUX.1, and Qwen-Image). The consistent performance across arbitrary transition permutations confirms the stability and invariance of the joint shared latent space under cascading cross-model transfers.
Metric
Individual Generators
Hybrid Generation
SD3
FLUX.1
Qwen
FLUX.1 → SD3
SD3 → Qwen
FLUX.1 → Qwen
FLUX.1 → SD3 → Qwen
Latency (s/image)
3.61
10.70
20.82
7.16
12.21
15.7
11.53
Theoretical NFE
50
50
50
25+25
25+25
25+25
17+17+16
Table 8: End-to-end inference latency and theoretical NFEs. Latency is measured at 1024×1024 resolution, averaged over 20 runs, and reported in seconds per image. For hybrid configurations, the NFE entries show how the 50 denoising evaluations are allocated across models.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Table 17
Shared Space
Reconstructed Model
Switch
PSNR ↑
SSIM ↑
LPIPS ↓
TaylorSeer ( N=6 )
FLUX.1
–
28.945
0.6556
0.4020
SD3–Qwen
Qwen
1
32.418
0.9433
0.03811
5
24.653
0.8719
0.10096
10
22.255
0.8253
0.13709
SD3
1
30.331
0.9401
0.04244
5
21.301
0.8143
0.15164
Appendix
Table 11: Reconstruction fidelity under different shared-space configurations.
Figure 10: Cross-model state handoffs at different denoising timesteps (Qwen-Image, Flux.1).
Figure 11: Cross-model state handoffs at different denoising timesteps (Qwen-Image, SD3).
Figure 12: Qualitative comparison of shared latent reconstruction (SD3).
Figure 13: Qualitative comparison of shared latent reconstruction (FLUX.1).
Figure 14: Qualitative comparison of shared latent reconstruction (Qwen-Image).
Figure 15: Cross-architecture state handoffs initiated from Qwen-Image to SD3.
Figure 16: Cross-architecture state handoffs initiated from Qwen-Image to FLUX.1.
Figure 17: Qualitative results under generation acceleration.
Figure 18: Multi-preference composition cases combining GenEval-v2 and HPSv2.
Figure 19: Multi-preference composition cases combining HPSv2 and HPSv3.
Most diffusion and flow-matching generators define the prior, probability path, and prediction target in the same representation space. Latent diffusion improves efficiency by moving this path into an autoencoder latent space, but the final sample is still produced by a separately trained decoder. This separation creates a mismatch: the generator is optimized for latent-space prediction, while final quality depends on how the decoder handles generated latents that may differ from clean encoder outputs. We introduce CrossFlow, a cross-space flow formulation that maps noisy latent inputs directly to pixel-space images. The key technical step is a velocity-free one-step objective: the latent trajectory defines the training path, but the supervised prediction is an image rather than a latent displacement. This lets one model act both as a one-step latent-to-pixel generator and as a decoder replacement for latent diffusion pipelines. On class-conditional ImageNet-1k at 256×256, CrossFlow-XL achieves 1.62 FID with one function evaluation. Ablations show that the latent encoder and pixel-space perceptual and adversarial losses are important for fidelity. These results indicate that cross-space flow objectives can combine the efficiency of latent representations with direct pixel-space supervision, without requiring a separate decoder at inference.
Xiyuan Wang, Xiao Zhang, Yang Li +4
Institute for Artificial Intelligence, Peking University · Tencent · Fudan University
Leading open text-to-image models often carry complementary strengths: one may lead on preference-aligned aesthetics while another follows compositional instructions more faithfully. However, differences in their autoencoders and noise schedules make it difficult to transfer these strengths across models. In this paper, we present Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student. To bridge the incompatible latent spaces of different teachers, Poly-OPD performs on-policy distillation through a pixel bridge. Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule. The resulting target is further matched to the student in frozen DINOv2 space, enabling supervision across incompatible latent spaces. To retain complementary capabilities without cross-teacher interference, Poly-OPD uses a gradient compatibility diagnostic to organize its adapters: attention LoRA modules are shared across teachers, whereas feed-forward adapters remain teacher-specific. During distillation, a gap-aware curriculum devotes more training to compositional categories where the student still falls short of the teacher. As each gap narrows, training shifts toward categories with larger remaining gaps. By distilling FLUX.1-dev and Z-Image into a 2.5B SD3.5-Medium student, Poly-OPD improves GenEval from 67.3 to 73.3, surpassing both larger teachers, and raises DrawBench HPSv3 from 9.34 to 11.35, consolidating both strengths within a switchable model.
Siming Fu, Haojun Xu, Ruizhe He +9
1Joy Future Academy · 2Zhejiang University · 3Beihang University
We introduce UNITY, a Universal-to-Specialized adapter for efficient and scalable composite conditioning in diffusion based image generation. Unlike prior methods that train separate adapters for each conditioning modality, UNITY jointly learns shared semantics across multiple conditioning types and subsequently specializes without modifying the underlying architecture. The proposed two stage training paradigm consists of a Universal Stage that captures cross modal representations across all conditioning modalities using half of the total training steps, followed by a Specialization Stage that refines modality specific features using the remaining training budget. At the core of UNITY are the Morphable Attention Flow (MAF) Network and Morph Wrapper modules, which enable channel aware and spatially adaptive feature alignment through learnable flow fields and attention based fusion. This constant complexity formulation supports flexible operation under both single and composite conditioning settings while significantly reducing inference latency and memory consumption. Extensive experiments across multiple datasets demonstrate that UNITY achieves state of the art image fidelity while maintaining superior memory efficiency. Code: https://github.com/arya-domain/UNITY
Aryan Das, Koushik Biswas, Moloud Abdar +1
VIT Bhopal, India · IIIT Delhi, India · The University of Queensland, Australia +1