Visual data are often expressed through multiple complementary modalities (e.g., images and segmentation masks), each capturing different aspects of the same underlying structure. Most existing multimodal methods adopt a top-down paradigm, learning a shared latent variable through joint training from scratch, which limits scalability. We propose Split and Bridge (SnB), a bottom-up framework for multimodal generation that couples pretrained unimodal diffusion models at sampling time. Rather than learning a unified latent space, SnB generates coherent multimodal samples by following a shared diffusion trajectory up to a split point and then branching into modality-specific trajectories guided by a bridge module. This design enables plug-and-play reuse of powerful pretrained models without joint training, making it scalable and easily adaptable. Across standard benchmarks (PolyMNIST, CelebAMask-HQ) and more complex settings (PIE-Bench++, ROCO), SnB achieves a strong balance between sample quality and coherence. We show that SnB naturally extends to downstream zero-shot tasks such as interleaved co-generation, image editing, and style coupling/decoupling. Our results demonstrate that bottom-up generation provides a practical framework that complements existing top-down multimodal approaches. Code is available at https://github.com/AhmadArrabi/SnB
Figures & tables
Figure 1: SnB leverages pretrained diffusion models by coupling reverse trajectories during sampling. We start from a common state sampled from a prior, then at tsplit , a bridge module is used to implement a score intervention on the reverse process. This intervention can be through modifying conditioning signals (e.g., prompts), changing diffusion models that estimate the score functions, or changing certain layers in the diffusion models. This flexibility applies SnB in various tasks, like interleaved co-generation (top left pair) and multimodal generation (top right pair). Learnable bridges allows stronger trajectory manipulation in mapping one modality to another.
Figure 2: PolyMNIST results using the identity bridge with SnB. SnB strikes a balance between coherence and quality across the 5 modalities.
Figure 3: Qualitative comparison of SnB and previous top-down multimodal approaches on CelebAMask-HQ. SnB balances between coherence and quality of generated modality pairs.
Method
Joint
M → F
F → M
Coherence ↑
FID ↓
Coherence ↑
FID ↓
Coherence ↑
F1 ↑
MoPoE Sutter et al. (2021)
0.59
151.23
0.50
163.83
0.51
0.72
CMVAE Palumbo et al. (2024)
0.52
303.50
0.48
233.04
0.47
0.56
MMVAE Shi et al. (2019)
0.58
358.21
0.46
375.79
0.47
0.58
MMVAE+ Palumbo et al. (2023)
0.56
345.29
0.48
202.86
0.47
0.57
MMVM VAE Sutter et al. (2024)
0.51
178.87
0.65
218.69
0.67
0.73
Table 1: Results of multimodal generation on CelebAMask-HQ. SnB achieved the highest coherence without losing generation quality in conditional and unconditional generation. † and * refer to the used backbones: SD and ours, respectively.
Figure 4: conditional generation of SnB on CelebAMask-HQ.
Ablation
Joint
M → F
F → M
Br.
N.I.
Coh. ↑
FID ↓
Coh. ↑
FID ↓
Coh. ↑
F1 ↑
✓
✓
0.73
60.84
0.60
107.22
0.72
0.85
✓
×
0.76
59.89
0.50
216.39
0.70
0.78
×
✓
0.48
60.82
0.50
52.03
0.48
0.53
×
×
0.48
59.90
0.50
292.58
0.49
0.02
Table 2: Ablation study of SnB. Br. and N.I. denote the bridge and noise injector. Coh. is coherence.
Bridge
Base
Params
Mem.
Variant
Depth
Channels
(M)
(MB)
Coh. ↑
FID ↓
Small
2
16
2.1
8.37
0.68
60.14
Medium
4
32
15.8
60.29
0.73
60.84
Large
6
64
233.3
890.18
0.76
60.72
Table 3: Ablation study on the learnable bridge ( Bθ ) capacity on CelebAMask-HQ at tsplit=50% . Medium denotes the default configuration used in all other experiments.
Model
SnB Variant
Interleaved Co-Generation
Image Editing
Target
Source
Consistency
Target
FID ↓
CLIP ↑
FID ↓
CLIP ↑
FID ↓
LPIPS ↓
MSE ↓
PSNR ↑
FID ↓
LPIPS ↓
MSE ↓
PSNR ↑
CLIP ↑
Flux Labs et al. (2025)
Independent
76.60
32.58
75.27
32.18
61.93
0.67
0.10
10.00
-
-
-
-
-
shared noise
76.09
32.38
74.74
32.18
56.32
0.48
0.03
14.76
78.24
0.76
0.15
8.25
32.52
SnB
75.95
31.79
74.86
32.23
51.69
0.39
0.02
17.03
76.94
0.70
0.09
10.55
32.45
SDXL Podell et al. (2024)
Independent
77.61
33.59
74.14
32.63
70.96
0.69
0.08
11.03
-
-
-
-
-
Table 4: Quantitative evaluation on PIE-Bench for interleaved co-generation and image editing tasks.
Figure 6: Sensitivity analysis on tsplit on PolyMNIST
Figure 7: Editing on PIE-Bench++. a gray horse in the field → a black cow with white patterns in the field.
Anatomy
Split Point
PubMedCLIP Eslami et al. (2023) ↑
FID ↓
Consistency
T1
T2
FLAIR
DWI
PD
T1
T2
FLAIR
DWI
PD
MSE ↓
PSNR ↑
LPIPS ↓
Brain
Independent
28.87
29.42
30.58
27.43
-
137.37
152.98
145.25
134.90
-
0.13
8.79
0.60
Shared Noise
28.95
29.28
30.70
27.38
-
137.57
154.88
138.01
132.59
-
0.04
14.06
0.38
SnB
28.85
29.17
30.24
27.08
-
134.67
147.44
134.37
132.46
-
0.02
17.90
0.27
Hip/Shld/Elbow
Independent
30.96
31.15
-
-
30.13
258.82
256.06
-
-
268.05
0.16
7.93
0.61
Shared Noise
31.38
31.34
-
-
30.04
247.53
255.18
-
-
254.40
0.04
14.60
0.34
Table 5: Quantitative results across three medical imaging categories of ROCO. Modalities not present in a specific anatomy are marked with a dash (-)
Figure 8: Interleaved co-generation on PIE-Bench++. SnB shows the most correlated co-generated pairs. Prompt 1: a girl and her dog in a field. Prompt 2: a boy and his monkey in a field.
Figure 9: Qualitative PIE-Bench++ style results. SnB demonstrates compositional abilities in combining and separating modalities. Prompt: a bird standing on tree branch.
Figure 10: Independent models lead to completely different structures, sharing the noise component increases alignment, while SnB is the most aligned.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: A sample of PolyMNIST. Unlike prior works that use binary segmentation masks (right samples), we use a more comprehensive mask with 15 classes (center samples) [ 4 , 53 ] .
Figure 12: A sample of PIE-Bench++. The prompts can be leveraged for interleaved co-generation.
Figure 13: Contrastive learning framework used to learn the embedding space for coherence measurement. Each representation has its own ResNet-34 model, sharing the architecture not the weights.
Initial trajectory
joint
M → F
F → M
coherence ↑
FID ↓
coherence ↑
FID ↓
coherence ↑
F1 ↑
face-first
0.73
60.84
-
-
0.72
0.85
mask-first
0.61
97.20
0.60
107.22
-
-
Appendix
Table 6: Results of multimodal generation on CelebAMask-HQ at tsplit=50% with different initial trajectories. Starting with the face modality leads to more coherent samples.
Figure 14: Unconditional generation batched samples on PolyMNIST. All modalities share the same structure while only changing the backgrounds.
Figure 15: Unconditional generation batched samples. In all cases, the poses and face structure is shared between the segmentation masks and faces. We notice a slight bias towards blonde females in unconditional generation
Figure 16: Unconditional generation batched samples. In all cases, the poses and face structure is shared between the segmentation masks and faces. We notice a slight bias towards blonde females in unconditional generation
Figure 17: Unlike previous VAE-based approaches, we are able to generate high-resolution 512×512 samples. These samples provide much more detail and quality which increase the scalability of joint unconditional generation.
Figure 18: Face → mask conditional generation. The generated masks are aligned with the given conditioning faces. The poses are consistent, while some hair details may not be always captured
Figure 19: Mask → Face conditional generation. The generated faces are aligned with the given conditioning masks. The poses, hair length, and expressions are consistent.
Figure 20: Failure cases in conditional generation on CelebAMask-HQ. Some generated faces misalign with the conditioning masks, particularly for accessories such as glasses, hats, and earrings (rightmost image). We also observe occasional inconsistencies in hair styles or lengths.
Figure 21: Interleave cogeneration on random samples from PIE-Bench++. Using SnB always leads to correlated pairs compared to independently using pretrained models. These results were generated through FLUX.
Figure 22: Interleave cogeneration on random samples from PIE-Bench++. Using SnB always leads to correlated pairs compared to independently using pretrained models. These results were generated through FLUX.
Figure 23: Style coupling and decoupling using parameter-based score interventions with an identity bridge. We sequentially apply two LoRA adaptations at different timesteps, resulting in a composition of both styles (middle). The left columns show individual styles, while the right columns demonstrate decoupling, recovering each style from the combined output.
Figure 24: Style coupling and decoupling using parameter-based score interventions with an identity bridge. We sequentially apply two LoRA adaptations at different timesteps, resulting in a composition of both styles (middle). The left columns show individual styles, while the right columns demonstrate decoupling, recovering each style from the combined output.
Figure 25: 4 Randomly sampled brain MRIs from SnB. We provide modality reference images from ROCO for those not familiar with MRI contrast modalities. Notice how SnB is both faithful to modality specific feature while maintaining coherence.
Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.