Organizations: Department of Biostatistics and Health Data Science, University of Pittsburgh · Department of Electrical and Computer Engineering, University of Virginia · Department of Electrical and Computer Engineering, University of Pittsburgh · Department of Pediatrics, University of Pittsburgh · Department of Mathematical Sciences, University of Texas at Dallas
Many multimodal alignment problems require mapping a source modality into a fixed space defined by a target modality, where the target space is not merely an intermediate embedding but is used directly for downstream analysis. In this setting, successful alignment should preserve both the global structure of the target distribution and the instance-level correspondence observed in paired source--target training data. Existing objectives typically address only part of this goal: contrastive losses encourage relative similarity, distribution-matching losses improve marginal agreement, and pointwise regression fits each source sample to its paired target without explicitly modeling target-space path geometry. We propose FlowAlign, a flow-guided target-space alignment method that repurposes conditional flow matching as a training-time alignment signal rather than an inference-time transport model. FlowAlign uses the learned flow field to guide the alignment of source-side representations with the fixed target space, encouraging them to preserve paired correspondence while remaining compatible with the global target-space structure. We further provide theoretical analysis showing how FlowAlign helps improve paired correspondence by controlling endpoint accuracy and capturing local target-space geometry. Experiments on synthetic data, paired single-cell multiome benchmarks, and image--text alignment demonstrate that FlowAlign improves instance level correspondence while preserving target-space structure across different modality settings.
Figures & tables
Figure 1: Path-consistency score. (a) Field learning on paired source-to-target interpolation paths. (b) A compatible candidate representation yields a small velocity residual. (c) An incompatible candidate representation yields a large velocity residual.
Figure 2: Synthetic fixed-target alignment. (A) Source and target manifolds colored by latent coordinate a . (B) Predicted target-space points. (C) Predicted versus true latent positions.
Table 2: Performance comparison on held-out single-cell multiome alignment. All methods are evaluated using the same paired-cell and biological-structure metrics.
Figure 3: BMMC held-out target-space visualization. (A) UMAP overlap between held-out RNA embeddings and mapped ATAC embeddings. (B) RNA reference embeddings colored by cell type. (C) Mapped ATAC embeddings colored by the same cell-type labels.
Method
FOSCTTM ↓
R@1 ↑
LTA (%) ↑
Identity
0.499
0.003
13.31
MSE
0.069
0.067
57.49
MSE+MMD
0.076
0.052
63.95
MSE+InfoNCE
0.078
0.051
59.77
MSE+Neighborhood
0.074
0.063
68.90
FlowAlign
0.063
0.075
70.42
Table 3: Performance comparison on paired image–text alignment using CUBICC. All methods map DINOv3 image representations into the fixed 128-dimensional SigLIP2 text space.
Figure 4: Effect of the local-start scale σ on PBMC.
Variant
FOSCTTM ↓
Cell match ↑
LTA ↑
NMI ↑
ARI ↑
FlowAlign
0.039
0.141
0.920
0.872
0.885
σ=0
0.063
0.082
0.812
0.798
0.747
Source-free field
0.055
0.104
0.872
0.809
0.822
CFM transport
0.034
0.142
0.892
0.826
0.834
Degenerate case
0.047
0.115
0.902
0.857
0.865
Table 4: Ablation study results on PBMC dataset.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
PBMC
BMMC
Latent dimension
25
25
Stage-1 input dimension
114
114
Stage-1 hidden dimension
128
128
Stage-1 residual blocks
6
6
Stage-1 batch size
256
256
Stage-1 learning rate
3×10−4
3×10−4
Appendix
Table 5: Network and training hyperparameters for PBMC and BMMC experiments.
Figure 5: Visualization of baseline results of synthetic fixed-target alignment.
Figure 6: PBMC held-out target-space visualization. (A) RNA reference embeddings colored by cell type. (B) Mapped ATAC embeddings colored by the same cell-type labels. (C) UMAP overlap between held-out RNA embeddings and mapped ATAC embeddings
Method
PBMC
BMMC
FOSCTTM med. ↓
MRR ↑
R@5 ↑
R@10 ↑
R@50 ↑
FOSCTTM med. ↓
MRR ↑
R@5 ↑
R@10 ↑
R@50 ↑
Seurat v5
0.026
0.079
0.096
0.185
0.564
0.020
0.032
0.030
0.064
0.250
GLUE
0.023
0.139
0.155
0.247
0.621
0.024
0.046
0.027
0.059
0.312
MultiVI
0.053
0.043
0.042
0.100
0.279
0.034
0.028
0.021
0.056
0.196
Cobolt
0.053
0.050
0.061
0.156
0.555
0.042
0.029
0.017
0.041
0.242
FlowAlign
0.011
0.241
0.321
0.464
0.797
0.006
0.116
0.158
0.237
0.528
Appendix
Table 6: Additional paired-cell retrieval metrics on PBMC and BMMC. Lower FOSCTTM median and mean rank are better; higher MRR and recall values are better.
Method
PBMC
BMMC
Balanced LTA ↑
Macro-F1 ↑
mAP ↑
Balanced LTA ↑
Macro-F1 ↑
mAP ↑
Seurat v5
0.764
0.762
0.809
0.648
0.652
0.642
GLUE
0.877
0.876
0.823
0.712
0.714
0.651
MultiVI
0.721
0.698
0.682
0.632
0.628
0.483
Cobolt
0.768
0.769
0.521
0.589
0.584
0.312
FlowAlign
0.894
0.896
0.787
0.765
0.770
0.576
Appendix
Table 7: Additional biological-structure metrics on PBMC and BMMC. Higher values indicate better performance for all metrics.
Method
FOSCTTM ↓
Cell match ↑
LTA ↑
NMI ↑
ARI ↑
ASW-O ↑
Seurat v5
0.058±0.000
0.019±0.000
0.850±0.000
0.783±0.000
0.785±0.000
0.995±0.000
GLUE
0.044±0.003
0.016±0.003
0.914±0.019
0.802±0.021
0.813±0.018
0.979±0.009
MultiVI
0.093±0.011
0.007±0.002
0.834±0.049
0.784±0.042
0.753±0.051
0.830±0.035
Cobolt
0.088±0.014
0.012±0.003
0.812±0.034
0.799±0.042
0.786±0.050
0.921±0.021
FlowAlign
0.039±0.004
0.141±0.009
0.920±0.012
0.872±0.020
0.885±0.025
0.978±0.006
Appendix
Table 8: Seed variability on the PBMC benchmark.
Method
FOSCTTM ↓
Cell match ↑
LTA ↑
NMI ↑
ARI ↑
ASW-O ↑
Seurat v5
0.045±0.000
0.008±0.000
0.756±0.000
0.714±0.000
0.655±0.000
0.978±0.000
GLUE
0.056±0.002
0.015±0.002
0.745±0.013
0.701±0.017
0.689±0.019
0.981±0.003
MultiVI
0.061±0.006
0.011±0.003
0.777±0.047
0.642±0.059
0.585±0.058
0.782±0.010
Cobolt
0.097±0.012
0.006±0.002
0.742±0.024
0.740±0.032
0.512±0.035
0.946±0.006
FlowAlign
0.019±0.001
0.050±0.005
0.836±0.014
0.797±0.019
0.768±0.021
0.988±0.003
Appendix
Table 9: Seed variability on the BMMC benchmark.
Model
FOSCTTM ↓
R@1 ↑
LTA ↑
NMI ↑
Peak AUROC ↑
Peak AUPR ↑
Pretrained PeakVI
0.548
0.000
0.047
0.457
0.782
0.637
PeakVI + FlowAlign reg.
0.068
0.046
0.815
0.801
0.765
0.630
Appendix
Table 10: Performance of FlowAlign as a plug-in regularizer for PeakVI-to-scVI latent alignment on PBMC. FlowAlign improves alignment of the pretrained PeakVI ATAC latent space to the fixed scVI RNA space while largely preserving peak reconstruction performance.
Modality gap significantly restricts the effectiveness of multimodal fusion. Previous methods often use techniques such as diffusion models and adversarial learning to reduce the modality gap, but they typically focus on one-to-one alignment without exposing the data points of the source modality to the global distribution information of the target modality. To this end, leveraging the characteristic of rectified flow that can map one distribution to another via a straight trajectory, we extend rectified flow for modality distribution mapping. Specifically, we leverage the one-to-many mapping' strategy in rectified flow that allows each data point of the source modality to observe the overall target distribution. This also alleviates the issue of insufficient paired data within each sample, enabling a more robust distribution transformation. Moreover, to achieve more accurate distribution mapping and address the ambiguous flow directions in one-to-many mapping, we design adaptive relaxed alignment', enforcing stricter alignment for modality pairs belonging to the same sample, while applying relaxed mapping for pairs not belonging to the same sample or category. Additionally, to prevent information loss during distribution mapping, we introduce `cyclic rectified flow' to ensure the transferred features can be translated back to the original features, allowing multimodal representations to learn sufficient modality-specific information. After distribution alignment, our approach achieves very competitive results on multiple tasks of multimodal affective computing even with a simple fusion method, and visualizations verify that it can effectively reduce the modality gap.
We study path-flow alignment as a unified training objective for flow matching. Instead of fixing the interpolation path and learning only the velocity field, we jointly train an endpoint-preserving path network and a flow network using the same alignment loss: the flow learns to match the path velocity, and the path learns to align its velocity to the current flow. Although every fixed learned path defines a valid flow-matching objective, the alignment loss alone is not a reliable criterion for path learning. We identify path overfitting, a failure mode in which the alignment loss decreases while sample quality worsens. We find that this failure is associated with low-entropy bottlenecks in the induced probability path, where the learned path routes samples through overly concentrated intermediate marginals. Motivated by this diagnosis, we introduce a stochastic path regularizer that hides part of the source information from the path network while preserving exact endpoints. The resulting regularization gives an explicit entropy floor for the stochastic training-path marginals and empirically suppresses the bottleneck in the learned sampler, making joint path-flow training effective. On ImageNet-256x256 with SiT backbones, our method consistently improves FID across model scales, extends to model-guidance training, and leaves the inference-time architecture and sampler unchanged. Code is available at https://github.com/lizeyu090312/traj_opt_paper
Multimodal representations enable zero-shot classification and retrieval, but aligning independently trained models usually requires large amounts of paired data. Yet, the Platonic Representation Hypothesis suggests that models trained on different modalities may converge spontaneously toward a shared representation geometry. But then, do we even need paired examples for cross-modal alignment? Remarkably, we show that paired examples are unnecessary for coarse cross-modal alignment. Our simple Wasserstein Procrustes method with a coarse geometric initialization (GeoWarP) aligns two disjoint embedding sets by estimating a single orthogonal map without seeing any pairs. Across datasets, modalities, and unimodal models, we show that we can consistently align independently trained representations without pairs, and standard geometric alignment metrics accurately predict when this is possible. Nevertheless, we can naturally benefit from paired examples. In the very few-pair regime, our method substantially outperforms existing ones, while staying competitive with pair-based methods with more added examples. Finally, we demonstrate that the resulting alignments can enable text-to-image generation without paired examples. These results show that independently trained models often share enough geometry to establish cross-modal correspondence with little or no paired data.