Drift Variation Autoencoder: Unifying Generation and Representation Learning through Conditional Posterior Flow Matching

Jiarui Cao

cs.LGAug 25, 2026

Drift Variation Autoencoder: Unifying Generation and Representation Learning through Conditional Posterior Flow Matching

Authors: Jiarui Cao

Organizations: The Chinese University of Hong Kong

Abstract

Stochastic masking, cropping, or modality removal makes deterministic reconstruction an incomplete target: one observation can admit many clean completions. This work takes the corresponding posterior $P(X\mid C)$ as the common statistical object for conditional generation and generatively sufficient representation learning. Drift Variation autoencoder trains a masked encoder $Z=E(C)$ and a conditional flow decoder with one clean-prediction Flow Matching loss. The analysis first decomposes the ideal conditional KL into generator approximation and the representation deficiency $I(X;C\mid Z)$ . It then derives orthogonal risk decompositions for conditional Flow Matching. For an affine Gaussian path, the clean-prediction representation gap is zero if and only if $P(X\mid Z)=P(X\mid C)$ . Thus the encoder-dependent excess clean-prediction risk induced by Flow Matching and the profiled ideal conditional KL have the same posterior-sufficient zero set, without being numerically equal objectives. An exact conditional field with a zero-noise endpoint then generates $P(X\mid Z)$ and hence $P(X\mid C)$ at a joint ideal optimum. The result extends to continuous multimodal product spaces when the complete modality tuple remains the Flow target for every observation mask. On CrossGeom-4, an 18-run controlled benchmark, observable factors have linear-probe $R^2$ of $0.9990$ - $0.9992$ , shuffling the joint model's encoder condition increases conditional error by $13.5\times$ - $15.7\times$ , and joint target attention reduces disagreement on an unobserved factor shared by two outputs by $90.1$ - $92.8\%$ relative to independent target decoders. Visible modalities are also generated and reconstructed, directly validating the full-tuple objective. Unconditional mode balance remains imperfect, delimiting the empirical claim to a controlled multimodal proof of concept.

Figures & tables

Input: encoder $E_{\theta}$ , clean predictor $D_{\phi}$ , augmentation law $A$ , time law $\rho$ , weight $w$
1	Sample a clean example $X$ and augmentation randomness, then form $C=A(X,\Xi_{A})$ .
2	Encode only the observation: $Z=E_{\theta}(C)$ .
3	Sample $X_{0}\sim\mathcal{N}(0,s_{0}^{2}I)$ independently and $t\sim\rho$ .
4	Form $\sigma_{t}=1-(1-\sigma_{\min})t$ and $X_{t}=tX+\sigma_{t}X_{0}$ .
5	Predict $\widehat{X}=D_{\phi}(X_{t},t,Z)$ and compute $w(t)\left\lVert\widehat{X}-X\right\rVert^{2}$ .
6	Backpropagate jointly through $D_{\phi}$ and $E_{\theta}$ ; no stopped target or external teacher is required by the theory.

Table 1: Drift Variation autoencoder training with an affine Gaussian path. Every prose step denotes a minibatch operation.

	Single-input conditional MAE		Unknown target-target MAE			Unknown-factor W1			Uncond. joint
Level	Ind.	Joint	Ind.	Joint	Reduction	Ind.	Joint	Reduction	reduction
L1	.1166	.1176	1.6397	.1186	92.8%	.6776	.2067	69.5%	88.8%
L2	.1145	.1154	1.6278	.1201	92.6%	.6878	.2669	61.2%	89.4%
L3	.1310	.1338	1.5651	.1542	90.1%	.7143	.1648	76.9%	86.6%

Table 2: Joint coupling under full-tuple Flow Matching. Conditional MAE is similar for matched independent (Ind.) and joint decoders. Joint target attention specifically reduces disagreement and distribution error for the unknown factor shared by two outputs. Values are three-seed means; complete standard deviations appear in Appendix G .

Figure 2: Known-factor conditional MAE, disagreement on the jointly sampled unknown factor, and unconditional TV-to-uniform. Joint and independent decoders have similar conditional accuracy, but only joint target attention coordinates residual uncertainty. Error bars are standard deviations across the three seeds, computed with denominator

n

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Required report
Observation	Augmentations, strengths, mask law, content dependence, and all declared side information.
Representation	Encoder/token architecture, pooling, frozen and nonlinear probes, rank statistics, and checkpoint rule.
Flow	Source, schedules, time law, weighting/clipping, endpoint noise, solver, steps, and final-time handling.
Generation	Fidelity, precision/recall, condition consistency, repeated-sample diversity, and sample count.
Multimodal	Visible subsets, full generated targets, per-modality fidelity, visible reconstruction, and joint consistency.
Ablations	Shuffled/zero condition, source construction, time allocation, bottleneck size, and joint versus independent targets.

Appendix

Table 3: Minimum information for evaluating a Drift Variation autoencoder instance.

Level	Decoder	Uncond. TV $\downarrow$	Uncond. joint MAE $\downarrow$	A $\to$ BC $\downarrow$	B $\to$ AC $\downarrow$	C $\to$ AB $\downarrow$	AB $\to$ C $\downarrow$	AC $\to$ B $\downarrow$	BC $\to$ A $\downarrow$
1	independent	0.0892 $\pm$ 0.0111	1.6462 $\pm$ 0.0460	0.1147 $\pm$ 0.0044	0.1177 $\pm$ 0.0072	0.1173 $\pm$ 0.0113	0.0962 $\pm$ 0.0029	0.0950 $\pm$ 0.0081	0.0943 $\pm$ 0.0006
1	joint	0.0990 $\pm$ 0.0128	0.1847 $\pm$ 0.0096	0.1123 $\pm$ 0.0093	0.1212 $\pm$ 0.0076	0.1193 $\pm$ 0.0064	0.0990 $\pm$ 0.0124	0.1011 $\pm$ 0.0136	0.0966 $\pm$ 0.0096
2	independent	0.0724 $\pm$ 0.0022	1.6674 $\pm$ 0.0426	0.1197 $\pm$ 0.0132	0.1132 $\pm$ 0.0130	0.1105 $\pm$ 0.0092	0.0975 $\pm$ 0.0153	0.0952 $\pm$ 0.0112	0.0831 $\pm$ 0.0116
2	joint	0.1082 $\pm$ 0.0186	0.1771 $\pm$ 0.0116	0.1055 $\pm$ 0.0040	0.1204 $\pm$ 0.0118	0.1203 $\pm$ 0.0119	0.0923 $\pm$ 0.0026	0.0968 $\pm$ 0.0009	0.1061 $\pm$ 0.0202
3	independent	0.0542 $\pm$ 0.0132	1.5977 $\pm$ 0.0220	0.1260 $\pm$ 0.0018	0.1543 $\pm$ 0.0112	0.1126 $\pm$ 0.0064	0.1150 $\pm$ 0.0083	0.0952 $\pm$ 0.0006	0.1072 $\pm$ 0.0090
3	joint	0.0861 $\pm$ 0.0172	0.2133 $\pm$ 0.0164	0.1295 $\pm$ 0.0178	0.1583 $\pm$ 0.0143	0.1137 $\pm$ 0.0022	0.1196 $\pm$ 0.0156	0.1196 $\pm$ 0.0200	0.1127 $\pm$ 0.0079

Appendix

Table 4: Conditional factor MAE and unconditional metrics under full-tuple Flow Matching. Every conditional query generates A, B, and C; direction labels identify the visible condition, not the only active targets.

Level	Decoder	Probe $R^{2}\uparrow$	Random R 2	Visible recon. $\downarrow$	Single cond. $\downarrow$	Double cond. $\downarrow$	Unknown joint $\downarrow$	Unknown W1 $\downarrow$	Both modes $\uparrow$	Shuffle $\times\uparrow$
1	independent	0.9991 $\pm$ 0.0001	0.8227 $\pm$ 0.0858	0.0840 $\pm$ 0.0038	0.1166 $\pm$ 0.0059	0.0951 $\pm$ 0.0033	1.6397 $\pm$ 0.0432	0.6776 $\pm$ 0.0117	1.0000 $\pm$ 0.0000	15.6921 $\pm$ 0.6424
1	joint	0.9992 $\pm$ 0.0000	0.8227 $\pm$ 0.0858	0.0918 $\pm$ 0.0066	0.1176 $\pm$ 0.0076	0.0989 $\pm$ 0.0114	0.1186 $\pm$ 0.0099	0.2067 $\pm$ 0.0539	1.0000 $\pm$ 0.0000	15.2957 $\pm$ 1.5546
2	independent	0.9992 $\pm$ 0.0000	-13.2173 $\pm$ 17.3786	0.0809 $\pm$ 0.0089	0.1145 $\pm$ 0.0104	0.0919 $\pm$ 0.0117	1.6278 $\pm$ 0.0313	0.6878 $\pm$ 0.0109	1.0000 $\pm$ 0.0000	16.4248 $\pm$ 1.9564
2	joint	0.9992 $\pm$ 0.0000	-13.2173 $\pm$ 17.3786	0.0912 $\pm$ 0.0069	0.1154 $\pm$ 0.0089	0.0984 $\pm$ 0.0078	0.1201 $\pm$ 0.0107	0.2669 $\pm$ 0.0309	1.0000 $\pm$ 0.0000	15.6814 $\pm$ 0.9940
3	independent	0.9992 $\pm$ 0.0000	0.7396 $\pm$ 0.2158	0.0910 $\pm$ 0.0050	0.1310 $\pm$ 0.0060	0.1058 $\pm$ 0.0054	1.5651 $\pm$ 0.0261	0.7143 $\pm$ 0.0133	0.9991 $\pm$ 0.0012	14.1745 $\pm$ 0.6261
3	joint	0.9990 $\pm$ 0.0001	0.7396 $\pm$ 0.2158	0.1041 $\pm$ 0.0069	0.1338 $\pm$ 0.0099	0.1173 $\pm$ 0.0122	0.1542 $\pm$ 0.0080	0.1648 $\pm$ 0.0516	1.0000 $\pm$ 0.0000	13.5004 $\pm$ 1.1064

Appendix

Table 5: Representation, visible reconstruction, uncertainty, and condition-use evidence. “Both modes” is the fraction of contexts whose 16 samples contain both signs of the shared unknown factor. “Shuffle” is shuffled-condition MAE divided by matched-condition MAE with identical initial ODE noise.

Conditional Generative Modeling

Conditional generative modeling learns to produce data distributions steered by labels, inputs, text, or physical and structural constraints.

See all 387 papers

MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation
Oct 7, 2026Kyeongmin Yeo, Minhyuk SungMultimodal Generation Unified Multimodal Models
KAIST
From Feed-Forward to Flow: Unifying Reconstruction and Generation Is Easier Than You Think
Sep 26, 2026Haoru Wang, Qianfan Shen, Kai Ye +2Flow-Based Generative Modeling Feed-Forward 3D Reconstruction
Peking University
Discriminative Flow Matching: Beyond Time-Conditioning in Generative Restoration via Flow-State Representations
Sep 3, 2026Shrishti Saha Shetu, Emanuël A. P. Habets, Andreas BrendelFlow Matching Conditional Generative Modeling
International Audio Laboratories Erlangen∗ Am Wolfsmantel 33, 91058 Erlangen, Germany · Fraunhofer IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany

Oct 7, 2026J/K move · Enter open · S save

MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation

Kyeongmin Yeo, Minhyuk Sung

KAIST

Multimodal Generation Unified Multimodal Models Conditional Flow Matching

Open paper AskPDF arXiv

We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same inference problem under different amounts of observed evidence. Given any subset of modalities, MUNITE models the conditional distribution over the latent representation associated with the complete observation. Full observation recovers deterministic encoding, no observation recovers the latent marginal, and intermediate subsets define conditional latent inference, all within a single conditional flow model. A shared latent sample captures variation that must remain consistent across generated targets, while modality-specific generative decoders model the remaining uncertainty independently. To learn these conditional distributions from incomplete training examples, we extend conditional flow matching through self-distillation: predictions conditioned on richer available observations supervise the same model conditioned on smaller subsets at the same intermediate latent state. When the richer-evidence trajectory follows the exact conditional flow, this provides the same expected learning signal as full-target denoising. Across PolyMNIST-D-Q, FFHQ64, and image-text-audio, MUNITE achieves competitive or better generation quality and source-target alignment, with higher joint-generation coherence. In particular, it attains the highest coherence in all one-to-many and unconditional image-text-audio comparisons, showing the effectiveness of unified latent inference across diverse multimodal settings.

Abstract

Conditional Generative Modeling

Explore similar work

MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation

From Feed-Forward to Flow: Unifying Reconstruction and Generation Is Easier Than You Think

Discriminative Flow Matching: Beyond Time-Conditioning in Generative Restoration via Flow-State Representations