Maximum Likelihood on the Joint (Data, Condition) Distribution for Solving Ill-Posed Problems with Conditional Flow Models

John S. Hyatt

cs.LGAug 24, 2022

Maximum Likelihood on the Joint (Data, Condition) Distribution for Solving Ill-Posed Problems with Conditional Flow Models

Authors: John S. Hyatt

Organizations: Computational and Information Sciences Directorate DEVCOM Army Research Laboratory Aberdeen Proving Ground, Maryland 21005

Abstract

In the absence of conditioning information, normalizing flow models are trained using maximum likelihood. Previous extensions to conditional settings have abandoned the simple maximum-likelihood approach, as obtaining the full joint distribution of data and conditions for training purposes (or equivalently, the true conditional distribution for each point in the training dataset) is impossible. However, under special conditions that nevertheless apply to a wide class of useful problems, it is possible to obtain a simple, computationally cheap proxy with the same global minimum as the conditional likelihood. Conditional flow models trained using this proxy do not require explicit knowledge of the conditional distribution, auxiliary networks or other specific architecture, or additional loss terms beyond maximum likelihood. They provide a natural and powerful way to solve a class of ill-posed inverse problems. I demonstrate these properties on easily visualized toy problems, then use the method to successfully generate class-conditional images and to reconstruct semantically degraded images via super-resolution.

Figures & tables

Figure 1: Schematic of the conditional flow model described in this paper (right), architecturally similar to a non-conditional Real NVP flow model (left) except for the presence of $Y$ in the inputs and outputs. Arrows on the outside of the figure represent squeeze/factor operations.

Figure 2: Toy discrete problem. Left side: Training (top row) and inference (bottom row) for a flow model $F$ conditioned on class label $y=\pm 1$ . Right side: Inference on $F$ , with $y\in[-2,2]$ in increments of $0.5$ ( $y=\pm 1$ are starred). The predictions degrade gracefully for out-of-distribution $y$ .

Figure 3: Inference for toy continuous problem, showing $\mathbf{z}=(z_{1},z_{2})$ and conditional distributions for $y^{\prime}\in[0,\pi/2,\pi,3\pi/2]$ . In both panels, each point is shaded proportional to $z_{1}$ . The conditional distributions all roughly preserve the spatial orientation of the original $p_{Z}$ .

Figure 4: Class-conditioned model predictions. (a) 25 randomly generated examples each for shirts and sneakers. (b) Interpolations along the same line in $\mathcal{Z}$ for different $\mathbf{y}$ . (c) Interpolations along a line in $\mathcal{Y}$ connecting two classes for fixed $\mathbf{z}$ (predictions on the true class labels are boxed).

Figure 5: Two-scale super-resolution of fashion-MNIST sneakers (top) and sandals (bottom), one example each. Column 1: ground truth. Column 2: $4\times 4$ -downsampled model input. Column 3: 10 different intermediate-resolution reconstructions. Columns 4-13: 10 different high-resolution reconstructions for each intermediate reconstruction in column 3. Image backgrounds have been cropped.

Figure 6: Histograms for evaluating how well the model maps validation examples to the latent prior for fashion-MNIST super-resolution step 1 (left side) and step 2 (right side). Each black line corresponds to one pixel in the input image domain. White lines show the prior.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: The compressed masking procedure for checkerboard masks (left) and channelwise masks (right). Dark squares are masked pixels, which are compressed out before being passed into the coupling layer.

Figure A.2: An example of $3\times 3$ kernels used in the convolutional model, with dilation factors 1, 2, and 4. Each dilated kernel spans two adjacent pixels in the next larger dilation.

Figure A.3: Generated digits from a class-conditional model trained on MNIST, conditioned on labels 0 (left) and 1 (right). Note that a small number of predicted pixels have unnaturally high or low values (particularly visible in row 1, column 3 of the 0-labeled images, where a single pixel is predicted to have higher intensity than the white background).

Figure C.1: Example of a conditional discrete problem where the target distributions overlap (so some points in different conditional distributions are distinguished only by label) and have different shapes and topologies. Left side: target data distributions. Right side: conditional distributions learned by a conditional flow model. Color represents input label on the left and output label on the right (they are identical for all mapped points).

Figure D.1: Histograms obtained by feeding validation data through the model in the $XY\rightarrow ZY^{\prime}$ direction, for class-conditional image generation on (a) MNIST and (b) fashion-MNIST, and for super-resolution on fashion-MNIST, (c) step one and (d) step two. Each pixel is represented by one black line, while the red line shows the prior. In the class-conditioned image generation, occasionally the model learns to map a small number of pixels, shown in green, to distributions significantly different from the prior.

Figure E.1: Two-scale super-resolution of an example from the fashion-MNIST T-shirt class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.2: Two-scale super-resolution of an example from the fashion-MNIST trouser class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.3: Two-scale super-resolution of an example from the fashion-MNIST pullover class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.4: Two-scale super-resolution of an example from the fashion-MNIST dress class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.5: Two-scale super-resolution of an example from the fashion-MNIST coat class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.6: Two-scale super-resolution of an example from the fashion-MNIST shirt class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.7: Two-scale super-resolution of an example from the fashion-MNIST bag class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Figure E.8: Two-scale super-resolution of an example from the fashion-MNIST ankle boot class. Column 1: original image, $\mathbf{x}^{\prime}_{0}$ . Column 2: $4\times 4$ -downsampled input to the model, $\mathbf{y}$ . Column 3: 10 different reconstructions of the image at $2\times 2$ -downsampled resolution, $\mathbf{x}_{1,i}^{\prime}$ . Columns 4-13: 10 different reconstructions of the image at its original resolution for each of the intermediate reconstructions in column 3, $\mathbf{x}_{0,ij}^{\prime}$ .

Conditional Generative Modeling

Conditional generative modeling learns to produce data distributions steered by labels, inputs, text, or physical and structural constraints.

See all 387 papers

Conditional flow matching for physics-constrained inverse problems with finite training data
Mar 14, 2026Agnimitra Dasgupta, Ali Fardisi, Mehrnegar Aminy +4Conditional Flow Matching Bayesian Inverse Problems
Department of Aerospace & Mechanical Engineering, University of Southern California, Los Angeles, 90089, California, USA · Optimization and Uncertainty Quantification, Sandia National Laboratories, Albuquerque, 87123, New Mexico, USA
FIRM: Flow-based Imaging via Regularized Minimization
Sep 11, 2026Shirin Shoushtari, Edward P. Chandler, Xiao Shi +1Conditional Flow Matching Image Reconstruction
WashU · UW-Madison
Trajectory Stitching for Solving Inverse Problems with Flow-Based Models
Feb 9, 2026Alexander Denker, Zeljko Kereta, Carola-Bibiane Schönlieb +1Generative Flow Networks Flow-Based Generative Modeling
Department of Computer Science, University College London · Department of Applied Mathematics and Theoretical Physics, University of Cambridge.

Mar 14, 2026J/K move · Enter open · S save

Conditional flow matching for physics-constrained inverse problems with finite training data

Agnimitra Dasgupta, Ali Fardisi, Mehrnegar Aminy, Brianna Binder+3

Department of Aerospace & Mechanical Engineering, University of Southern California, Los Angeles, 90089, California, USA · Optimization and Uncertainty Quantification, Sandia National Laboratories, Albuquerque, 87123, New Mexico, USA

Conditional Flow Matching Bayesian Inverse Problems Physics-Informed Generative Modeling Memorization in Generative Models

Open paper AskPDF arXiv

This study presents a conditional flow matching framework for solving physics-constrained Bayesian inverse problems. In this setting, samples from the joint distribution of inferred variables and measurements are assumed available, while explicit evaluation of the prior and likelihood densities is not required. We derive a simple and self-contained formulation of both the unconditional and conditional flow matching algorithms, tailored specifically to inverse problems. In the conditional setting, a neural network is trained to learn the velocity field of a probability flow ordinary differential equation that transports samples from a chosen source distribution directly to the posterior distribution conditioned on observed measurements. This black-box formulation accommodates nonlinear, high-dimensional, and potentially non-differentiable forward models without restrictive assumptions on the noise model. We further analyze the behavior of the learned velocity field in the regime of finite training data. Under mild architectural assumptions, we show that overtraining can induce degenerate behavior in the generated conditional distributions, including variance collapse and a phenomenon termed selective memorization, wherein generated samples concentrate around training data points associated with similar observations. A simplified theoretical analysis explains this behavior, and numerical experiments confirm it in practice. We demonstrate that standard early-stopping criteria based on monitoring test loss effectively mitigate such degeneracy. The proposed method is evaluated on several physics-based inverse problems. We investigate the impact of different choices of source distributions, including Gaussian and data-informed priors. Across these examples, conditional flow matching accurately captures complex, multimodal posterior distributions while maintaining computational efficiency.

Abstract

Conditional Generative Modeling

Explore similar work

Conditional flow matching for physics-constrained inverse problems with finite training data

FIRM: Flow-based Imaging via Regularized Minimization

Trajectory Stitching for Solving Inverse Problems with Flow-Based Models