cs.LGFeb 24, 2026

VINA: Variational Invertible Neural Architectures

Authors: Shubhanshu Shekhar, Mohammad Javad Khojasteh, Ananya Acharya, Tony Tohme, Kamal Youcef-Toumi

Organizations: EME Department, Rochester Institute of Technology · Department of Mechanical Engineering, Massachusetts Institute of Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural networks (INNs) build on these principles to address supervised inverse problems, enabling direct modeling of both forward and inverse mappings. In this paper, we revisit these architectures from both theoretical and practical perspectives and address a key gap in the literature: the lack of theoretical guarantees on approximation quality under realistic assumptions, whether for posterior inference in INNs or for generative modeling with NFs. We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions, inspired by analogous formulations in related areas such as generative adversarial networks (GANs) and the Precision-Recall divergence for training normalizing flows. Within this framework, we derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs, under assumptions that are weaker and more practically realistic than those used in prior work. Building on these theoretical results, we conduct extensive case studies to distill general design principles and practical guidelines. We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.

Figures & tables

Figure 1: INN can be represented by an invertible map TT that approximates the relation from the input X∈RdxX\in\mathbb{R}^{d_{\mathbf{x}}} to the output Y∈RdyY\in\mathbb{R}^{d_{\mathbf{y}}} and a latent variable Z∈Rdx−dyZ\in\mathbb{R}^{d_{\mathbf{x}}-d_{\mathbf{y}}} . The invertibility of TT means that for any y∈Rdy\mathbf{y}\in\mathbb{R}^{d_{\mathbf{y}}} , we also have an approximate posterior sampling distribution via T−1(y,Z)T^{-1}(\mathbf{y},Z) .
NFNFINNINN
IKIKBNNBNN
NLLNLLMMDMMD
ERMERMKLKL
JSJSIPMIPM
SLSLUSLUSL
GANGANGIGI
Table 1: Definitions of important acronyms.
Figure 2: The four plots illustrate the results of the inverse kinematics case study with different prior regularizations. (a) shows samples from a model trained without any prior loss (i.e., with prior weight λ′=0\lambda^{\prime}=0 ). (b) and (c) display samples from models trained with a prior weight of λ′=1\lambda^{\prime}=1 . In (b), the model assumes a Gaussian prior over the joint configuration, resulting in a scaled L2L_{2} loss between the ground truth and the generated samples. In contrast, (c) uses a uniform prior U(0,1)\mathcal{U}(0,1) , which is inconsistent with the true distribution. (d) illustrates results from a model trained with a Gaussian prior, using λ′=100\lambda^{\prime}=100 . The cross indicates the true end-effector position. All models were trained using the NLL loss formulation for the INN.
Figure 3: Schematic illustration of how different training paradigms are used to solve inverse problems in the Gaussian setting. (a) Bayesian neural network for solving inverse problem. There is supervised cost (SC) between predicted and true x . (b) INN Trained without \prescriptpLx\prescript{p}{}{L_{x}} . There is unsupervised cost (USC) between predicted and true x . (c) Shows an INN model trained with \prescriptpLx\prescript{p}{}{L_{x}} , which serves as a reconstruction loss.
Figure 4: Comparison of ff -divergence performance for coupling-based and iResNet architectures in the inverse kinematics case study. The x-axis shows training time and the y-axis shows ℓ2\ell_{2} -distance between true and generated end-effector positions. Ellipses are centered at the mean values, with horizontal and vertical diameters indicating the standard deviations in training time and ℓ2\ell_{2} -distance. The right and left columns correspond to forward loss ( 51 ) and backward losses ( 50 ), respectively. The top row corresponds to the coupling-based architecture whereas the figures on the bottom row generated based on iResNet layers.
Figure 6
Figure 7: Top Row: Samples from NF models trained using the Sinkhorn divergence. The left image corresponds to a latent distribution U(0,1)\mathcal{U}(0,1) used to simulate the data, the middle image uses U(3,4)\mathcal{U}(3,4) , and the right image uses U(5,6)\mathcal{U}(5,6) . Bottom Row: Samples from NF models trained using an ff -divergence approximation of the forward KL divergence. The latent distributions mirror those used in the Sinkhorn divergence experiments.
Table 2: This table shows the MMD between samples from the true distribution U(0,1)\mathcal{U}(0,1) and those generated by the trained model.
Figure 8: This figure illustrates the Wasserstein-1 distance for increasing values of α\alpha in the Pareto distribution, which corresponds to increasing number of finite moments. The y-axis is set to a logarithmic (base 10) scale. Configurations 1–3 specify INN with 2, 6, 8 coupling layers and subnetworks of hidden sizes 128, 64, 128, respectively.
Figure 9: The SWellEx-96 experiment environment. The acoustic source is towed by a research vessel and transmits signals at various frequencies. The acoustic sensor consists of a vertical line array (VLA). Based on the measurements collected at the VLA, the objective is to estimate posterior distributions over parameters of interest (e.g. water depth, sound speed at the water-sediment interface, source range and depth, etc.).
Figure 10: Posterior comparisons between models trained with different loss functions for the GI dataset, using signal measurements from five hydrophones. The inferred parameter is water depth , with the red dotted line indicating the true value. The INN configuration for the upper row uses Configuration 1 whereas the configuration used in the bottom row is Configuration 2 . The y-axis shows un-normalized probability scores.
Figure 11: Posterior comparisons between models trained with different loss functions for the GI dataset, using signal measurements from five hydrophones. The inferred parameter is sound speed , with the red dotted line indicating the true value. The INN configuration for the upper row uses Configuration 1 whereas the configuration used in the bottom row is Configuration 2 .
Table 3: Comparison of the L2L_{2} loss for water depth using two different model configurations. The parameter being recovered is water depth.
Table 4: Comparison of the L2L_{2} loss for sound speed using two different model configurations introduced. The parameter being recovered is sound speed.
Table 5: Comparison of training times for two model configurations. The configuration used for the left table achieves an inference time of 0.0060 seconds per 1000 samples, while the configuration used for the right table achieves 0.0036 seconds for the same number of samples.
More in this topic

Invertible Neural Networks

Invertible neural networks are models with learnable mappings that can be reversed, supporting density estimation, generation, inference, and control.

See all 14 papers

Explore similar work

CardsList
Mar 16, 2026

Generative Inverse Design with Abstention via Diagonal Flow Matching

Inverse design aims to find design parameters xx achieving target performance y∗y^*. Generative approaches learn bidirectional mappings between designs and labels, enabling diverse solution sampling. However, standard conditional flow matching (CFM), when adapted to inverse problems by pairing labels with design parameters, exhibits strong sensitivity to their arbitrary ordering and scaling, leading to unstable training. We introduce Diagonal Flow Matching (Diag--CFM), which resolves this through a zero-anchoring strategy that pairs design coordinates with noise and labels with zero, making the learning problem provably invariant to coordinate permutations. This yields substantially lower round-trip error than CFM and invertible neural network baselines across design dimensions up to P=784P{=}784, including order-of-magnitude gains on several benchmarks. We develop two architecture-intrinsic uncertainty metrics, Zero-Deviation and Self-Consistency, that enable three practical capabilities: selecting the best candidate among multiple generations, abstaining from unreliable predictions, and detecting out-of-distribution targets; consistently outperforming ensemble and general-purpose alternatives across all tasks. We validate on airfoil, gas turbine combustor, scalable analytical benchmarks, a photonics inverse-design task, and an image-statistics benchmark.

Miguel de Campos, Werner Krebs, Hanno Gottschalk

Institute of Mathematics, TU Berlin · Siemens Energy

Flow MatchingUncertainty QuantificationFlow-Based Generative ModelingInverse Design
AskPDF
Oct 15, 2026

Normalizing Flow Maps

Scalable approaches to learning neural dynamical systems have placed diffusion, flow matching, and few-step generators in flow maps at the forefront of generative modeling. Such state-of-the-art models have limited access to explicit model densities despite evolving from classical generative modeling approaches that leveraged tractable likelihoods, e.g., Normalizing Flows and Autoregressive models. In this paper, we combine these threads of few-step generation and tractable sample likelihoods and introduce Normalizing Flow Maps (NFM), a class of exactly invertible pushforward maps that generalizes normalizing flows and flow maps. More precisely, NFMs learn the integrator of an underlying dynamical system, but crucially, they provide efficient exact likelihood evaluation by construction. To enable scalable training of NFMs, we derive two complementary axes of consistency inherent to NFMs that yield (self) distillation objectives: (1) pointwise matching, which unlocks ℓ2\ell_2-based flow-map style regression losses, and (2) distributional matching, which matches pushforward distributions to the marginals of a prescribed forward process. On conditional image generation on CIFAR-10 and ImageNet-64, NFMs achieve state-of-the-art sample quality among parameter-matched normalizing flows, while supporting likelihood-based RLHF. For equilibrium sampling in Boltzmann Generators, NFMs outperform prior state-of-the-art normalizing flows on both single-system and transferable peptide benchmarks.

Francesco M. Ruscio, Timothy James Hitge, Fred Zhangzhi Peng +6

AITHYRA · Imperial College London · TU Wien +5

Normalizing FlowsFlow Map LearningConditional Generative Modeling
AskPDF
Mar 20, 2026

Inverting Neural Networks: New Methods to Generate Neural Network Inputs from Prescribed Outputs

Neural network systems describe complex mappings that can be very difficult to understand. In this paper, we study the inverse problem of determining the input images that get mapped to specific neural network classes. Ultimately, we expect that these images contain recognizable features that are associated with their corresponding class classifications. We introduce two general methods for solving the inverse problem. In our forward pass method, we develop an inverse method based on a root-finding algorithm and the Jacobian with respect to the input image. In our backward pass method, we iteratively invert each layer, at the top. During the inversion process, we add random vectors sampled from the null-space of each linear layer. We demonstrate our new methods on both transformer architectures and sequential networks based on linear layers. Unlike previous methods, we show that our new methods are able to produce random-like input images that yield near perfect classification scores in all cases, revealing vulnerabilities in the underlying networks. Hence, we conclude that the proposed methods provide a more comprehensive coverage of the input image spaces that solve the inverse mapping problem.

Rebecca Pattichis, Sebastian Janampa, Constantinos S. Pattichis +1

Department of Electrical Engineering, University of California, Los Angeles, Los Angeles CA, USA · Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM, USA · Department of Computer Science, University of Cyprus, Cyprus

Feature VisualizationImage Inverse ProblemsFeedforward Neural Networks
AskPDF