VINA: Variational Invertible Neural Architectures

Shubhanshu Shekhar; Mohammad Javad Khojasteh; Ananya Acharya; Tony Tohme; Kamal Youcef-Toumi

cs.LGFeb 24, 2026

VINA: Variational Invertible Neural Architectures

Authors: Shubhanshu Shekhar, Mohammad Javad Khojasteh, Ananya Acharya, Tony Tohme, Kamal Youcef-Toumi

Organizations: EME Department, Rochester Institute of Technology · Department of Mechanical Engineering, Massachusetts Institute of Technology

Abstract

The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural networks (INNs) build on these principles to address supervised inverse problems, enabling direct modeling of both forward and inverse mappings. In this paper, we revisit these architectures from both theoretical and practical perspectives and address a key gap in the literature: the lack of theoretical guarantees on approximation quality under realistic assumptions, whether for posterior inference in INNs or for generative modeling with NFs. We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions, inspired by analogous formulations in related areas such as generative adversarial networks (GANs) and the Precision-Recall divergence for training normalizing flows. Within this framework, we derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs, under assumptions that are weaker and more practically realistic than those used in prior work. Building on these theoretical results, we conduct extensive case studies to distill general design principles and practical guidelines. We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.

Figures & tables

Panel of Figure 1 — Figure 1: INN can be represented by an invertible map $T$ that approximates the relation from the input $X\in\mathbb{R}^{d_{\mathbf{x}}}$ to the output $Y\in\mathbb{R}^{d_{\mathbf{y}}}$ and a latent variable $Z\in\mathbb{R}^{d_{\mathbf{x}}-d_{\mathbf{y}}}$ . The invertibility of $T$ means that for any $\mathbf{y}\in\mathbb{R}^{d_{\mathbf{y}}}$ , we also have an approximate posterior sampling distribution via $T^{-1}(\mathbf{y},Z)$ .

NF	NF	INN	INN
IK	IK	BNN	BNN
NLL	NLL	MMD	MMD
ERM	ERM	KL	KL
JS	JS	IPM	IPM
SL	SL	USL	USL
GAN	GAN	GI	GI

Table 1: Definitions of important acronyms.

Panel of Figure 2 — Figure 2: The four plots illustrate the results of the inverse kinematics case study with different prior regularizations. (a) shows samples from a model trained without any prior loss (i.e., with prior weight $\lambda^{\prime}=0$ ). (b) and (c) display samples from models trained with a prior weight of $\lambda^{\prime}=1$ . In (b), the model assumes a Gaussian prior over the joint configuration, resulting in a scaled $L_{2}$ loss between the ground truth and the generated samples. In contrast, (c) uses a uniform prior $\mathcal{U}(0,1)$ , which is inconsistent with the true distribution. (d) illustrates results from a model trained with a Gaussian prior, using $\lambda^{\prime}=100$ . The cross indicates the true end-effector position. All models were trained using the NLL loss formulation for the INN.

Panel of Figure 3 — Figure 3: Schematic illustration of how different training paradigms are used to solve inverse problems in the Gaussian setting. (a) Bayesian neural network for solving inverse problem. There is supervised cost (SC) between predicted and true x . (b) INN Trained without $\prescript{p}{}{L_{x}}$ . There is unsupervised cost (USC) between predicted and true x . (c) Shows an INN model trained with $\prescript{p}{}{L_{x}}$ , which serves as a reconstruction loss.

Panel of Figure 4 — Figure 4: Comparison of $f$ -divergence performance for coupling-based and iResNet architectures in the inverse kinematics case study. The x-axis shows training time and the y-axis shows $\ell_{2}$ -distance between true and generated end-effector positions. Ellipses are centered at the mean values, with horizontal and vertical diameters indicating the standard deviations in training time and $\ell_{2}$ -distance. The right and left columns correspond to forward loss ( 51 ) and backward losses ( 50 ), respectively. The top row corresponds to the coupling-based architecture whereas the figures on the bottom row generated based on iResNet layers.

Panel of Figure 7 — Figure 7: Top Row: Samples from NF models trained using the Sinkhorn divergence. The left image corresponds to a latent distribution $\mathcal{U}(0,1)$ used to simulate the data, the middle image uses $\mathcal{U}(3,4)$ , and the right image uses $\mathcal{U}(5,6)$ . Bottom Row: Samples from NF models trained using an $f$ -divergence approximation of the forward KL divergence. The latent distributions mirror those used in the Sinkhorn divergence experiments.

Panel of Table 2 — Table 2: This table shows the MMD between samples from the true distribution $\mathcal{U}(0,1)$ and those generated by the trained model.

Panel of Figure 8 — Figure 8: This figure illustrates the Wasserstein-1 distance for increasing values of $\alpha$ in the Pareto distribution, which corresponds to increasing number of finite moments. The y-axis is set to a logarithmic (base 10) scale. Configurations 1–3 specify INN with 2, 6, 8 coupling layers and subnetworks of hidden sizes 128, 64, 128, respectively.

Figure 9: The SWellEx-96 experiment environment. The acoustic source is towed by a research vessel and transmits signals at various frequencies. The acoustic sensor consists of a vertical line array (VLA). Based on the measurements collected at the VLA, the objective is to estimate posterior distributions over parameters of interest (e.g. water depth, sound speed at the water-sediment interface, source range and depth, etc.).

Panel of Figure 10 — Figure 10: Posterior comparisons between models trained with different loss functions for the GI dataset, using signal measurements from five hydrophones. The inferred parameter is water depth , with the red dotted line indicating the true value. The INN configuration for the upper row uses Configuration 1 whereas the configuration used in the bottom row is Configuration 2 . The y-axis shows un-normalized probability scores.

Panel of Figure 11 — Figure 11: Posterior comparisons between models trained with different loss functions for the GI dataset, using signal measurements from five hydrophones. The inferred parameter is sound speed , with the red dotted line indicating the true value. The INN configuration for the upper row uses Configuration 1 whereas the configuration used in the bottom row is Configuration 2 .

Panel of Table 3 — Table 3: Comparison of the $L_{2}$ loss for water depth using two different model configurations. The parameter being recovered is water depth.

Panel of Table 4 — Table 4: Comparison of the $L_{2}$ loss for sound speed using two different model configurations introduced. The parameter being recovered is sound speed.

Panel of Table 5 — Table 5: Comparison of training times for two model configurations. The configuration used for the left table achieves an inference time of 0.0060 seconds per 1000 samples, while the configuration used for the right table achieves 0.0036 seconds for the same number of samples.

Invertible Neural Networks

Invertible neural networks are models with learnable mappings that can be reversed, supporting density estimation, generation, inference, and control.

See all 14 papers

Generative Inverse Design with Abstention via Diagonal Flow Matching
Mar 16, 2026Miguel de Campos, Werner Krebs, Hanno GottschalkFlow Matching Uncertainty Quantification
Institute of Mathematics, TU Berlin · Siemens Energy
Normalizing Flow Maps
Oct 15, 2026Francesco M. Ruscio, Timothy James Hitge, Fred Zhangzhi Peng +6Normalizing Flows Flow Map Learning
AITHYRA · Imperial College London · TU Wien +5
Inverting Neural Networks: New Methods to Generate Neural Network Inputs from Prescribed Outputs
Mar 20, 2026Rebecca Pattichis, Sebastian Janampa, Constantinos S. Pattichis +1Feature Visualization Image Inverse Problems
Department of Electrical Engineering, University of California, Los Angeles, Los Angeles CA, USA · Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM, USA · Department of Computer Science, University of Cyprus, Cyprus

Mar 16, 2026J/K move · Enter open · S save

Generative Inverse Design with Abstention via Diagonal Flow Matching

Miguel de Campos, Werner Krebs, Hanno Gottschalk

Institute of Mathematics, TU Berlin · Siemens Energy

Flow Matching Uncertainty Quantification Flow-Based Generative Modeling Inverse Design

Open paper AskPDF arXiv

Inverse design aims to find design parameters

x

achieving target performance

y^*

. Generative approaches learn bidirectional mappings between designs and labels, enabling diverse solution sampling. However, standard conditional flow matching (CFM), when adapted to inverse problems by pairing labels with design parameters, exhibits strong sensitivity to their arbitrary ordering and scaling, leading to unstable training. We introduce Diagonal Flow Matching (Diag--CFM), which resolves this through a zero-anchoring strategy that pairs design coordinates with noise and labels with zero, making the learning problem provably invariant to coordinate permutations. This yields substantially lower round-trip error than CFM and invertible neural network baselines across design dimensions up to

P{=}784

, including order-of-magnitude gains on several benchmarks. We develop two architecture-intrinsic uncertainty metrics, Zero-Deviation and Self-Consistency, that enable three practical capabilities: selecting the best candidate among multiple generations, abstaining from unreliable predictions, and detecting out-of-distribution targets; consistently outperforming ensemble and general-purpose alternatives across all tasks. We validate on airfoil, gas turbine combustor, scalable analytical benchmarks, a photonics inverse-design task, and an image-statistics benchmark.

NF	NF	INN	INN
IK	IK	BNN	BNN
NLL	NLL	MMD	MMD
ERM	ERM	KL	KL
JS	JS	IPM	IPM
SL	SL	USL	USL
GAN	GAN	GI	GI

NF	NF	INN	INN
IK	IK	BNN	BNN
NLL	NLL	MMD	MMD
ERM	ERM	KL	KL
JS	JS	IPM	IPM
SL	SL	USL	USL
GAN	GAN	GI	GI

Abstract

Invertible Neural Networks

Explore similar work

Generative Inverse Design with Abstention via Diagonal Flow Matching

Normalizing Flow Maps

Inverting Neural Networks: New Methods to Generate Neural Network Inputs from Prescribed Outputs

NF	NF	INN	INN
IK	IK	BNN	BNN
NLL	NLL	MMD	MMD
ERM	ERM	KL	KL
JS	JS	IPM	IPM
SL	SL	USL	USL
GAN	GAN	GI	GI