Normalizing Flows

Ask about this topic

Overview

Normalizing flows learn reversible transformations from a simple distribution, such as a Gaussian, into a complex one, allowing both samples and exact probability densities to be computed. Common approaches include coupling and autoregressive layers, spline transformations, continuous flows, and structured or conditional flows for tasks such as image generation, Bayesian inference, anomaly detection, and control. Current work also combines flows with compact latent representations, flow matching, and physics-informed models, extending their use in scientific simulation and uncertainty estimation. A central tension is balancing expressive, multimodal models against efficient and stable computation, while ensuring that good likelihoods translate into reliable mode coverage and calibrated uncertainty.

Latest papers

CardsList

73 papers7 papers in the last four weeks, down 12% on the four weeks before. 0.0% of all new papers.

Mar 9, 2026

The Coupling Within: Flow Matching via Distilled Normalizing Flows

Flow models have rapidly become the go-to method for training and deploying large-scale generators, owing their success to inference-time flexibility via adjustable integration steps. A crucial ingredient in flow training is the choice of coupling measure for sampling noise/data pairs that define the flow matching (FM) regression loss. While FM training defaults usually to independent coupling, recent works show that adaptive couplings informed by noise/data distributions (e.g., via optimal transport, OT) improve both model training and inference. We radicalize this insight by shifting the paradigm: rather than computing adaptive couplings directly, we use distilled couplings from a different, pretrained model capable of placing noise and data spaces in bijection -- a property intrinsic to normalizing flows (NF) through their maximum likelihood and invertibility requirements. Leveraging recent advances in NF image generation via auto-regressive (AR) blocks, we propose Normalized Flow Matching (NFM), a new method that distills the quasi-deterministic coupling of pretrained NF models to train student flow models. These students achieve the best of both worlds: significantly outperforming flow models trained with independent or even OT couplings, while also improving on the teacher AR-NF model.

David Berthelot, Tianrong Chen, Jiatao Gu +6

Flow-Based Generative ModelingFlow MatchingNormalizing Flows
AskPDF
Mar 8, 2026

Image Generation Models: A Technical History

Image generation has advanced rapidly over the past decade, yet the literature seems fragmented across different models and application domains. This paper aims to offer a comprehensive survey of breakthrough image generation models, including variational autoencoders (VAEs), generative adversarial networks (GANs), normalizing flows, autoregressive and transformer-based generators, and diffusion-based methods. We provide a detailed technical walkthrough of each model type, including their underlying objectives, architectural building blocks, and algorithmic training steps. For each model type, we present the optimization techniques as well as common failure modes and limitations. We also go over recent developments in video generation and present the research works that made it possible to go from still frames to high quality videos. Lastly, we cover the growing importance of robustness and responsible deployment of these models, including deepfake risks, detection, artifacts, and watermarking.

Rouzbeh Shirvani

Image GenerationDiffusion-Based Image GenerationGenerative Adversarial NetworksVariational AutoencodersNormalizing FlowsAutoregressive Image Generation
AskPDF
Mar 3, 2026

Variance reduction in lattice QCD observables via normalizing flows

Normalizing flows can be used to construct unbiased, reduced-variance estimators for lattice field theory observables that are defined by a derivative with respect to action parameters. This work implements the approach for observables involving gluonic operator insertions in the SU(3) Yang-Mills theory and two-flavor Quantum Chromodynamics (QCD) in four space-time dimensions. Variance reduction by factors of 1010-6060 is achieved in glueball correlation functions and in gluonic matrix elements related to hadron structure, with demonstrated computational advantages. The observed variance reduction is found to be approximately independent of the lattice volume, so that volume transfer can be utilized to minimize training costs.

Ryan Abbott, Denis Boyda, Yang Fu +5

Physics Department, Columbia University, New York, NY 10027, USA · Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions +3

Lattice Field TheoryVariance ReductionHigh-Energy PhysicsNormalizing Flows
AskPDF
Feb 25, 2026

IHF-Harmony: Multi-Modality Magnetic Resonance Images Harmonization using Invertible Hierarchy Flow Model

Retrospective MRI harmonization is limited by poor scalability across modalities and reliance on traveling subject datasets. To address these challenges, we introduce IHF-Harmony, a unified invertible hierarchy flow framework for multi-modality harmonization using unpaired data. By decomposing the translation process into reversible feature transformations, IHF-Harmony guarantees bijective mapping and lossless reconstruction to prevent anatomical distortion. Specifically, an invertible hierarchy flow (IHF) performs hierarchical subtractive coupling to progressively remove artefact-related features, while an artefact-aware normalization (AAN) employs anatomy-fixed feature modulation to accurately transfer target characteristics. Combined with anatomy and artefact consistency loss objectives, IHF-Harmony achieves high-fidelity harmonization that retains source anatomy. Experiments across multiple MRI modalities demonstrate that IHF-Harmony outperforms existing methods in both anatomical fidelity and downstream task performance, facilitating robust harmonization for large-scale multi-site imaging studies. Code is available at https://github.com/Idea89560041/IHF-Harmony.

Pengli Zhu, Yitao Zhu, Haowen Pang +1

Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1

Medical Image HarmonizationMagnetic Resonance ImagingUnpaired Image-to-Image TranslationNormalizing Flows
AskPDF
Feb 24, 2026

VINA: Variational Invertible Neural Architectures

The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural networks (INNs) build on these principles to address supervised inverse problems, enabling direct modeling of both forward and inverse mappings. In this paper, we revisit these architectures from both theoretical and practical perspectives and address a key gap in the literature: the lack of theoretical guarantees on approximation quality under realistic assumptions, whether for posterior inference in INNs or for generative modeling with NFs. We introduce a unified framework for INNs and NFs based on variational unsupervised loss functions, inspired by analogous formulations in related areas such as generative adversarial networks (GANs) and the Precision-Recall divergence for training normalizing flows. Within this framework, we derive theoretical performance guarantees, quantifying posterior accuracy for INNs and distributional accuracy for NFs, under assumptions that are weaker and more practically realistic than those used in prior work. Building on these theoretical results, we conduct extensive case studies to distill general design principles and practical guidelines. We conclude by demonstrating the effectiveness of our approach on a realistic ocean-acoustic inversion problem.

Shubhanshu Shekhar, Mohammad Javad Khojasteh, Ananya Acharya +2

EME Department, Rochester Institute of Technology · Department of Mechanical Engineering, Massachusetts Institute of Technology

Invertible Neural NetworksNormalizing FlowsBayesian Inverse Problems
AskPDF
Feb 10, 2026

Why Does Counterintuitive Likelihood Behavior Rarely Emerge in Tabular Anomaly Detection?

Deep generative models with tractable likelihoods, exemplified by normalizing flows, offer an effective basis for anomaly detection through likelihood-based scoring. We demonstrate that, unlike in the image domain where deep generative models frequently assign higher likelihoods to anomalous data, such counterintuitive behavior occurs rarely in tabular settings. We first introduce a domain-agnostic formulation that enables consistent detection and evaluation of the counterintuitive phenomenon, addressing the lack of a precise definition in prior work. Through extensive experiments on 47 tabular datasets benchmarked against 13 baseline models and on 10 CV/NLP embedding datasets from ADBench, we show that the phenomenon is consistently rare in general tabular data and that normalizing flow likelihood achieves competitive anomaly detection performance. We further provide theoretical and empirical analyses attributing this disparity to differences in dimensionality and feature correlation between tabular and image data. Therefore, our results indicate that likelihood-based detection with normalizing flows is a practical approach for tabular anomaly detection.

Donghwan Kim, Junghun Phee, Hyunsoo Yoon

Department of Industrial Engineering, University of Yonsei, Republic of Korea, Seoul · H. Milton Stewart School of Industrial and Systems Engineering (ISyE), Georgia Institute of Technology, USA, GA

Anomaly DetectionTabular Anomaly DetectionNormalizing Flows
AskPDF
Feb 10, 2026

Mitigating the Likelihood Paradox in Flow-based OOD Detection: An Entropic Perspective

Deep generative models that can tractably compute input likelihoods, including normalizing flows, often assign unexpectedly high likelihoods to out-of-distribution (OOD) inputs. We mitigate this paradox from an entropic perspective with Semantic Proportional Entropy Manipulation (SPEM), a training-free procedure that applies stronger perturbations to inputs less similar to an in-distribution (ID) memory bank. We derive theoretical lower bounds showing that this entropy gap increases the expected log-likelihood difference in favor of the ID, and experiments on standard benchmarks show that this manipulation realigns the likelihood ordering with human intuition. Strikingly, this realignment persists even when the flow is given only visually meaningless noise instead of the original input, and we analyze this phenomenon theoretically to characterize the conditions under which it occurs.

Donghwan Kim, Hyunsoo Yoon

Department of Industrial Engineering University of Yonsei

OOD DetectionNormalizing Flows
AskPDF
Feb 6, 2026

From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows

The unsupervised discovery of features that are both semantically meaningful and stable across runs remains a central challenge in representation learning. We introduce entropy-ordered flows (EOFlows), a normalizing flow (NF) framework that augments standard maximum likelihood training with an orthogonality regularizer on the decoder Jacobian. The regularizer is rooted in Independent Mechanism Analysis and encourages geometric disentanglement, and a stochastic estimator makes it tractable at image scale (CelebA at D=2352D=2352 and 1228812288). Learned features form near-orthogonal curvilinear coordinates and can be ordered by their explained (manifold) entropy\textit{explained (manifold) entropy} after training, analogous to the ranking by explained variance in PCA, which turns EOFlows into a non-linear generalization of PCA. EOFlows identify an order of magnitude more stable features than existing methods, and these features emerge in distinguishable categories (global, local, and generic) and support tentative semantic interpretations. The local features have strikingly sparse support in pixel space, although our method never enforces this. Retaining only the most important, i.e. highest entropy, features turns the bijective flow into an autoencoder with adjustable bottleneck, rivaling the rate-distortion performance of dedicated autoencoders.

Daniel Galperin, Ullrich Köthe

Heidelberg University

Unsupervised LearningDisentangled Representation LearningNormalizing FlowsOrthogonal Representation LearningFlow-Based Generative Modeling
AskPDF
Jan 11, 2026

Enhancing Low-resolution Image Representation Through Normalizing Flows

Low-resolution image representation can be regarded as a special form of sparse representation that retains only low-frequency information while discarding high-frequency components. This property reduces storage and transmission costs and benefits various image processing tasks. However, a key challenge is to preserve essential visual content while maintaining the ability to accurately reconstruct the original images. This work proposes LR2Flow, a nonlinear framework that learns low-resolution image representations by integrating wavelet tight frame blocks with normalizing flows. We conduct a reconstruction error analysis of the proposed network, which demonstrates the necessity of designing invertible neural networks in the wavelet tight frame domain. Experimental results on various tasks, including image rescaling, compression, and denoising, demonstrate the effectiveness of the learned representations and the robustness of the proposed framework. Code is available at https://github.com/Evanescentlove/LR2Flow-main.

Chenglong Bao, Tongyao Pang, Zuowei Shen +2

Yau Mathematical Sciences Center, Tsinghua University, Beijing, China, and Beijing Institute of Mathematical Sciences and Applications, Beijing, China · Yau Mathematical Sciences Center, Tsinghua University, Beijing, China · Department of Mathematics, National University of Singapore, Singapore +2

Learned Image CompressionNormalizing FlowsVisual Representation Learning
AskPDF
Oct 29, 2025

CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices

Normalizing flows are deep generative models that enable efficient likelihood estimation and sampling through invertible transformations. A key challenge is to design linear layers that enhance expressiveness while maintaining efficient computation of the Jacobian determinant and inverse. We introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition reduces parameter complexity from O(n2)\mathcal{O}(n^2) to O(mn)\mathcal{O}(mn) using mm diagonal matrices and m−1m-1 circulant matrices while still approximating general linear transformations. By leveraging the Fast Fourier Transform, our approach reduces the time complexity of matrix inversion from O(n3)\mathcal{O}(n^3) to O(mnlog⁡n)\mathcal{O}(mn\log n) and that of computing the log-determinant from O(n3)\mathcal{O}(n^3) to O(mn)\mathcal{O}(mn), where nn is the input dimension. We build upon this layer to develop Circulant-Diagonal Flow (CDFlow), which achieves strong density estimation on natural image datasets and effectively models data with inherent periodic structure. Furthermore, CDFlow significantly accelerates key operations in normalizing flows, providing practical benefits for scalable generative modeling.

Xuchen Feng, Siyu Liao

School of Integrated Circuits Sun Yat-sen University

Normalizing FlowsDensity EstimationMatrix Factorization
AskPDF
Oct 11, 2025

Neural Variational Cut Posteriors without Upstream Data

In many applications, one must propagate parameter uncertainty from an earlier (upstream) analysis, available as samples, to subsequent (downstream) analyses without feedback. This problem is called cutting feedback or cut-Bayes, and the cut-posterior, the optimal posterior preserving information-flow constraints, is well characterized. However, sampling from it (e.g., via nested MCMC) is computationally intensive, while existing variational inference methods for cut-Bayes require access to upstream data and model, often unavailable. We propose a modular and provably accurate cut-Bayes approach requiring no access to upstream data or model. We leverage the characterization of the cut-posterior as the minimizer of the expected downstream conditional Kullback-Leibler divergence over the upstream posterior, replacing the expectation with the sample average over upstream draws. Our method, NeVI-Cut (neural variational inference for cut-Bayes), employs conditional normalizing flows as the variational family for downstream parameters. We provide fixed-data convergence rates of NeVI-Cut in terms of the richness of neural architecture and complexity of the cut-posterior. We establish, to our knowledge, first results on uniform Kullback-Leibler approximation rates of conditional distributions by common flow classes, yielding widely applicable fixed-data error rates for variational flows. A stochastic algorithm implements NeVI-Cut efficiently, and we demonstrate its speed and accuracy on multiple applications.

Jiafang Song, Sandipan Pramanik, Abhirup Datta

Department of Biostatistics, Johns Hopkins University, USA

Bayesian InferenceVariational InferenceNormalizing Flows
AskPDF
May 29, 2025

Normalizing Flows are Capable Models for Continuous Control

Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models. To this end, RL researchers often choose to pay the price of accommodating these models into their algorithms -- diffusion models are expressive, but are computationally intensive due to their reliance on solving differential equations, while autoregressive transformer models are scalable but typically require learning discrete representations. Normalizing flows (NFs), by contrast, seem to provide an appealing alternative, as they enable likelihoods and sampling without solving differential equations or autoregressive architectures. However, their potential in RL has received limited attention, partly due to the prevailing belief that normalizing flows lack sufficient expressivity. We show that this is not the case. Building on recent work in NFs, we propose a single NF architecture which integrates seamlessly into RL algorithms, serving as a policy, Q-function, and occupancy measure. Our approach leads to much simpler algorithms, and achieves higher performance in imitation learning, offline, goal conditioned RL and unsupervised RL.

Raj Ghugare, Benjamin Eysenbach

Department of Computer Science Princeton University

Normalizing FlowsReinforcement LearningUnsupervised RLContinuous ControlFlow-Based Generative ModelingFlow-Based Policy Learning
AskPDF
May 6, 2025

Physics-Informed Sylvester Normalizing Flows for Bayesian Inference in Magnetic Resonance Spectroscopy

Magnetic resonance spectroscopy (MRS) is a non-invasive technique to measure the metabolic composition of tissues, offering valuable insights into neurological disorders, tumor detection, and other metabolic dysfunctions. However, accurate metabolite quantification is hindered by challenges such as spectral overlap, low signal-to-noise ratio, and various artifacts. Traditional methods like linear-combination modeling are susceptible to ambiguities and commonly only provide a theoretical lower bound on estimation accuracy in the form of the Cramér-Rao bound. This work introduces a Bayesian inference framework using Sylvester normalizing flows (SNFs) to approximate posterior distributions over metabolite concentrations, enhancing quantification reliability. A physics-based decoder incorporates prior knowledge of MRS signal formation, ensuring realistic distribution representations. We validate the method on simulated 7T proton MRS data, demonstrating accurate metabolite quantification, well-calibrated uncertainties, and insights into parameter correlations and multi-modal distributions.

Julian P. Merkofer, Dennis M. J. van de Sande, Alex A. Bhogal +1

Eindhoven University of Technology, Eindhoven, The Netherlands · University Medical Center Utrecht, Utrecht, The Netherlands

Normalizing FlowsBayesian InferencePhysics-Informed MLMagnetic Resonance Spectroscopy
AskPDF