Energy-Based Models

Follow this topic Ask about this topic

Energy-based models represent how compatible a data point or configuration is with a task: lower energy generally means greater plausibility, without requiring probabilities to be normalized explicitly. They are trained with methods such as contrastive divergence, noise-contrastive estimation, and score or energy matching, then used for inference and generation with Langevin MCMC, tempering, or diffusion and flow-based samplers. Work spans Boltzmann machines and Hopfield networks as well as image and molecule generation, structured prediction, robotics, and scientific modeling. A central challenge is making sampling and training efficient and reliable; in particular, equilibrium guarantees may not reflect finite-time outputs, and lower energy does not always mean better predictions or reasoning.

Latest papers

91 papers9 papers in the last four weeks, up 125% on the four weeks before. 0.1% of all new papers.

Learning from Noise: Effective-Rank Collapse and Out-of-Distribution Rejection in Restricted Boltzmann Machines
Jul 11, 2026Oshada Rathnayake, Nikhil ShuklaEnergy-Based Models Classification
University of Virginia, Charlottesville, VA, 22903, USA
Energy-guided Recursive Model
Jul 11, 2026Yifei Zhao, Ying TangLanguage Modeling Recursive Language Models
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physics, University of Electronic Science and Technology of China, Chengdu 611731, China
Answer Set Programming Energised! End-to-End Neurosymbolic Reasoning and Learning with ASP and Energy Based Models
Jul 9, 2026Jakob Suchan, Julius Monsen, Salim Baloch +1Energy-Based Models Answer Set Programming
Constructor University Bremen, Bremen, Germany · CoDesign Lab > Cognitive Vision · Örebro University, Örebro, Sweden
From Graphs to Gradients: Physics-Inspired Structural Attribution for Cyber-Physical IoT Systems and Beyond
Jul 6, 2026Spyridon Evangelatos, Christos Diou, Georgios Th. Papadopoulos +2Cyber-Physical Systems Feature Attribution
Research & Innovation Development Department, Netcompany S.A., Luxembourg · Department of Informatics and Telematics, Harokopio University of Athens, Greece · Department of Electrical and Computer Engineering, Hellenic Mediterranean University, Greece +1
Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space
Jul 2, 2026Long Minh Bui, Tuan Anh Le Van, Tung Phi Duc +3Product of Experts Energy-Based Models
Washington State University · Hanoi University of Science and Technology
Data-Driven Energy-Based Learning via Gibbs Measures on Hierarchical Structures
Jun 29, 2026L. U. Abdullaev, F. Herrera, U. A. Rozikov +1Statistical Physics of Learning Phase Transitions
National University of Singapore, Singapore · Department of Computer Science and Artificial Intelligence, University of Granada, E-18071 Granada, Spain · V.I. Romanovskiy Institute of Mathematics, Uzbekistan Academy of Sciences, 9, Universitet str., 100174, Tashkent, Uzbekistan +3
Retrieval-Warmed Energy-Based Reasoning: A Five-Arm Ablation Methodology for Diffusion-as-Inference on Structured Reasoning Tasks
Jun 25, 2026Libo Sun, Po-Wei Harn, Zewei Zhang +2Diffusion Sampling Energy-Based Models
Department of Computer Science and Software Engineering, Auburn University, Auburn, AL 36830, USA · Department of Information Management, National Central University, Taoyuan 320317, Taiwan
An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series
Jun 23, 2026Shubham Kumar Singh, Peilei Fan, Suraj A. Yadav +2Agricultural Remote Sensing Transformer
Department of Urban and Environmental Policy and Program, Tufts University, 503 Boston Ave., Medford, MA 02155 · Electrical and Computer Engineering Department, Mississippi State University, Mississippi, USA, 39762 · Department of Physics, Indian Institute of Technology (BHU), Varanasi, India 221005 +1
Energy-Based Transformers as Predictors of Reading Difficulty
Jun 22, 2026Jakub Dotlacil, Ece TakmazEnergy-Based Models Computational Psycholinguistics
Utrecht University
AURA: Active-Response Attribution under Treatment Ambiguity in Bacterial Cytological Profiling
Jun 15, 2026Kartik Jhawar, Mrunmayee Deshpande, Wilfried Moreira +2Energy-Based Models Microscopy Image Analysis
Institute for Digital Molecular Analytics and Science, Nanyang Technological University, Singapore 636921 · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 · Institute for Functional Intelligent Materials, National University of Singapore, Singapore 117544
Controlled Dynamics Attractor Transformer
Jun 13, 2026Cheng Zhang, Minnan Luo, Zesheng Yang +3Energy-Based Models Continuous Attractor Neural Networks
School of Computer Science and Technology, MOEK-LINNS Lab, Xi’an Jiaotong University, Xi’an, Shaanxi, China · Department of Computer Science and Technology, Tsinghua University, Beijing, China
ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs
Jun 9, 2026Xianlin Zeng, Fan Xia, Xiangyu ChenGraph Representation Learning Cross-Modal Alignment
Postdoctoral Research Station at China RongTong Academy of Sciences Group Corporation Limited, Beijing, China · Duke University, Durham, NC, USA.
Unified Energy for Invariant and Independent Decoding in Diffusion Language Models
Jun 8, 2026Yuchen Yan, Minkai Xu, Zaiquan Yang +1Energy-Based Models Diffusion Language Model Decoding
Department of Computer Science National University of Singapore · Department of Computer Science Stanford University · Department of Computer Science City University of Hong Kong
Optimizing Energy-based Neural Network Training with Coherent Ising Machine
Jun 8, 2026Chen-Rui Fan, Bo Lu, Zhi-Hong Zhang +3Equilibrium Propagation Neural Network Optimization
School of Artiﬁcial Intelligence, Beijing Normal University, Beijing, 100875, China. · Laboratory for Advanced Computing and Intelligence Engineering, Information Engineering University, Zhengzhou, 450001, China. · China Mobile (Suzhou) Software Technology Company Limited, Suzhou, 215163, China.
Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning
Jun 8, 2026Chen-Rui Fan, Bo Lu, Xing-Yu Wu +2Equilibrium Propagation Hopfield Networks
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China · Laboratory for Advanced Computing and Intelligence Engineering, Information Engineering University, Zhengzhou 450001, China · School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
Training a Predictive Coding Network on ImageNet using Equilibrium Propagation
Jun 2, 2026Tugdual Kerjan, Rasmus Høier, Benjamin ScellierEquilibrium Propagation Predictive Coding
Rain AI
Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences
May 26, 2026Hanlin Yu, RuiKang OuYang, Partha Kaushik +3Contrastive Learning Stochastic Interpolants
University of Helsinki · University of Cambridge · Carnegie Mellon University +1
Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot
May 22, 2026Jorge Chang Ortega, Bastien Le Lan, Thomas Serre +1Visual Representation Learning Energy-Based Models
ANITI · Brown University, ANITI · CNRS, ANITI
Convex Compositional Reasoning Models
May 22, 2026Meir Roketlishvili, Semyon Semenov, Maksim Bobrin +7Compositional Reasoning Energy-Based Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Applied AI Institute, Computational Imaging Lab · AXXX
Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model
May 20, 2026Jack KendallEquilibrium Propagation Dynamical Systems
Zyphra · San Francisco, CA
Generating Physically Consistent Molecules with Energy-Based Models
May 18, 2026Christoph Griesbacher, Lea Bogensperger, Andreas Habring +1Energy-Based Models 3D Molecular Generation
Graz University of Technology · University of Zurich
Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning
May 15, 2026Shireen Kudukkil Manchingal, Abhey Kalia, Fernanda Gonçalves +1LLM Answer Verification Energy-Based Models
Oxford Dynamics · Harwell Science and Innovation Campus, United Kingdom
Learning Normalized Energy Models for Linear Inverse Problems
May 15, 2026Nicolas Zilberstein, Santiago Segarra, Eero Simoncelli +1Bayesian Inverse Problems Energy-Based Models
Rice University, Houston, TX, USA · Flatiron Institute, New York, NY, USA · New York University, New York, NY, USA
Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks
May 11, 2026Cheol Young Park, Shou MatsumotoBayesian Networks Energy-Based Models
ATOS Co., Ltd., 273, Digital-ro, Guro-gu, Seoul, 08381, Republic of Korea · C5I Center, George Mason University, Fairfax, VA 22030, USA
Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation
May 9, 2026Zhijun Zeng, Yixuan Jiang, Pipi Hu +1Energy-Based Models Density Estimation
Qiuzhen College Tsinghua University Beijing 100084, China · Yanqi Lake Beijing Institute of Mathematical Sciences and Applications Beijing 101408, China · Yau Mathematical Sciences Center Tsinghua University, Beijing 100084, China and Yanqi Lake Beijing Institute of Mathematical Sciences and Applications Beijing 101408, China
Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models
May 9, 2026Thomas Tulinski, Simona Cocco, Rémi Monasson +1Statistical Physics of Learning Phase Transitions
Laboratoire de Physique de l’École Normale Supérieure, PSL, CNRS UMR8023, Sorbonne Université, 24 rue Lhomond, 75005 Paris, France · Institut de Physique Théorique, Université Paris-Saclay,2026 CNRS UMR3681, CEA, Gif-sur-Yvette, France
Energy-based models for diagnostic reconstruction and analysis in a laboratory plasma device
May 9, 2026Phil Travis, Troy CarterEnergy-Based Models Conditional Generative Modeling
Ergodic LLC, USA · Oak Ridge National Laboratory, Oak Ridge, Tennessee 37830, USA
Distributional simplicity bias and effective convexity in Energy Based Models
May 8, 2026Aurélien Decelle, Alfonso de Jesús Navas Gómez, Beatriz SeoaneEnergy-Based Models Gradient Descent Dynamics
Escuela Técnica Superior de Ingenieros Industriales Universidad Politécnica de Madrid Calle de José Gutiérrez Abascal 2, Madrid 28006, Spain · Departamento de Física Teórica Universidad Complutense de Madrid Plaza de las Ciencias 1, Madrid 28040 · Departamento de Física Teórica & IPARCOS Universidad Complutense de Madrid Plaza de las Ciencias 1, Madrid 28040
Revisiting Transformer Layer Parameterization Through Causal Energy Minimization
May 8, 2026Jin Xu, Camille Couturier, Victor Rühle +2Transformer Energy-Based Models
Microsoft · Microsoft Research Cambridge
Three-in-One World Model: Energy-Based Consistency, Prediction, and Counterfactual Inference for Marketing Intervention
May 8, 2026Junichiro NiimiCausal Effect Estimation World Model Learning
1Meijo University

Jul 11, 2026J/K move · Enter open · S save

Learning from Noise: Effective-Rank Collapse and Out-of-Distribution Rejection in Restricted Boltzmann Machines

Oshada Rathnayake, Nikhil Shukla

University of Virginia, Charlottesville, VA, 22903, USA

Energy-Based Models Classification OOD Detection Restricted Boltzmann Machines

Open paper AskPDF arXiv

Restricted Boltzmann machines (RBMs) represent data by shaping an energy landscape over visible and hidden configurations, but their discriminative use is fragile under out-of-distribution (OOD) inputs: samples outside the training distribution can be absorbed into one of the learned class basins rather than rejected. Here, we analyze this failure mode through the spectrum of the induced visible--visible interaction

J=WW^{T}

, where

W

is the visible--hidden weight matrix. Relative to a Marchenko--Pastur random-matrix reference, conventional training spreads spectral weight into many weak, bulk-compatible directions, increasing the effective rank of

J

. When auxiliary random binary images are assigned to a rejection label during training, the learned interaction undergoes effective-rank collapse: weak bulk-like modes are depleted, spectral weight concentrates into fewer dominant eigendirections, and the effective rank of

J

approaches that of the empirical data covariance matrix. The resulting RBM rejects structured OOD image datasets while preserving MNIST classification accuracy, showing that random auxiliary exposure can reshape both the interaction spectrum and the free-energy landscape of an energy-based classifier.