cs.AINov 10, 2026

An Energy-Based Perspective on Machine Learning

Authors: Bo Dai, Yilun Du

Organizations: Harvard University

Abstract

Energy-based models (EBMs) define a distribution through a scalar energy, pθ(x)∝exp⁡(−Eθ(x))p_θ(x)\propto\exp(-E_θ(x)), and are usually presented as one model family among many. This tutorial has a broader goal: beyond introducing EBMs, it uses them as a single lens that connects the different aspects of machine learning. Any EBM is specified by three choices, the energy, which determines what the model represents; the inference procedure, which samples from or optimizes the energy; and the learning method, which fits the energy to data. We first develop these components, including sampling, variational inference, and learning by maximum likelihood, noise contrastive estimation, pseudolikelihood, score matching, and adversarial training, and show that the learning methods share one contrastive structure. We then show that discriminative models, generative models such as GANs, normalizing flows, diffusion and autoregressive models, representation learning methods such as contrastive learning, word2vec, BERT, denoising auto-encoders, and diffusion representations, reinforcement learning with EBM policies and world models, inverse reinforcement learning, and planning, as well as inference-time computation with verifiers, are all EBMs that differ in these three choices, mostly in how they handle the partition function and sampling. This view explains existing methods, organizes them into a common framework, and suggests new algorithms by replacing one component while keeping the others, with composition of energies as a principled way to combine learned models at inference time.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learning Multimodal Energy-Based Model with Multimodal Variational Auto-Encoder via MCMC Revision

    May 1, 2026Jiali Cui, Zhiqiang Lao, Heather YuVariational AutoencodersMarkov Chain Monte Carlo

  2. Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

    May 9, 2026Thomas Tulinski, Simona Cocco, Rémi Monasson +1Statistical Physics of LearningPhase Transitions

  3. Score-Based Training for Energy-Based TTS Models

    May 19, 2025Wanli Sun, Anton RagniEnergy-Based ModelsScore Matching