Energy-Based Models

Ask about this topic

Overview

Energy-based models represent how compatible a data point or configuration is with a task: lower energy generally means greater plausibility, without requiring probabilities to be normalized explicitly. They are trained with methods such as contrastive divergence, noise-contrastive estimation, and score or energy matching, then used for inference and generation with Langevin MCMC, tempering, or diffusion and flow-based samplers. Work spans Boltzmann machines and Hopfield networks as well as image and molecule generation, structured prediction, robotics, and scientific modeling. A central challenge is making sampling and training efficient and reliable; in particular, equilibrium guarantees may not reflect finite-time outputs, and lower energy does not always mean better predictions or reasoning.

Latest papers

CardsList

91 papers9 papers in the last four weeks, up 125% on the four weeks before. 0.1% of all new papers.

Oct 19, 2023

Energy-Based Models For Speech Synthesis

Recently there has been a lot of interest in non-autoregressive (non-AR) models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike AR models, these models do not have autoregressive dependencies among outputs which makes inference efficient. This paper expands the range of available non-AR models with another member called energy-based models (EBMs). The paper describes how noise contrastive estimation, which relies on the comparison between positive and negative samples, can be used to train EBMs. It proposes a number of strategies for generating effective negative samples, including using high-performing AR models. It also describes how sampling from EBMs can be performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of Langevin MCMC enables to draw connections between EBMs and currently popular diffusion models. Experiments on LJSpeech dataset show that the proposed approach offers improvements over Tacotron 2.

Wanli Sun, Zehai Tu, Anton Ragni

Department of Computer Science, University of Sheffield, Sheffield, UK

TTS SynthesisEnergy-Based Models
AskPDF