Entropy Regularization

Ask about this topic

Overview

Entropy regularization adds an entropy-based term or constraint to an objective to control how concentrated or diverse a model’s distributions and representations become. Methods use Shannon, Rényi, or Tsallis entropy in settings such as reinforcement-learning exploration, test-time adaptation, language-model training, latent-variable learning, and entropically regularized optimal transport solved with Sinkhorn methods. Some approaches minimize prediction entropy for confident decisions, while others preserve or increase entropy to encourage exploration, prevent collapse, or maintain diverse outputs; adaptive and class-, token-, or perceptual-level controls aim to apply these effects selectively. The central challenge is balancing useful confidence or sparsity against bias, lost diversity, and unstable optimization, alongside developing guarantees for convergence and statistical performance.

Latest papers

CardsList

34 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. Information-Theoretic Foundations of Online Temperature Sampling

    Oct 10, 2026Jin Sima, Nikolaos Papagiannis, Ananth Grama +1Repetition in Language GenerationLanguage Model Decoding

    Purdue University

  2. Pareto-Optimal Entropy-Regularized Trajectory Optimization

    Oct 6, 2026Dimitrios S. Georgiou, Augustinos D. Saravanos, Evangelos A. TheodorouTrajectory OptimizationTrajectory Generation

    Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA. · Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA.

  3. The Arbitrary-Placement Problem in Entropy-Minimizing Selection, and a Residual-Entropy Formulation

    Oct 5, 2026Alyssa H. Shin, Claire H. ShinEntropy Regularization

    Division of Biology and Biological Engineering California Institute of Technology · Robert Frederick Smith School of Chemical and Biomolecular Engineering Cornell University

  4. Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations

    Sep 24, 2026Mihir Dhanakshirur, Adam Ousherovitch, Ambuj TewariEntropy RegularizationLLM Reasoning

    Department of Statistics, University of Michigan

  5. Uniform Statistical Convergence of Empirical Sinkhorn Potentials with Exponential and Polynomial Dependence on the Regularization Parameter

    Aug 29, 2026Denis BelomestnyOptimal TransportEntropy Regularization

    Department of Mathematics, University of Duisburg-Essen

  6. SinkSLOT: Sinkhorn via Sparse Lifted Optimal Transport

    Aug 28, 2026Ian Hsieh, Soumya Snigdha Kundu, Tom Vercauteren +1Optimal TransportEntropy Regularization

    Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière, 75013 Paris, France · King’s College London, UK

  7. On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method

    Aug 27, 2026Matthew King-Roskamp, Gabriel Rioux, Rustum Choksi +1Entropy Regularization

  8. Maximum Tsallis Entropy Distributions for Robust and Efficient Sparse Learning from Correlated Data

    Aug 18, 2026Kai Yang, Masoud Asgharian, Celia M. T. GreenwoodStatistical Learning TheoryRobust Estimation

  9. Density-Reweighted Entropic Optimal Transport: Decoupling Geometry from Sampling Density

    Aug 17, 2026Keyi Li, Yuval Kluger, Boris LandaOptimal TransportEntropy Regularization

    Program of Applied & Computational Mathematics, Yale University · Department of Pathology, Yale University · Department of Electrical and Computer Engineering, Yale University

  10. Soft-Constrained Optimization of Latent Space in Variational Autoencoders

    Jul 26, 2026Ye ShiVariational AutoencodersDisentangled Representation Learning

    Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, 47906 USA

  11. The SIGReg Objective as Variational Free Energy: A Theoretical Active-Inference Account of JEPA World Models

    Jul 15, 2026Fabio Arnez, Alexandra Gomez-VillaWorld Model LearningActive Inference

    Université Paris-Saclay, CEA, List F-91120, Palaiseau, France · Computer Vision Center Barcelona, Spain

  12. Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learning

    Jun 30, 2026Krishna Harsha Kovelakuntla Huthasana, Alireza Olama, Andreas LundellSparse RecoveryNon-IID Federated Learning

    Department of Engineering and Information Technology, Åbo Akademi University

  13. Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability

    Jun 29, 2026Srinivasa Rao P., Vangmayi P ReddyStatistical Physics of LearningNeural Network Generalization

    Curlvee Technolabs, India, Former Scientist, C-DAC · Former Scientist, C-DAC · Indian Institute of Technology Madras, India

  14. Sample complexity of unbalanced entropic OT

    Jun 23, 2026Francisco Andrade, Gabriel Peyré, Clarice PoonSample ComplexityEntropy Regularization

    INRIA (PreMeDICaL & HeKA) · INRIA · CNRS and ENS, PSL Universit´e +1

  15. Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging

    Jun 1, 2026Tim Nielen, Sameer Ambekar, Johannes Kiechle +2Distribution Shift RobustnessTest-Time Adaptation

    School of Computation, Information and Technology, Technical University of Munich, Germany · Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany · Munich Center for Machine Learning (MCML) +3

  16. Human-Centered Learning Mechanics: Observable Entropy Control under Protected-Risk Constraints

    May 21, 2026Kim Phuc TranRepresentation LearningNeural Network Training Dynamics

    Univ. Lille, ENSAIT, ULR 2461 – GEMTEX – Génie et Matériaux Textiles, F-59000 Lille, France · International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

  17. Behavior-Consistent Deep Reinforcement Learning

    May 20, 2026Marcel Hussing, Liv G. d'Aliberti, Claas Voelcker +2Reinforcement LearningPolicy Optimization

    University of Pennsylvania · Princeton University · University of Texas at Austin

  18. Normative Networks for Source Separation via Local Plasticity and Dendritic Computation

    May 19, 2026Bariscan Bozkurt, Efe Ali Gorguner, Francesco Innocenti +1Local LearningEntropy Regularization

    Gatsby Computational Neuroscience Unit, University College London, UK · Brain Network Dynamics Unit, University of Oxford, UK · Department of Computer Science, University of Oxford, UK +1

  19. DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularization

    May 18, 2026Chengpeng Hu, Yingqian Zhang, Hendrik BaierReinforcement LearningProgram Synthesis

    Eindhoven University of Technology, Eindhoven, the Netherlands · Centrum Wiskunde & Informatica, Amsterdam, the Netherlands

  20. Graph Transductive Sharpening: Leveraging Unlabeled Predictions in Node Classification

    May 18, 2026Brown Zaz, Mar Gonzàlez I Català, Ferran Hernandez Caralt +2Graph Neural NetworksGraph-Based Semi-Supervised Learning

    University of Cambridge

  21. On Gaussian approximation for entropy-regularized Q-learning with function approximation

    May 17, 2026Artemy Rubtsov, Rahul Singh, Eric Moulines +2Reinforcement LearningStochastic Approximation

    HSE University, Moscow, Russia · Mohamed Bin Zayed University of AI, UAE · EPITA, France +1

  22. Entropic Auto-Encoding via Implicit Free-Energy Minimization

    May 15, 2026Hazhir Aliahmadi, Irina Babayan, Greg van AndersVariational AutoencodersPosterior Collapse

    Department of Physics, Engineering Physics & Astronomy, Queen’s University, Kingston, ON, Canada

  23. Sharp Spectral Thresholds for Logit Fixed Points

    May 15, 2026Tongxi WangDynamical SystemsLyapunov Stability

    Southeast University Nanjing, China

  24. When Policy Entropy Constraint Fails: Preserving Diversity in Flow-based RLHF via Perceptual Entropy

    May 12, 2026Xiaofeng Tan, Jun Liu, Bin-Bin Gao +5Flow MatchingGenerative Modeling

    Southeast University · Tencent Youtu Lab · Southern University of Science and Technology

  25. Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization

    May 12, 2026Huimin Xu, Shuai Zhao, Xiaobao Wu +1Reinforcement LearningRL for Language Model Reasoning

    Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · 3VinUniversity, Vietnam

  26. Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models

    May 5, 2026Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin +1Test-Time AdaptationEntropy Regularization

    Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan

  27. MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization

    May 3, 2026Yavuz Yarici, Ghassan AlRegibMultimodal Domain GeneralizationDomain Generalization

    1OLIVES at the Center for Signal and Information Processing CSIP, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA.

  28. Asymmetric Advantage Modulation Calibrates Entropy Dynamics in RLVR

    Apr 6, 2026Hengrui Gu, Xiaotian Han, Yujing Bian +2RL for Language Model ReasoningReinforcement Learning with Verifiable Rewards

    North Carolina State University · Case Western Reserve University · Oak Ridge National Laboratory

  29. On the Efficiency of Sinkhorn-Knopp for Entropically Regularized Optimal Transport

    Apr 4, 2026Kun HeOptimal TransportEntropy Regularization

    Renmin University of China

  30. A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning

    Apr 2, 2026Ming Lei, Christophe BaehrEntropy RegularizationRL for Language Model Reasoning

    Shanghai Jiao Tong University, Shanghai · M´et´eo France/CNRS, Universit´e de Toulouse, CNRM UMR 3589, Toulouse