Entropy

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 96

All topics

Entropy Equivalence Testing
May 22, 2026Clément L. Canonne, Yash Pote, Jonathan Scarlett +1Two-Sample Testing Optimal Sample Complexity
University of Sydney · National University of Singapore
When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
May 20, 2026Wei Xia, Haoqing Wang, Zhi-Hong Deng +1LLM Reasoning Strategies LLM Inference Optimization
Samsung Research, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes
May 19, 2026Mohammed Alshaalan, Miguel R. D. RodriguesAdversarial Prompts Token-Level Entropy
Department of Electronic and Electrical Engineering, University College London, London, United Kingdom.
Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers
May 15, 2026Bruno Trentini, Dejan Stancevic, Michael M. Bronstein +2Schrödinger Bridges Accelerated Sampling
NVIDIA Corporation, Santa Clara, CA, USA · University of Oxford, Dept. of Computer Science, Oxford, UK · Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, NL +1
Conditional Entropy of Heat Diffusion on Temporal Networks
May 15, 2026Samuel Koovely, Alexandre BovetChange-Point Detection Temporal Dynamics
Department of Mathematical Modeling and Machine Learning University of Zurich Zurich, Switzerland
Tight Sample Complexity Bounds for Entropic Best Policy Identification
May 13, 2026Amer Essakine, Claire VernadeEntropy Regularized Reinforcement Learning Boltzmann Policies
ENS Paris Saclay · University of Technology Nuremberg
LLMs as Implicit Imputers: Uncertainty Should Scale with Missing Information
May 13, 2026Stef van BuurenLarge Language Model Uncertainty Incomplete
1. TNO - Netherlands Organization for Applied Scientific Research, Leiden · 2. Dept. of Methodology and Statistics, University of Utrecht
ERPPO: Entropy Regularization-based Proximal Policy Optimization
May 13, 2026Changha Lee, Gyusang ChoProximal Policy Optimization Multi-Agent Reinforcement Learning
Korea Advanced Institute of Science and Technology, Daejeon, South Korea
Manifold Sampling via Entropy Maximization
May 12, 2026Cornelius V. Braun, Tilman Burghoff, Marc ToussaintInequality Constraints Optimal Sample Complexity
Technische Universität Berlin · Robotics Institute Germany
Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
May 12, 2026Jiazheng Zhang, Ziche Fu, Junrui Shen +17Frictive Policy Optimization Token-Level Entropy
Fudan NLP Group, Honor Device Co., Ltd · University of Hong Kong, Shanghai Jiao Tong University and Tencent Hunyuan
Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging
May 11, 2026Guisong Liu, Xin Gao, Martin Dresler +2Sleep Spatio-Temporal Transformers
School of Biological Science and Medical Engineering Southeast University Nanjing, China · University of Bath Bath, England · Donders Institute for Brain, Cognition and Behaviour Radboud University Medical Center Nijmegen, The Netherlands +1
Measuring and Decomposing Mode Separation via the Canonical Diffusion
May 9, 2026Shaul Tolkovsky, Ori Meidler, Or ZukDispersion Dynamic Mode Decomposition
Department of Statistics and Data Science Hebrew University of Jerusalem
Direct Estimation of Schrödinger Bridge Time-Series Drifts: Finite-Sample, Asymptotic, and Adaptive Guarantees
May 6, 2026Othmane Mazhar, Huyên PhamSchrödinger Bridges Finite-Sample
LPSM, Sorbonne Université and Université Paris Cité · École Polytechnique, CMAP
Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels
May 3, 2026Vishnu Teja Kunde, Jean-Francois Chamberland, Krishna R. Narayanan +1Source-Channel Coding Data Compression Methods
Texas A&M University · Brigham Young University
Entropy of Ukrainian
Apr 30, 2026Anton Lavreniuk, Mykyta Mudryi, Markiian ChakloshEntropy Ukrainian
Polish-Japanese Academy of Information Technology · University of the National Education Commission in Kraków
Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation
Apr 29, 2026Jon-Paul CacioliInstruction-Tuned Models Collapse
Independent Researcher, Melbourne, Australia
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Diffusion Language Models Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Apr 29, 2026Bolian Li, Yifan Wang, Yi Ding +3Large Language Model Reinforcement Learning Entropy Regularized Reinforcement Learning
Purdue University, West Lafayette, IN, USA
Option-Order Randomisation Reveals a Distributional Position Attractor in Prompted Sandbagging
Apr 29, 2026Jon-Paul CacioliMmlu-Pro Position Bias
Independent Researcher, Melbourne, Australia
Entropy Centroids as Intrinsic Rewards for Test-Time Scaling
Apr 28, 2026Wenshuo Zhao, Qi Zhu, Xingshan Zeng +4Token-Level Entropy Test-Time Scaling
May · HKUST · ZJU +1
Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
Apr 26, 2026Nikolaos Al. Papadopoulos, Konstantinos E. PsannisInformation-Theoretic Limits Entropy
Department of Applied Informatics, University of Macedonia, Egnatia 156, 54636 Thessaloniki, Greece
Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy
Apr 24, 2026Asim Ukaye, Mubarak Abdu-Aguye, Nurbek Tastan +1Federated Learning Contributions
MBZUAI, UAE · Michigan State University, USA
Leakage-Audited Benchmarking Reveals Limited Evidence for Cross-Subject Auditory-Evoked EEG Vowel Perception Decoding
Apr 22, 2026Xiaoyang Li, Zeyan TaoElectroencephalography Auditory Attention
College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, 110016, China
Targeted Exploration via Unified Entropy Control for Reinforcement Learning
Apr 16, 2026Chen Wang, Lai Wei, Yanzhi Zhang +5Large Language Model Reinforcement Learning Reasoning Trajectory
College of Software, Nankai University · Zhongguancun Academy · Shanghai Jiao Tong University +3
INTRYGUE: Induction-Aware Entropy Gating for Reliable RAG Uncertainty Estimation
Mar 23, 2026Alexandra Kuleshova, Andrei Volodichev, Daria Kotova +1Large Language Model Uncertainty Large Language Model Hallucination
Applied AI Institute
Semantic Chunking and the Entropy of Natural Language
Feb 13, 2026Weishun Zhong, Doron Sivan, Tankut Can +2Semantic Entropy Entropy
School of Natural Sciences, Institute for Advanced Study, Princeton, NJ, 08540, USA · Department of Brain Sciences, Weizmann Institute of Science, Rehovot, 76100, Israel
Entropy-Generated Attention Beyond Softmax and Entmax: Kaniadakis and Reciprocal-Symmetric Abe Operators
Feb 9, 2026Gunn KimEntropy Gumbel-Softmax Relaxation
Department of Physics, Sejong University, Seoul 05006, Republic of Korea
Persistent Entropy as a Detector of Phase Transitions
Feb 8, 2026Marcos Gutierrez-del-Pozo, Eduardo Paluzo-Hidalgo, Matteo RuccoPhase Transitions Change-Point Detection
Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning
Jan 27, 2026Tongxi Wang, Zhuoyang Xia, Xinran Chen +1Entropy Regularized Reinforcement Learning Offline Reinforcement Learning
School of Future Technology, Southeast University, Nanjing, China · School of Automation, Southeast University, Nanjing, China
DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy
Aug 16, 2025Frederik L. Dennig, Daniel A. KeimConditional Variational Autoencoder Star-Vae
University of Konstanz
Learning from samples: inverse problems over measures
May 11, 2025Francisco Andrade, Gabriel Peyré, Clarice PoonBayesian Inverse Problems Inverse Problem
INRIA (PreMeDICaL & HeKA) · CNRS and ENS, PSL Universit´e · Mathematics Institute, University of Warwick
An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression
Apr 25, 2025Christopher Bülte, Yusuf Sale, Timo Löhr +3Uncertainty Quantification Uncertainty
1LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2
Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation
Dec 10, 2024Qinhong Lin, Yinglun Feng, Yuhao Zhang +2Large Language Model Uncertainty Uncertainty
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · QuanCheng Laboratory, Jinan, China
TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning
Jan 21, 2024Charles Westphal, Stephen Hailes, Mirco MusolesiProximal Policy Optimization Entropy
UCL Centre for Artificial Intelligence Department of Computer Science University College London Gower Street, London WC1E 6BT, United Kingdom · Department of Computer Science University College London Gower Street, London WC1E 6BT, United Kingdom · Department of Computer Science and Engineering University of Bologna Viale Del Risorgimento 2, 40136 Bologna, Italy
Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy
Feb 17, 2023Łukasz DębowskiEntropy Scaling Laws
Institute of Computer Science, Polish Academy of Sciences, ul. Jana Kazimierza 5, 01-248 Warszawa, Poland
Information entropy as an anthropomorphic concept
Mar 6, 2015Panteleimon RodisEntropy Thermodynamics
Diploma in Computer Science, Hellenic Open University

May 22, 2026, cs.DSJ/K move · Enter open · S save

Entropy Equivalence Testing

Clément L. Canonne, Yash Pote, Jonathan Scarlett, Joy Qiping Yang

University of Sydney · National University of Singapore

Two-Sample Testing Optimal Sample Complexity Entropy Equivalence Bayesian Networks Proximity Operations

Open paper AskPDF arXiv

We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given samples from two unknown distributions

p,q

and a parameter

\varepsilon \in(0,1/2]

, between

p=q

and

|H(p)-H(q)| \geq \varepsilon

(where

H

denotes the Shannon entropy). We provide a time- and sample-efficient algorithm for this task, showing that the optimal sample complexity for this task can be significantly lower than that of closeness testing. As an application, we leverage this result to provide the first non-trivial testing algorithm for (standard) closeness of low-degree \emph{Bayesian networks}, which significantly improves on either the sample or time complexity of a baseline based on full learning.

Momentum

Related topics

Latest papers 96