Information-Theoretic Lower Bounds

Ask about this topic

Overview

This field studies fundamental limits on how accurately or efficiently algorithms can learn, estimate, predict, or make decisions from limited information. Researchers derive lower bounds using tools such as Fano’s and Le Cam’s inequalities, KL divergence, mutual information, and hard-instance constructions, often comparing them with algorithms’ upper bounds. Common settings include bandit regret and sample complexity, quantum-state tomography, differentially private learning, compressed sensing, and communication-constrained optimization. Matching upper and lower bounds are known in many precisely defined models, while a continuing challenge is understanding how adaptivity, feedback, computational restrictions, and structural assumptions change the limits.

Latest papers

CardsList

83 papers18 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

  1. Information-theoretic Limits of Learning and Estimation

    May 6, 2026Abbas El Gamal, Maxim RaginskyInformation-Theoretic Generalization BoundsMinimax Estimation

    Stanford University* · UIUC**

  2. The Impossibility Triangle of Long-Context Modeling

    May 6, 2026Yan ZhouLong-Context ModelingAssociative Memory

    School of Mathematics and Statistics Changsha University of Science and Technology Changsha, Hunan 410114, China

  3. Provable Accuracy Collapse in Embedding-Based Representations under Dimensionality Mismatch

    May 5, 2026Dionysis Arvanitakis, Vaggos Chatziafratis, Yiyuan LuoIntrinsic DimensionalityContrastive Learning

    Northwestern University, IL, USA · University of California

  4. The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy

    May 4, 2026Seyed Morteza EmadiStructural Causal ModelsCausal Inference

    Kenan-Flagler Business School, University of North Carolina at Chapel Hill

  5. Mean Testing under Truncation beyond Gaussian

    May 2, 2026Yuhao Wang, Roberto Imbuzeiro Oliveira, Themis GouleakisSample ComplexityInformation-Theoretic Lower Bounds

    Amazon · IMPA, Rio de Janeiro, RJ, Brazil · Nanyang Technological University

  6. The Exact Replica Threshold for Nonlinear Moments of Quantum States

    Apr 24, 2026Shuai ZengSample ComplexityInformation-Theoretic Lower Bounds

    School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P.R. China

  7. Watts-per-Intelligence Part II: Algorithmic Catalysis

    Apr 21, 2026Elija PerrierEnergy-Efficient MLInformation-Theoretic Lower Bounds

    Centre for Quantum Software and Information, University of Technology Sydney, Australia.

  8. The Query Channel: Information-Theoretic Limits of Masking-Based Explanations

    Apr 17, 2026Erciyes Karakaya, Ozgur ErcetinFeature AttributionExplainable Artificial Intelligence

    *Department of Electrical and Computer Engineering, University of Maryland, College Park, USA · †Faculty of Engineering and Natural Sciences, Sabanci University, Turkiye

  9. Instantiating Bayesian CVaR lower bounds in Interactive Decision Making Problems

    Apr 14, 2026Raghav Bongole, Tobias J. Oechtering, Mikael SkoglundGaussian Process BanditsDecision-Making under Uncertainty

    Department of Information Science and Engineering (ISE), KTH Royal Institute of Technology

  10. Query Lower Bounds for Diffusion Sampling

    Apr 12, 2026Zhiyang Xun, Eric PriceQuery ComplexityDiffusion Sampling

    UT Austin

  11. Semantic Channel Theory: Deductive Compression and Structural Fidelity for Multi-Agent Communication

    Apr 10, 2026Jianfeng XuSemantic CommunicationInformation-Theoretic Lower Bounds

    Koguan School of Law, China Institute for Smart Justice, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200030, China

  12. Optimal Multi-bit Generative Watermarking Schemes Under Worst-Case False-Alarm Constraints

    Apr 9, 2026Yu-Shin Huang, Chao Tian, Krishna NarayananMulti-Bit WatermarkingText Watermarking

  13. Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimes

    Apr 9, 2026Ivan Lau, Jonathan ScarlettMinimax EstimationAdaptive Sampling

    National University of Singapore

  14. Forecastability as an Information-Theoretic Limit on Prediction

    Mar 28, 2026Peter Maurice CattTime Series ForecastingInformation-Theoretic Lower Bounds

    Independent Researcher, Auckland, New Zealand

  15. On the Peril of (Even a Little) Nonstationarity in Satisficing Regret Minimization

    Mar 19, 2026Yixuan Zhang, Ruihao Zhu, Qiaomin XieRegret MinimizationNon-Stationary Bandits

    Department of Industrial & Systems Engineering, University of Wisconsin-Madison · SC Johnson College of Business, Cornell University

  16. What resources are needed for optimal learning of bosonic Gaussian states?

    Mar 18, 2026Senrui Chen, Francesco Anna Mele, Marco Fanizza +5Quantum State TomographyInformation-Theoretic Lower Bounds

    Institute for Quantum Information and Matter, Caltech, Pasadena, CA 91125, USA · Scuola Normale Superiore, Piazza dei Cavalieri 7, 56126 Pisa, Italy · Inria, Télécom Paris – LTCI, Institut Polytechnique de Paris, France

  17. The Sampling Complexity of Condorcet Winner Identification in Dueling Bandits

    Mar 16, 2026El Mehdi Saad, Victor Thuot, Nicolas VerzelenDueling BanditsBest-Arm Identification

    CC-UM6P-Rabat · INRAE, MISTEA, Institut Agro, Univ. Montpellier Montpellier, France · MISTEA +1

  18. WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization

    Mar 5, 2026Egor Lifar, Semyon Savkin, Or Ordentlich +1Neural Network QuantizationLLM Quantization

    MIT · Independent Researcher · Hebrew University of Jerusalem

  19. Low-degree Lower bounds for clustering in moderate dimension

    Feb 26, 2026Alexandra Carpentier, Nicolas VerzelenLow-Degree Polynomial MethodClustering

    Institut für Mathematik, Universität Potsdam, Germany. · INRAE, MISTEA, Univ. Montpellier, Montpellier, France

  20. Tighter Regret Lower Bound for Gaussian Process Bandits with Squared Exponential Kernel in Hypersphere

    Feb 20, 2026Shogo IwazakiGaussian Process BanditsMinimax Regret

    LY Corporation, Tokyo, Japan

  21. Fundamental Limits of Black-Box Safety Evaluation: Information-Theoretic and Computational Barriers from Latent Context Conditioning

    Feb 19, 2026Vishal SrivastavaAI Safety EvaluationBlack-Box Model Auditing

    Whiting School of Engineering, John Hopkins University, Baltimore, MD 21218

  22. On the Rate-Distortion-Complexity Tradeoff for Semantic Communication

    Feb 16, 2026Jingxuan Chai, Yong Xiao, Guangming ShiSemantic CommunicationRate-Distortion Theory

    the School of Artificial Intelligence at the Xidian University, Shaanxi 710071, China · the School of Electronic Information and Communications at the Huazhong University of Science and Technology, Wuhan 430074, China · the Pengcheng Laboratory, Shenzhen, Guangdong 518055, China +2

  23. Calculating Mutual Information between a Reward Maximizer and its Environment

    Feb 13, 2026Alfred Harwood, Jose Faustino, Alex AltairMarkov Decision ProcessesReinforcement Learning

    1Dovetail Research · University of Sao Paulo