Information-Theoretic Lower Bounds

Ask about this topic

Overview

This field studies fundamental limits on how accurately or efficiently algorithms can learn, estimate, predict, or make decisions from limited information. Researchers derive lower bounds using tools such as Fano’s and Le Cam’s inequalities, KL divergence, mutual information, and hard-instance constructions, often comparing them with algorithms’ upper bounds. Common settings include bandit regret and sample complexity, quantum-state tomography, differentially private learning, compressed sensing, and communication-constrained optimization. Matching upper and lower bounds are known in many precisely defined models, while a continuing challenge is understanding how adaptivity, feedback, computational restrictions, and structural assumptions change the limits.

Latest papers

CardsList

83 papers18 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

  1. The Cost of Working Memory in Streaming Mean Estimation

    Oct 29, 2026Zeyu Wu, Zihao Wang, Minghua DengStreaming AlgorithmsMinimax Estimation

  2. No Escape from the Quadratic Barrier: Adaptive Attacks on Linear ℓ2\ell_2 Sketches

    Oct 18, 2026Edith Cohen, Uri StemmerRandomized SketchingInformation-Theoretic Lower Bounds

    Google Research · Tel Aviv University

  3. Query-Dependent Reading for Quantized Inner-Product Estimation: A Minimax Separation

    Oct 17, 2026Krishnakumar Balasubramanian, Siddharth Bhandari, Shiva Prasad KasiviswanathanQuantizationMinimax Estimation

    Amazon* · Department of Statistics, University of California, Davis

  4. Regret Lower Bound for Online Learning of Quantum States with Logarithmic Loss, with Applications to Quantum State Tomography and Confidence Sequences for Matrix Means

    Oct 15, 2026Yen-Huan Li, Wei-Fu TsengQuantum State TomographyInformation-Theoretic Lower Bounds

    Department of Computer Science and Information Engineering, National Taiwan University · Department of Mathematics, National Taiwan University · Center for Quantum Science and Engineering, National Taiwan University

  5. On Global Joint Lower Bounds in Zeroth-Order Nonconvex Optimization

    Oct 14, 2026Yuxuan Xia, Xiao-Tong YuanZeroth-Order OptimizationNonconvex Stochastic Optimization

  6. Rarity Determines Minimax Rates of Semantic Entropy From Weak Pairwise Judgments With a Known Channel

    Oct 13, 2026Xiaoli Yu, Jiamiao LiuSemantic EntropyMinimax Estimation

    School of Cyber Security and Information Law, Chongqing University of Posts and Telecommunications, Chongqing 400065, China · Department of Information, Xinqiao Hospital, Army Medical University (Third Military Medical University), Chongqing 400037, China

  7. Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets

    Oct 9, 2026Sebastian PokuttaLMO MethodsOracle Complexity

    Zuse Institute Berlin and TU Berlin

  8. Verification with Transfer: Exact Information Frontiers and Their Price in Calls

    Oct 8, 2026Hazar YuekselInformation-Theoretic Lower BoundsQuery Complexity

  9. Recovery Guarantees for Posterior Sampling of One-Bit Compressed Sensing

    Oct 8, 2026Jing Ma, Yujia Wu, Zhaoqiang LiuCompressed SensingSample Complexity

    University of Electronic Science and Technology of China

  10. New Lower Bound and Upper Bounds on the Regret for Online Sparse Linear Regression

    Oct 8, 2026Xiaofeng Cao, Junfan Li, Langzhang Liang +2Linear RegressionMinimax Regret

    School of Computer Science and Technology, Tongji University, Shanghai, China · School of Computing and Artificial Intelligence, Shanghai University of Finance and Economics, Shanghai, China · AI3 Institute, Fudan University and Shanghai Innovation Institute +2

  11. A General Ω~(TγT)\widetildeΩ(\sqrt{T γ_T}) Lower Bound for Kernel Bandits

    Oct 8, 2026Chenkai Ma, Jonathan ScarlettMinimax RegretMulti-Armed Bandits

    National University of Singapore

  12. An Accuracy-Information Tradeoff for Loss-Difference Conditional Mutual Information

    Oct 6, 2026Hazar YuekselStatistical Learning TheoryInformation-Theoretic Generalization Bounds

  13. Information Limits of Low-Rank Approximation Certification

    Oct 2, 2026Kang Liu, Bohao QuLow-Rank ApproximationInformation-Theoretic Lower Bounds

    School of Future Technology, Xi’an Jiaotong University, Xi’an, China. · Centre for Frontier AI Research (CFAR), A*STAR, Singapore.

  14. Nearly Optimal Fixed-Confidence Best-Arm Identification with 1-Bit Feedback

    Oct 2, 2026Khang Luong, Dinh Thai Son, Hoang Ta +2Best-Arm IdentificationMulti-Armed Bandits

    Hanoi University of Science and Technology · Quantum AI & Cyber Security Institute, FPT Corporation

  15. Structural Limits of the Information-Theoretic Uncertainty Decomposition

    Sep 30, 2026Jakob Lønborg Christensen, Christian F. Baumgartner, Morten Rieger Hannemose +2Aleatoric UncertaintyInformation-Theoretic Lower Bounds

    DTU Compute Technical University of Denmark · Faculty of Health Sciences and Medicine University of Lucerne

  16. Lower Bounds for Linear-Oracle Online Learning

    Sep 29, 2026Mohit SinhaOnline Convex OptimizationInformation-Theoretic Lower Bounds

  17. On Task Scope and Information Retention in Source Coding

    Sep 29, 2026Alireza Furutanpey, Kerstin BunteRate-Distortion TheoryInformation-Theoretic Lower Bounds

    NeverBlink, Warszawa, Poland · Rijksuniversiteit Groningen, Groningen, Netherlands

  18. A Sharp Transition in Data Reconstruction under Differential Privacy

    Sep 29, 2026Max Cairney-Leeming, Simone Bombari, Marco MondelliAdversarial AttacksTraining Data Reconstruction Attacks

    Institute of Science and Technology Austria

  19. QAM: Quadratic-Accurate Checkpoint Merging via Sequential Consistency

    Sep 28, 2026Shihao Wang, Rui Kong, Xinran Chen +7Language Model MergingModel Merging

    University of Wisconsin – Madison · Baidu Inc. · University of Arizona +3

  20. An Exposition of GPT Astra's Proof of Lower Bound on DP Continual Counting

    Sep 22, 2026Jalaj UpadhyayDifferential PrivacyInformation-Theoretic Lower Bounds

    Rutgers University

  21. When are bosonic Gaussian states classical to learn?

    Sep 22, 2026Senrui Chen, Antonio Anna Mele, Francesco Anna Mele +1Quantum State TomographyStatistical Learning Theory

    Institute for Quantum Information and Matter, Caltech, Pasadena, CA 91125, USA · Dahlem Center for Complex Quantum Systems, Freie Universität Berlin, 14195 Berlin, Germany

  22. On the Information-Theoretic Limits of Latent-Space Watermarking Through Pretrained Generators

    Sep 21, 2026Jinwan Jeon, Minju Lee, Sung Hoon LimRobust WatermarkingInformation-Theoretic Lower Bounds

    Hallym University, CHUNCHEON-SI, GANGWON-DO, South Korea

  23. What Must Survive? Exact Task-Information--State Frontiers Under Partial Task Revelation

    Sep 18, 2026Ronald KatendeInformation-Theoretic Lower BoundsLow-Rank Compression

  24. When Can World Models Recover Physical Laws?

    Sep 15, 2026Ye Yuan, Jun LiuWorld Model LearningSystem Identification

    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China · Department of Applied Mathematics, University of Waterloo, Waterloo, Canada

  25. Learning CNF Formulas from Uniform Random Solutions: Near-Tight Sample Complexity for Valiant's Algorithm

    Sep 14, 2026Weiming Feng, Yixiao Yu, Yiyao ZhangBoolean Function LearningSample Complexity

    School of Computing and Data Science, The University of Hong Kong. · State Key Laboratory for Novel Software Technology, New Cornerstone Science Laboratory, Nanjing University.

  26. Gap Entropy and Almost Instance-Wise Optimal Best-Arm Identification

    Sep 12, 2026Jiarui Yao, Jiaxi Zhao, Xiangxin ZhouBest-Arm IdentificationSample Complexity

    UIUC · NUS · Tencent Hunyuan

  27. A positive resolution of the gap-entropy conjecture

    Sep 9, 2026P. M. Aronow, Nathan Kallus, Patrick LopattoBest-Arm IdentificationInformation-Theoretic Lower Bounds

  28. Optimal Low-Rank Quantum State Tomography with Bounded-Sample Joint Measurements

    Sep 9, 2026Ashwin Nayak, Xingyu ZhouQuantum State TomographyInformation-Theoretic Lower Bounds

    Department of Combinatorics and Optimization, and Institute for Quantum Computing, University of Waterloo. · Department of Computer Science, University of British Columbia.

  29. Oracle Complexity of Stochastic Fixed-Point Equations with Nonexpansive Maps

    Sep 8, 2026Jelena Diakonikolas, Cristóbal Guzmán, David Martínez-RubioOracle ComplexityStochastic Approximation

    Department of Computer Sciences, University of Wisconsin-Madison · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile · IMDEA Software Institute, Madrid, Spain

  30. Speed Limit for Information Acquisition in Stochastic Learning Dynamics

    Sep 8, 2026Shuta Kobayashi, Andreas DechantNeural Network Training DynamicsInformation-Theoretic Lower Bounds

    Department of Physics, Kyoto University, Kyoto 606-8502, Japan