Information-Theoretic Limits

Ask about this topic

Momentum

15 papers in the last four weeks, up 275% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 97

All topics
CardsList

  1. On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck

    Oct 5, 2026Dier Tang, Jun ChenInformation-Theoretic Limits

    Department of Mathematics, The University of Hong Kong · Electrical and Computer Engineering, McMaster University

  2. Exact information accounting for SGD methods

    Sep 30, 2026Akshay BalsubramaniStochastic Gradient DescentGradient-Based Optimization

  3. The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

    Sep 28, 2026Yuxiang Liu, Lizhi Yang, Fengze Xie +2Contrastive LearningInformation-Theoretic Limits

    University of California, Berkeley · California Institute of Technology

  4. Optimizing and Securing the Modern Watermarking Channel for Images

    Sep 28, 2026Enoal Gesny, Eva GiboulotMulti-Bit WatermarkingWatermarks

    Inria, Rennes, France

  5. Information-Theoretic Analysis of Next-Token Prediction under Markovian Data

    Sep 28, 2026Masoud Kavian, Abdellatif Zaidi, Milad SefidgaranNext-Token PredictionInformation-Theoretic Limits

    Mathematical and Algorithmic Science Laboratory, Huawei Paris Research Center, 92100 Boulogne-Billancourt, France · Laboratoire d’Informatique Gaspard Monge, Université Gustave Eiffel, 77420 Champs-sur-Marne, France

  6. Understanding Generalization Requires Universal Induction

    Sep 28, 2026Aram Ebtekar, Marcus Hutter, Danica J. SutherlandInductive BiasInduction

    AIXI Labs · Google DeepMind & ANU · UBC & Amii

  7. Test-Time Scaling via Budgeted Multi-Attribute Verification

    Sep 28, 2026Bo Xue, Ji Cheng, Shen-Huan Lyu +2Token Budget AllocationLarge Language Model Responses

    City University of Hong Kong · Hohai University · Zhejiang University

  8. On the Information-Theoretic Limits of Latent-Space Watermarking Through Pretrained Generators

    Sep 21, 2026Jinwan Jeon, Minju Lee, Sung Hoon LimWatermarksWatermarking

    Hallym University, CHUNCHEON-SI, GANGWON-DO, South Korea

  9. Quantifying Mechanical Intelligence in Legged Robots with Information Theory

    Sep 17, 2026Zach J. PattersonLegged RobotsSoft Robotics

    Mechanical and Aerospace Engineering, Case Western Reserve University

  10. Efficient Robust Learning at the Information-Theoretic Limit

    Sep 15, 2026Adam R. Klivans, Konstantinos Stavropoulos, Sergei Tikhonov +1LearnabilityEmpirical Risk Minimization

    UT Austin · Institute for Advanced Study · Aarhus University

  11. Quantifying the Value of Privileged Information Using a PAC-Bayesian Approach

    Sep 14, 2026Vasily Bokov (aQa, Leiden University, The Netherlands +24Privacy-Preserving Machine LearningPac-Bayesian Theory

  12. When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

    Sep 12, 2026Emma Andrews, Gianmarco MengaldoText-DrivenInformation-Theoretic Limits

    National University of Singapore

  13. A Mathematical Theory of Pragmatic Information

    Sep 10, 2026Kai Niu, Ping ZhangInformation-Theoretic LimitsTheory

    Key Laboratory of Universal Wireless Communications, Ministry of Education · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China

  14. Speed Limit for Information Acquisition in Stochastic Learning Dynamics

    Sep 8, 2026Shuta Kobayashi, Andreas DechantStochastic Gradient DescentFisher Information Matrix

    Department of Physics, Kyoto University, Kyoto 606-8502, Japan

  15. Microcanonical Hamiltonian Monte Carlo and the Helmholtz Theorem

    Sep 7, 2026Heinrich von Campe, Bjoern Malte SchaeferMarkov Chain Monte CarloThermodynamics

    Interdisziplinäres Zentrum für wissenschaftliches Rechnen, Universität Heidelberg · Tübingen AI Center, University of Tübingen · ELLIS Institute Tübingen +2

  16. CPR-IE:A Compression-Prediction-Resource Intelligence Efficiency Metric

    Sep 7, 2026Xiantao JiangEfficiencyPrioritization

    College of Information Engineering, Shanghai Maritime University Shanghai 201306, China

  17. Strengthening Recursive Constructions for Zero-Error Shannon Capacity

    Aug 31, 2026Ravi TandonSource-Channel CodingInformation-Theoretic Limits

    School of Electrical, Computing and Software Engineering, University of Arizona, Tucson, AZ 85721, USA.

  18. Converse and Collision-Based Achievability for Node Localization with Hybrid Distance-Spectral Graph Positional Encodings

    Aug 31, 2026Zimo Yan, Yifan Li, Hao Li +4Positional EncodingGraph Laplacians

    National University of Defense Technology, Changsha, China · Wuhan University, Wuhan, China

  19. Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

    Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Replay-Based Continual LearningContinual Learning

    Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China

  20. ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover

    Aug 11, 2026Minwoo Kim, Soochang Song, Namyoon Lee +2Kv-Cache ManagementKey-Value Cache

    Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea · Department of Electrical and Computer Engineering, Ajou University, Suwon 16499, South Korea

  21. Memory--Batch Tradeoffs in Lipschitz Bandits

    Aug 8, 2026Zicheng Lyu, Zengfeng HuangContextual Bandit FrameworkInformation-Theoretic Limits

    Fudan University Shanghai, China · Fudan University Shanghai Innovation Institute Shanghai, China

  22. POMDPs for Autonomous Science Exploration

    Aug 4, 2026Daniel Guirguis, Nathan Wallace, Hanna Kurniawati +1Partially Observable Markov Decision ProcessStochastic Exploration

  23. An Information Theoretic Treatment of Yager's Probability Distribution Negation

    Aug 1, 2026Roberto Bruno, Ugo VaccaroProbabilityInformation-Theoretic Limits

    Department of Computer Science, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, 84084, Salerno, Italy.

  24. Improved lower bounds for the Shannon capacity of odd cycles

    Jul 23, 2026Nathaniel Itty, Christopher D. Rosin, Chase Carstensen +1Maximum Independent SetLower Bounds

    Worcester Polytechnic Institute · Constructive Codes

  25. Rate-Distortion-Perception Theory: Redefining the Fundamental Limits of Information Representation

    Jul 19, 2026Photios A. Stavrou, Giuseppe Serra, Marios KountourisRate-Distortion TheorySource-Channel Coding

  26. CAS I: A Geometric Coding Theorem

    Jul 15, 2026Romie BanerjeeFinite GroupsSymmetry

  27. Local Redundancy: An Information-Theoretic Measure of Plasticity from Synthetic Memorization

    Jul 15, 2026Jiaxuan ChengPlasticitySingular Learning Theory

    Massachusetts Institute of Technology, Cambridge, MA, USA.

  28. Watermark Forensics for Generative Models: An Information-Theoretic Perspective

    Jul 14, 2026Xiaoyu Li, Zheng Gao, Xiaoyan Feng +3WatermarksMulti-Bit Watermarking

    University of New South Wales · 2Griffith University

  29. Conservation Laws for Diffusion Models

    Jul 11, 2026Ziv Aharoni, Henry D. PfisterCross-Entropy LossesDiffusion Models

    Department of Electrical and Computer Engineering Duke University Durham, NC, USA

  30. An exact information theory of generalization phase transitions in Bayesian diffusion models

    Jul 9, 2026Henry Hunt, Mason Kamb, Surya GanguliDiffusion ModelsInformation-Theoretic Limits

    Department of Applied Physics Stanford University Stanford, CA 94305

  31. LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation

    Jul 5, 2026Hongchen Li, Bohao Wang, Jingbang Chen +5RecommendationLarge Language Model Bias

    State Key Laboratory of Blockchain and Data Security, Zhejiang University. · College of Computer Science and Technology, Zhejiang University. · Zhejiang University Hangzhou, Zhejiang, China +3

  32. Co-Adaptive Multi-Task LoRA: Transfer-Aware, Label-Free Control of Domain Participation

    Jul 3, 2026Wei Zhang, Lin Tang, Ming Zhao +1Domain AdaptationTraining-Side Stage-Aware Low-Rank Adaptation

    1Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China

  33. Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability

    Jun 29, 2026Srinivasa Rao P., Vangmayi P ReddyOverparameterizationLearnability

    Curlvee Technolabs, India, Former Scientist, C-DAC · Former Scientist, C-DAC · Indian Institute of Technology Madras, India

  34. TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference

    Jun 25, 2026Tinghao Wang, Yichen Guo, Rui Huang +11Visual Token PruningMultimodal Large Language Models

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanyang Technological University · University of Electronic Science and Technology of China +1

  35. Text Distance from Nested and Hierarchical Repetitions: A Compression-Based Perspective

    Jun 23, 2026Xiaojun Hu, Jing Wang, Jingwen Zhang +5Sequence ModelingLadder

    Department of Systems Science, Faculty of Arts and Sciences, Beijing Normal University, Zhuhai, China. · International Academic Center of Complex Systems, Beijing Normal University, Zhuhai, China. · School of Systems Science, Beijing Normal University, Beijing, China. +2

  36. Quantifying Theoretical AI Alignment Guarantees: Receiver-Utility Bounds in Bayesian Persuasion

    Jun 20, 2026Eric Yachbes, Eva TardosArtificial Intelligence AlignmentPersuasion

    Cornell University, United States

  37. What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis

    Jun 18, 2026Xinghao Chen, Chak Tou Leong, Wenjin Guo +3Efficient Latent ReasoningChain-of-Thought Reasoning

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China.

  38. An Information Theoretic Framework for Graph Novelty Generation via Latent Mixture Modeling

    Jun 18, 2026Itsuki Nakagawa, Kenji YamanishiMixture ModelsGenerative Models

    Graduate School of Information Science and Technology, The University of Tokyo

  39. Pride and Prejudice: Toward an Information-Theoretic Framework for Mutually Communicative Driver Behavior Modeling

    Jun 15, 2026Tingjun Li, Nan Xu, Shuo Feng +3PersuasionMulti-Agent Reinforcement Learning

    State Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University, 5988 Renmin Street, Changchun, 130025, Jilin, China · Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Engineering, Brock University, 1812 Sir Isaac Brock Way, St. Catharines, L2S3A1, Ontario, Canada +1

  40. The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

    Jun 14, 2026Thomas Dittrich, Oliver Potocki, Philipp GrohsMulti--Task LearningInformation-Theoretic Limits

    Johann Radon Institute of Computational and Applied Mathematics, Austrian Academy of Sciences · Faculty of Mathematics, University of Vienna

  41. Behavioral Audit of Machine Unlearning Has a Privacy Cost

    Jun 12, 2026Liou Tang, James Joshi, Ashish KunduMachine UnlearningModel Auditing

    University of Pittsburgh, Pittsburgh, PA, USA · Cisco Research, San Jose, CA, USA

  42. Recovery thresholds for hidden weighted sparse graphs

    Jun 12, 2026Zhe Hou, Jingcheng LiuInhomogeneous Random GraphsInformation-Theoretic Limits

    State Key Laboratory for Novel Software Technology, Nanjing University

  43. How Task Structure Limits Multi-Agent Success: An Information-Theoretic Analysis

    Jun 11, 2026Shi Pan, Ming LuoModel-Based Multi-Agent SystemsSingle-Agent

    University College London London, UK · University of Bristol Bristol, UK

  44. Information-Theoretic Decomposition for Multimodal Interaction Learning

    Jun 10, 2026Zequn Yang, Yake Wei, Haotian Ni +2Multimodal LearningInformation-Theoretic Limits

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +2

  45. Bellman-sufficient Information Complexity

    Jun 9, 2026Yunbei XuInformation-Theoretic LimitsBellman Equation

    National University of Singapore

  46. The Bioelectrical Information Theory: Investigating the theoretical compression limit of bioelectrical signals under artificial intelligence

    Jun 7, 2026Jiawen Zou, Bo YanBrain-Computer InterfaceInformation-Theoretic Limits

    College of Computer Science and Artificial Intelligence, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai 200433, P.R. China

  47. An Information-Theoretic Definition for Open-Ended Learning

    Jun 6, 2026Wanqiao Xu, Yifan Zhu, Benjamin Van RoyInformation-Theoretic LimitsStochastic Exploration

  48. The Capacity of Information-Theoretic Secure Aggregation in Federated Learning

    Jun 5, 2026Lanxin Yi, Jinbao Zhu, Kai Wan +1Secure AggregationFederated Learning

    Information Coding and Transmission (ICT) Key Laboratory of Sichuan Province, Southwest Jiaotong University, Chengdu 611756, China · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China

  49. InfoShield: Privacy-Preserving Speech Representations for Mental Health Screening via Information-Theoretic Optimization

    Jun 4, 2026Xueyang Wu, Siyuan Liu, Kezhuo Yang +1DepressionMental Health

    Shenzhen NeurStar Inc., China · University of York, United Kingdom · Shanghai Jiao Tong University, China

  50. Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration

    Jun 2, 2026Achraf Ben AhmedRemote PhotoplethysmographyCardiovascular Risk Prediction

    PlesmoSense SARL

  51. Query-Limited Community Recovery in Stochastic Block Models

    Jun 1, 2026Sabyasachi Basu, Manuj Mukherjee, Lutz Oettershagen +1O(\Bar{K}\Log N)$Subgraphs

    1Microsoft Research, India · 2Indraprastha Institute of Information Technology, India · University of Liverpool, United Kingdom +1

  52. Physics-Informed Deep Learning for Entropy Prediction in Heterogeneous Systems: Thermodynamic and Information-Theoretic Case Studies

    May 31, 2026Biswajeet Sahoo, Debadutta PatraParametric Physics-Informed Neural NetworkPhysics-Informed Learning

    Durham University, Durham, UK · Department of Chemical Engineering Veer Surendra Sai University of Technology, Burla, India

  53. Information-Theoretic Lower Bounds for Bit-Constrained Stochastic Optimization via a Reduction to Compressed Gaussian Mean Estimation

    May 30, 2026Munsik KimInformation-Theoretic LimitsStochastic Optimization

  54. The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs

    May 30, 2026Zihan Chen, Yiming Zhang, Wenxiang Geng +2LLM Reasoning StrategiesProcess Reward Model

    1HFIPS, Chinese Academy of Sciences · University of Science and Technology of China

  55. Sequential Neural Probabilistic Amplitude Shaping: Learning the Channel's Language

    May 27, 2026Mohammad Taha Askari, Lutz Lampe, Amirhossein GhazisaeidiSource-Channel CodingInformation-Theoretic Limits

    Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada · Nokia Bell Labs, 12 rue Jean Bart, 91300 Massy, France

  56. From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithms

    May 25, 2026Christoph H. Lampert, Max Cairney-Leeming, Hossein ZakeriniaStandard Differential-PrivacyGeneralization Bounds

    Institute of Science and Technology Austria (ISTA) Klosterneuburg, Austria

  57. Capability and Robustness Cannot Both Be Free: An Information-Theoretic Bound for Vision-Language-Action Models

    May 25, 2026Jianwei TaiCross-Architecture GeneralizationEncoders

    School of Internet, Anhui University

  58. Adversarial Water-Filling: Theory, Algorithms, and a Domain-Specific Wireless Foundation Model

    May 24, 2026Xindi Tong, Chee Wei Tan, H. Vincent PoorPower AllocationFrequency-Aware Regularization

    College of Computing and Data Science (CCDS), Nanyang Technological University, Singapore · Princeton University, United States

  59. The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems

    May 21, 2026Dongxin GuoTrustworthy Artificial IntelligenceImpossibility

    UNIVERSITY OF HONG KONG

  60. A Van Trees Lower Bound for Fully Interactive Differentially Private Federated Learning

    May 19, 2026T. Tony Cai, Yicheng LiFederated LearningFisher Information Matrix

    Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, USA. · KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China.