Rate-Distortion Theory

Follow this topic Ask about this topic

Rate-distortion theory asks how many bits are needed to represent a source while keeping reconstruction errors within an acceptable level. Classical bounds guide methods such as scalar, vector, and transform quantization, while learned codecs use neural representations and rate allocation to approach these limits. Current work extends the objective to perceptual realism, semantic meaning, downstream task utility, and resource costs in settings ranging from images and video to agent memory and distributed communication. A central tension is that standard distortion measures may not reflect what people perceive or tasks require, while guarantees for richer objectives often depend on restrictive assumptions.

Latest papers

28 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Neural Video Coding under Limited Delay: Theoretical Optimality and System Design
Oct 19, 2026Likun Cai, Jun Chen, Ashish KhistiNeural Video Compression Video Compression
University of Toronto · McMaster University
On Task Scope and Information Retention in Source Coding
Sep 29, 2026Alireza Furutanpey, Kerstin BunteRate-Distortion Theory Information-Theoretic Lower Bounds
NeverBlink, Warszawa, Poland · Rijksuniversiteit Groningen, Groningen, Netherlands
A Mathematical Theory of Pragmatic Information
Sep 10, 2026Kai Niu, Ping ZhangRate-Distortion Theory Value of Information
Key Laboratory of Universal Wireless Communications, Ministry of Education · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization
Sep 2, 2026Xianghong Fang, Wenlong Mou, Yuan Yuan +2Vector Quantization Product Quantization
University of Toronto · 2Boston College · 3Vijil
UniTAC: Universal Task-Aware Compression via Weighted Distortion Measures
Aug 17, 2026Homa Esfahanizadeh, Matin Mortaheb, Adeel Mahmood +2Model Compression Learned Image Compression
Nokia Bell Labs, Murray Hill, NJ 07974, USA
Rate-Distortion-Perception Theory: Redefining the Fundamental Limits of Information Representation
Jul 19, 2026Photios A. Stavrou, Giuseppe Serra, Marios KountourisLearned Image Compression Rate-Distortion Theory
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
Jul 9, 2026Ashwin Gerard Colaco, Nada LahjoujiRate-Distortion Theory LLM Memory
University of California, Irvine Irvine, California, USA
Minimum Distortion Quantization with Specified Output Distribution
Jun 9, 2026Aolin XuRate-Distortion Theory Rate-Distortion Optimization
A Geometric Lens on Physics-Aligned Data Compression
Jun 2, 2026Aleix Segui, Wesley ArmourRate-Distortion Theory Rate-Distortion Optimization
Department of Engineering Science, University of Oxford, Oxford, UK.
How Neural Losses Shape VAE Latents
May 30, 2026Giorgio Strano, Luca Cerovaz, Michele Mancusi +2Variational Autoencoders Neural Representation Geometry
1Sapienza University of Rome · 2Paradigma, Inc. · 3Moises Systems, Inc. +1
Path-dependent program induction under resource constraints explains human sequence learning
May 26, 2026Hanqi Zhou, David G. Nagy, Peter Dayan +1Computational Cognitive Modeling Rate-Distortion Theory
University of Tubingen, Tubingen, Germany · Center for Cognitive Science, Technical University Darmstadt, Darmstadt, Germany · 3Hessian.AI, Darmstadt, Germany +1
Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner-Ziv Bounds for KV Cache Compression
May 24, 2026Munsik KimKV Caching Rate-Distortion Theory
Independent Researcher.
Sparse In-Network Learning via Shortest-Path Backpropagation and Finite-Rate Gating
May 22, 2026Mohammad Reza Deylam SalehiCommunication-Efficient Distributed Training Rate-Distortion Theory
Nice, France
PrismQuant: Rate-Distortion-Optimal Vector Quantization for Gaussian-Mixture Sources
May 15, 2026Bumsu Park, Chanho Park, Youngmok Park +1Rate-Distortion Theory Gaussian Mixture Models
Department of Electrical Engineering POSTECH Pohang, South Korea
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
May 14, 2026Tommaso Mencattini, Francesco Montagna, Francesco LocatelloRate-Distortion Theory Representation Disentanglement
EPFL · Institute of Science and Technology Austria
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory
May 11, 2026Mingxi Zou, Zhihan Guo, Langzhang Liang +6Rate-Distortion Theory Long-Term Conversational Memory
Fudan University · The Chinese University of Hong Kong · Meta AI +3
Continuous First, Discrete Later: VQ-VAEs Without Dimensional Collapse
May 7, 2026Xinyu Zhao, Nikita Karagodin, Hamed Hassani +3Variational Autoencoders Rate-Distortion Theory
MIT
Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization
Apr 30, 2026Huayu Li, ZhengXiao He, Xiwen Chen +4Disentangled Representation Learning Rate-Distortion Theory
University of Arizona, AZ, USA · Morgan Stanley, NY, USA · Clemson University, SC, USA +2
Why Self-Supervised Encoders Want to Be Normal
Apr 30, 2026Yuval DombRepresentation Collapse Self-Supervised Learning
Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision
Apr 23, 2026Leyla Roksan Caglar, Pedro A. M. Mediano, Baihan LinVisual Representation Learning Rate-Distortion Theory
Windreich Department of AI and Human Health Icahn School of Medicine at Mount Sinai New York, NY, USA · Department of Computing Imperial College London London, UK · Windreich Department of AI and Human Health Department of Psychiatry Icahn School of Medicine at Mount Sinai New York, NY, USA
Secure Rate-Distortion-Perception: A Randomized Distributed Function Computation Approach for Realism
Apr 22, 2026Gustaf Åhlgren, Onur GünlüRate-Distortion Theory Rate-Distortion Optimization
Information Theory and Security Laboratory (ITSL), Linköping University, Sweden · Lehrstuhl für Nachrichtentechnik, Technische Universität Dortmund, Germany
A Synonymous Variational Perspective on the Rate-Distortion-Perception Tradeoff
Apr 16, 2026Zijian Liang, Kai Niu, Changshuo Wang +2Rate-Distortion Theory Rate-Distortion Optimization
Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
Semantic Rate-Distortion Theory: Deductive Compression and Closure Fidelity
Apr 13, 2026Jianfeng XuRate-Distortion Theory
Koguan School of Law, China Institute for Smart Justice, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200030, China.
Semantic Rate-Distortion for Bounded Multi-Agent Communication: Capacity-Derived Semantic Spaces and the Communication Cost of Alignment
Apr 10, 2026Anthony T. NixonMulti-Agent Communication Task-Oriented Semantic Communication
Training-Free Rate-Distortion-Perception Traversal With Diffusion
Mar 4, 2026Yuhan Wang, Suzhi Bi, Ying-Jun Angela ZhangGenerative Image Compression Rate-Distortion Theory
Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong · College of Electronic and Information Engineering, Shenzhen University, Shenzhen 518060, China
Same Compression Principle, Different Geometry: Rate-Distortion Signatures Dissociate Biological and Artificial Visual Systems
Mar 2, 2026Leyla Roksan Caglar, Pedro A. M. Mediano, Baihan LinRate-Distortion Theory Rate-Distortion Optimization
Windreich Department of AI and Human Health Icahn School of Medicine at Mount Sinai New York, NY, USA · Department of Computing Imperial College London London, UK · Windreich Department of AI and Human Health Department of Psychiatry Icahn School of Medicine at Mount Sinai New York, NY, USA
On the Rate-Distortion-Complexity Tradeoff for Semantic Communication
Feb 16, 2026Jingxuan Chai, Yong Xiao, Guangming ShiSemantic Communication Rate-Distortion Theory
the School of Artificial Intelligence at the Xidian University, Shaanxi 710071, China · the School of Electronic Information and Communications at the Huazhong University of Science and Technology, Wuhan 430074, China · the Pengcheng Laboratory, Shenzhen, Guangdong 518055, China +2

Oct 19, 2026J/K move · Enter open · S save

Neural Video Coding under Limited Delay: Theoretical Optimality and System Design

Likun Cai, Jun Chen, Ashish Khisti

University of Toronto · McMaster University

Neural Video Compression Video Compression Predictive Coding Rate-Distortion Theory

Open paper AskPDF arXiv

Traditional neural video compression (NVC) methods predominantly follow strictly causal predictive-coding paradigm, treating zero-frame delay as a fixed constraint rather than delay as an explicit design parameter. From an information-theoretic perspective, allowing limited frame delay enables a codec to exploit correlations with future frames within the delay window, thereby reducing uncertainty that is inaccessible to strictly causal prediction and potentially improving the rate-distortion tradeoff. In this work, we formalize this intuition through a rigorous rate-distortion analysis builds on the clique factorization of Markov networks, and propose Graph-Factored Sequential Coding (GFSC), a principled framework for sequential video coding under limited delay. To our knowledge, we provide the first theoretical result showing that, for general

K

-order Markov sources with positive distributions, a

K

-frame delay is sufficient to attain joint-coding rate-distortion optimality over a nontrivial distortion region, a guarantee unavailable to zero-delay causal coding in general. In practice, we instantiate GFSC through lightweight neural modules, allowing existing causal predictive codecs to be augmented without modifying their core architectures. Experiments across multiple NVC models and datasets show that GFSC consistently improves rate-distortion performance over the corresponding causal counterparts. Under matched delay budgets, GFSC also achieves competitive performance compared with representative bidirectional codecs.