Information Bottleneck

Also known as IB

Information Bottleneck methods seek compact representations that preserve information relevant to a task while discarding irrelevant details. Common approaches include variational and conditional bottlenecks, mutual-information or rate-distortion objectives, and contrastive or structured regularizers. They are used in learning robust features, protecting privacy, explaining predictions, and limiting communication or memory in agents and multimodal systems. A central challenge is choosing and optimizing the compression level: too little compression retains nuisance information, while too much removes evidence needed for the task, and the right balance depends on the setting.

Latest papers

56 papers10 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment
May 28, 2026Tianchao Li, Shujian Yu, Xinrui Zu +4Cross-Modal Learning Cross-Modal Alignment
Hong Kong University of Science and Technology, Hong Kong · Vrije Universiteit Amsterdam, Netherlands · UiT – The Arctic University of Norway, Norway +3
Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization
May 27, 2026Hao Jiang, Shurui Li, Tianpeng Bu +7RL for Language Models Exploration-Exploitation Tradeoff
Alibaba Cloud Computing, Alibaba Group.
Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning
May 25, 2026Aleksandar Todorov, Matthia SabatelliRepresentation Learning Reinforcement Learning
University of Groningen, Groningen, The Netherlands
Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learning
May 17, 2026Wei Duan, Junyu Xuan, En Yu +2Graph Structure Learning Information Bottleneck
Australian Artificial Intelligence Institute (AAII), University of Technology Sydney
MLGIB: Multi-Label Graph Information Bottleneck for Expressive and Robust Message Passing
May 13, 2026Chaokai Wu, Haofu Shi, Ningxuan Ma +2Multi-Label Classification Graph Neural Networks
VNDUQE: Information-Theoretic Novelty Detection using Deep Variational Information Bottleneck
May 12, 2026Aryan Gondkar, Hayder Radha, Yiming DengUncertainty Quantification Information Bottleneck
Nondestructive Evaluation Lab, Department of Electrical and Computer Engineering Michigan State University East Lansing, MI · Department of Electrical and Computer Engineering Michigan State University East Lansing, MI
Generalization Bounds of Emergent Communications for Agentic AI Networking
May 9, 2026Yong Xiao, Jingxuan Chai, Guangming Shi +16G Networks Information Bottleneck
School of Elect. Inform. & Commun., Huazhong Univ. of Science & Technology, China · School of Artificial Intelligence, Xidian University, Xi’an, China · Peng Cheng Laboratory, Shenzhen, China +1
Distributed Deep Variational Approach for Privacy-preserving Data Release
May 4, 2026Zahir Alsulaimawi, Huaping LiuInformation Bottleneck Privacy-Preserving ML
Mixed-Precision Information Bottlenecks for On-Device Trait-State Disentanglement in Bipolar Agitation Detection
May 4, 2026Joydeep ChandraMixed-Precision Quantization Information Bottleneck
Department of Computer Science & Technology, Tsinghua University, Beijing, China.
Why Self-Supervised Encoders Want to Be Normal
Apr 30, 2026Yuval DombRepresentation Collapse Self-Supervised Learning
Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data
Apr 27, 2026K. Michael Martini, Eslam Abdelaleem, Paarth Gulati +1Dynamical Systems Information Bottleneck
Department of Physics, Emory University, Atlanta, GA, USA · Schools of Physics and Psychology, Georgia Institute of Technology, Atlanta, GA, USA · Initiative in Theory and Modeling of Living Systems, Emory University, Atlanta, GA, USA +1
Modeling Higher-Order Brain Interactions via a Multi-View Information Bottleneck Framework for fMRI-based Psychiatric Diagnosis
Apr 20, 2026Kunyu Zhang, Qiang Li, Vince D. Calhoun +1Functional Connectivity Multi-View Learning
Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, 1081 HV, The Netherlands · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University, Atlanta, GA, 30303, USA · Department of Physics and Technology, UiT - The Arctic University of Norway, Tromsø, 9019, Norway
DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation
Apr 13, 2026Yang Yan, Yunxuan Li, Qiuyan Wang +2Online Knowledge Distillation Graph Transfer Learning
School of Information Technology and Engineering, Tianjin University of Technology and Education, Tianjin, China · School of Computer Science and Technology, Tiangong University, Tianjin, China · Department of Computer Science at University of Exeter
Learning is Forgetting: LLM Training As Lossy Compression
Apr 8, 2026Henry C. Conklin, Tom Hosking, Tan Yi-Chern +5Language Model Pretraining LLM Compression
Princeton University · Cohere
The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling
Apr 3, 2026Takuya ShibaVision-Language-Action Models Information Bottleneck
Shibattic Inc.
Translation from the Information Bottleneck Perspective: an Efficiency Analysis of Spatial Prepositions in Bitexts
Mar 20, 2026Antoine Taroni, Ludovic Moncla, Frederique LaforestInformation Bottleneck
INSA Lyon, CNRS, Lyon 1 Universi
Agent-based imitation dynamics can yield efficiently compressed population-level vocabularies
Mar 16, 2026Nathaniel Imel, Richard Futrell, Michael Franke +1Information Bottleneck Evolutionary Game Theory
Department of Psychology, New York University · Department of Language Science, University of California, Irvine · Department of Linguistics, University of Tübingen
Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck
Mar 9, 2026Fabio Valerio Massoli, Andrey Kuzmin, Arash BehboodiEfficient Language Model Reasoning CoT Compression
Qualcomm AI Research
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
Mar 2, 2026Niu Lian, Yuting Wang, Hanshu Yao +5Multimodal Memory Memory-Augmented Video Understanding
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
Artificial Agency Program: Curiosity, compression, and communication in agents
Feb 27, 2026Richard CsakyAgentic AI Embodied AI
Human Supervision as an Information Bottleneck: A Unified Theory of Error Floors in Human-Guided Learning
Feb 26, 2026Alejandro Rodriguez DominguezInformation Bottleneck Learning with Noisy Labels
Miralta Finance Bank S.A., Spain · Department of Computer Science, University of Reading, UK
Concepts' Information Bottleneck Models
Feb 16, 2026Karim Galliamov, Syed M Ahsan Kazmi, Adil Khan +1Concept Bottleneck Models Information Bottleneck
University of Amsterdam · University of West of England · University of Hull +1
XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
Jan 29, 2026Alexandre Myara, Nicolas Bourriez, Thomas Boyer +3Variational Autoencoders Disentangled Representation Learning
ENS Paris
The Inductive Bottleneck: Data-Driven Emergence of Representational Sparsity in Vision Transformers
Dec 8, 2025Kanishk AwadhiyaRepresentation Learning Vision Transformer
Indian Institute of Technology, Delhi
Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity
Nov 4, 2025Ronald KatendeIntrinsic Dimensionality Information Bottleneck
Department of Mathematics, Kabale University, Katuna Road, Kikungiri Hill, Kabale, 317, Uganda
VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR
May 23, 2025Shenghui Chen, Po-han Li, Sandeep Chinchali +1Information Bottleneck Video Summarization
The University of Texas at Austin

May 28, 2026J/K move · Enter open · S save

OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment

Tianchao Li, Shujian Yu, Xinrui Zu, Zhaolong Wei+3

Hong Kong University of Science and Technology, Hong Kong · Vrije Universiteit Amsterdam, Netherlands · UiT – The Arctic University of Norway, Norway+3

Cross-Modal Learning Cross-Modal Alignment Information Bottleneck Multimodal Contrastive Learning Multimodal Learning Multimodal Understanding

Open paper AskPDF arXiv

Contrastive learning is effective for aligning paired views or modalities, but alignment beyond two modalities remains non-trivial and comparatively underexplored. Pairwise CLIP-style losses decompose multi-modal alignment into independent two-way comparisons and therefore do not explicitly model higher-order dependencies among multiple modalities. Recent beyond-pairwise objectives approach this problem from statistical or geometric perspectives, but arbitrary-modality alignment still lacks a principled criterion for defining what each modality should preserve and compress relative to the others. We revisit arbitrary-modality alignment through the Information Bottleneck principle. In multi-modal learning, sufficiency should preserve information predictable from the remaining modalities, while minimality should compress modality-specific information not supported by them. This naturally leads to a One-vs-All view, where each modality is characterized with respect to the remaining modalities. We propose OVA-IB, an Information Bottleneck framework for arbitrary-modality alignment. OVA-IB optimizes a tractable One-vs-All contrastive lower bound for sufficiency connected to a Dual Total Correlation-style objective, uses a parameter-free geometry-aware projection score, and derives a tractable upper-bound regularizer for minimality by bounding each representation's dependence on its own input with representation distributions induced by the remaining modalities. Experiments on classification, regression, modality-agnostic evaluation, and cross-modal retrieval benchmarks demonstrate strong and robust performance.