Embedding Compression

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

OrBIT: Structure-Guided Embedding Compression
Oct 7, 2026Yunied Puig, Amit Kumar JaiswalEmbedding Compression LLM Compression
Jay Chaudhry Software Innovation Centre Indian Institute of Technology (BHU) Varanasi, India
Planetary Feature Fields are Scalable Earth Representations
Sep 29, 2026Arjun Rao, Sebastian Loeschcke, Anthony Fuller +3Implicit Neural Representations Geospatial Representation Learning
University of British Columbia · University of Copenhagen · Carleton University +2
Exact semantic readout from compressed vector representations
Sep 16, 2026Daniel QuigleyLinear Probing Embedding Compression
Center for Possible Minds Indiana University Bloomington Bloomington, IN 47408
Hybrid Quantum-Classical NLP Classification with Compact Semantic Representations: An Experimental Analysis of Representation Compression
Sep 9, 2026Ali Hassan, Zijia Zhao, Maha A. MetaweiHybrid Quantum-Classical ML Text Classification
German University in Cairo, Cairo, Egypt · University of Melbourne, Melbourne, Australia · High Performance Computing Lab, Electronics Research Institute, Cairo, Egypt
Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval
Sep 7, 2026Peichun Hua, Yunming XiaoMatryoshka Representation Learning Embedding Compression
The Chinese University of Hong Kong, Shenzhen
Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings
Sep 3, 2026Peichun Hua, Yunming XiaoSecure Multi-Party Computation Embedding Compression
The Chinese University of Hong Kong, Shenzhen · State Key Laboratory of Internet Architecture, Tsinghua University
RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search
Aug 31, 2026Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato +2Rotation-Based Quantization Vector Quantization
Google Zurich, Switzerland · Technical University of Munich Munich, Germany · University of Copenhagen Copenhagen, Denmark +1
Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems
Jul 26, 2026Baolei Li, Yiping Yuan, Yilin Zheng +6Semantic ID Recommendation Embedding Compression
YouTube · Google Deepmind
Progressive Cramming: Reliable Token Compression and What It Reveals
Jul 23, 2026Dmitrii Tarasov, Timofei Lashukov, Elizaveta Goncharova +1LLM Compression Embedding Compression
FusionBrain Lab, AXXX, Moscow, Russia · HSE University, Moscow, Russia · Artificial Intelligence Institute, Innopolis University, Innopolis, Russia
TurboVec: A Case Study in Cost-Efficient Private Retrieval for Enterprise RAG via Codebook-Oblivious Quantization
Jul 18, 2026Navnit Shukla, Kamal Pandey, Omsankar TiwariRetrieval-Augmented Generation Vector Quantization
Snowflake Inc. Tustin, California, USA · Rivian Automotive Inc. Irvine, United States of America · Google Deepmind Sunnyvale, California, USA
BitNet Text Embeddings
Jun 24, 2026Zhen Li, Xin Huang, Liang Wang +8LLM Quantization Embedding Compression
Peking University · Microsoft Research
Geometric and Information Compression of Representations in Deep Learning
Jun 19, 2026Linara Adilova, Henning Petzka, Asja Fischer +1Neural Representation Geometry Embedding Compression
Research Center Trustworthy Data Science and Security, Dortmund · Faculty of Computer Science, Ruhr University Bochum · Signal Processing and Speech Communication Laboratory, Graz University of Technology +2
Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach
Jun 19, 2026Pedro M. R. Pereira, Felipe A. P. de Figueiredo, Rausley A. A. de SouzaLLM Quantization Rotation-Based Quantization
National Institute of Telecommunications (Inatel), Santa Rita do Sapucaí, MG 37536-001, Brazil
Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation
Jun 17, 2026Shanshan Lyu, Yiwei Wang, Yujun Cai +2Long-Context Retrieval Document Chunking
Chongqing University · State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences +2
Sustainable Face Recognition on Low-Power Devices with VQ-VAE Embeddings
Jun 13, 2026Christos Chronis, Georgios Th. Papadopoulos, Iraklis VarlamisEnergy-Efficient ML Edge Inference
Dep. of Informatics and Telematics Harokopio University of Athens Athens, Greece
ColBERTSaR: Sparsified ColBERT Index via Product Quantization
Jun 4, 2026Eugene Yang, Andrew Yates, Dawn Lawrie +3Vector Quantization Multi-Vector Retrieval
Johns Hopkins University, Baltimore, MD, USA
When Is 0.1% Enough? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression
May 31, 2026Riku Kisako, Hayato Tsukagoshi, Ryohei SasanoEmbedding Compression Dimensionality Reduction
Graduate School of Informatics, Nagoya University
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval
May 28, 2026Lixuan Guo, Yifei Wang, Tiansheng Wen +3Learned Sparse Retrieval Multi-Vector Retrieval
Stony Brook University · Xidian University · Amazon AGI SF Lab +4
Clark Hash: Stateless Sparse Johnson-Lindenstrauss Quantization for Neural Embeddings
May 27, 2026Stanislav Kirdey, Clark Labs IncEmbedding Compression Neural Network Quantization
Clark Labs Inc.
DIVE: Embedding Compression via Self-Limiting Gradient Updates
May 20, 2026Dongfang ZhaoContrastive Learning Embedding Compression
SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation
May 18, 2026Longtao Jiang, Jianmin Bao, Zhendong Wang +4Normalizing Flows Embedding Compression
University of Science and Technology of China · Kling Team, Kuaishou Technology
Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings
May 17, 2026Wenxuan XiaoNeural Representation Geometry Embedding Compression
Changsha University
RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
May 14, 2026Zhengjia Zhong, Shuyan Ke, Zaizhou Lin +3Residual VQ Vector Quantization
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China.
Compressed Video Aggregator: Content-driven Module for Efficient Micro-Video Recommendation
May 9, 2026Yang Xiao, Huiyuan Chen, Kaiyuan Deng +5Video Representation Learning Embedding Compression
University of Tulsa · Amazon · University of Arizona +1
Benchmarking Wireless Representations: High-Dimensional vs. Compressed Embeddings for Efficiency and Robustness
May 3, 2026Murilo Batista, Shirin Salehi, Saeed Mashdour +3Representation Learning Embedding Compression
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, Aachen, Germany · Centre for Telecommunications Studies, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro 22541-041, Brazil
Adaptive Transform Coding for Semantic Compression
Apr 29, 2026Andriy Enttsel, Vincent CorlayLearned Image Compression Embedding Compression
Mitsubishi Electric R&D Centre Europe, Rennes, 35700, France
ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models
Apr 27, 2026Orhan Demirci, Sezer Aptourachman, Aydın KayaText Embeddings Word Embeddings
Department of Computer Engineering Hacettepe University Ankara, Turkey
Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling
Apr 23, 2026Yilong Chen, Yanxi Xie, Zitian Gao +10Memory-Augmented Language Models Language Model Scaling
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · IQuest Research +1
Balanced Co-Clustering of Users and Items for Embedding Table Compression in Recommender Systems
Apr 20, 2026Runhao Jiang, Renchi Yang, Donghao WuEmbedding Compression Recommender Systems
Hong Kong Baptist University · The Chinese University of Hong Kong
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Apr 20, 2026Yunjia Xi, Menghui Zhu, Jianghao Lin +4LLM Compression Embedding Compression
Shanghai Jiao Tong University Shanghai, China · Huawei Noah’s Ark Lab Shanghai, China · Antai College of Economics and Management, Shanghai Jiao Tong University Shanghai, China
Compressing Sequences in the Latent Embedding Space: $K$ -Token Merging for Large Language Models
Apr 16, 2026Zihao Xu, John Harvill, Ziwei Fan +3Token Merging LLM Compression
♢Rutgers University · ♡Mistral AI · ♠AWS AI Labs +3
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Jun 2, 2025Yimin DuText Embeddings Word Embeddings
Independent Researcher Beijing, China
FastE: Readout-Triggered Token Compression for LLM Embedding Inference
Date pendingJinsong Shu, Jinyong Wen, Baokun Wang +4Token Pruning LLM Inference Acceleration
Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · Ant Group +1

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

OrBIT: Structure-Guided Embedding Compression

Yunied Puig, Amit Kumar Jaiswal

Jay Chaudhry Software Innovation Centre Indian Institute of Technology (BHU) Varanasi, India

Embedding Compression LLM Compression Vector Quantization

Open paper AskPDF arXiv

Embedding tables are among the largest components of modern language models. Most compression methods fix a coding geometry such as coordinate blocks, low-rank subspaces, or unrestricted codebooks, and optimize within it. We instead ask whether the coding geometry can itself be discovered. We introduce \emph{OrBIT}, a structure-guided embedding compression framework that learns reusable local geometry from orbit dynamics and uses it to constrain a small set of shared codewords. The global reconstruction residual then decides where the fixed coding budget is spent, while redundant overlapping charts let local errors compensate one another after gluing. Our theory shows how tight-chart geometry controls distortion, how the global residual directs sequential allocation, and how data-geometry-guided refinement improves the codec. The resulting orbit machinery is compiled away, leaving a compact decoder in which the learned structure governs what is stored, where capacity is allocated, and how local information is assembled globally. Across four LLM embedding tables, OrBIT achieves

37.9\times

compression on GPT-2 and over

23\times

on each 7B table relative to 16-bit storage, while delivering competitive rate-distortion performance against established quantization and low-rank baselines.

Momentum

Related topics

Latest papers 32