Product Quantization

Also known as PQ

Product quantization (PQ) divides a vector into subvectors and replaces each with the index of a learned codeword, reducing storage and speeding up similarity comparisons. Common approaches combine PQ with inverted or hierarchical indexes for nearest-neighbor retrieval, or tune codebooks and specialized kernels to compress model weights and embeddings. The same discrete codes are also used as tokens for language, protein, and generative retrieval models. A central challenge is balancing compression and search speed against distortion and downstream accuracy, with comparisons depending on code rate, data characteristics, and the quantization method.

Latest papers

11 papers2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization
Oct 13, 2026Romain Hardy, Xiaoman Zhang, Pranav RajpurkarProtein Representation Learning Protein Language Models
Department of Biomedical Informatics, Harvard Medical School
A Systematic Study of Semantic ID Spaces for Generative Information Retrieval
Oct 6, 2026Alexia Allal, Hicham Randrianarivo, Sylvain LamprierResidual VQ Vector Quantization
Artefact Research Center Paris, France · LERIA, Angers University Angers, France
Product-Aware Deterministic Rounding for Quantized Matrix Multiplication
Sep 11, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Neural Network Quantization Activation Quantization
Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA
NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
Sep 9, 2026The Intern-NCP Team, :, Jiaqi Cao +26Language Model Pretraining Autoregressive Language Modeling
A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization
Sep 2, 2026Xianghong Fang, Wenlong Mou, Yuan Yuan +2Vector Quantization Product Quantization
University of Toronto · 2Boston College · 3Vijil
ColBERTSaR: Sparsified ColBERT Index via Product Quantization
Jun 4, 2026Eugene Yang, Andrew Yates, Dawn Lawrie +3Vector Quantization Multi-Vector Retrieval
Johns Hopkins University, Baltimore, MD, USA
Adversarial Attacks on Robot Localization Systems via Deep Feature Perturbation
Jun 1, 2026Zhenyu Li, Tianyi ShangAdversarial Attacks Vector Quantization
Shandong Academy of Sciences · Fuzhou University
Efficient Multivector Retrieval with Token-Aware Clustering and Hierarchical Indexing
Apr 30, 2026Silvio Martinico, Franco Maria Nardini, Cosimo Rulli +1Clustering Vector Quantization
University of Pisa & ISTI–CNR · Pisa, Italy · ISTI–CNR +1
Large-Scale Data Parallelization of Product Quantization and Inverted Indexing Using Dask
Apr 23, 2026Ashley N. Abraham, Andrew Strelzoff, Haley R. Dozier +2Information Retrieval Product Quantization
Information Technology Laboratory, U.S Army Engineer Research and Development Center, Vicksburg, MS, USA · Environmental Laboratory, U.S Army Engineer Research and Development Center,2026 Vicksburg, MS, USA
FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression
Apr 22, 2026Ye Qiao, Yian Wang, Zhiheng Chen +2LLM Compression GPU Acceleration
University of California, Irvine Irvine, California, USA
Panorama: Fast-Track Nearest Neighbors
Oct 1, 2025Vansh Ramani, Alexis Schlomer, Akash Nayar +3Product Quantization Approximate Nearest Neighbor Search
IIT Delhi & CMU & UCPH · Carnegie Mellon University · Indian Institute of Technology Delhi +1

Oct 13, 2026J/K move · Enter open · S save

ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization

Romain Hardy, Xiaoman Zhang, Pranav Rajpurkar

Department of Biomedical Informatics, Harvard Medical School

Protein Representation Learning Protein Language Models Product Quantization

Open paper AskPDF arXiv

Protein sequence, structure, and functional annotations each capture complementary aspects of protein biology, yet existing tokenizers encode each modality independently, limiting the expressiveness of discrete protein representations. We present ProTok, a unified protein tokenizer that maps all three modalities into a modular discrete vocabulary via product quantization with per-modality codebook groups and modality dropout. A single ProTok checkpoint supports flexible inference from any modality subset without retraining. On seven StructTokenBench benchmarks spanning 14 tasks and 28 test splits, ProTok achieves an average rank of 1.4 across all structure-only tokenizers, placing first on 21 splits and among the top two on 27 of the 28. These gains come without sacrificing reconstruction: ProTok's tokens reach 0.939 lDDT on CATH 4.2, showing that multi-codebook quantization and multi-modal training yield tokens that are both reconstructable and informative. ProTok's structure tokens further lead all baseline structure tokenizers on zero-shot functional retrieval across CATH, Enzyme Commission, and Gene Ontology label hierarchies. The vocabulary also supports generation: a compact language model trained from scratch over ProTok tokens jointly generates backbone and sequence, with 52% of sampled backbones designable versus 21% for a parameter-matched GeoBPE baseline. ProTok shows that structure, sequence, and function can share one discrete vocabulary, motivating protein language models that reason over a unified token stream.