Matryoshka Representation Learning

Also known as MRL

Follow this topic Ask about this topic

Latest papers 15

0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

All topics

Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval
Sep 7, 2026Peichun Hua, Yunming XiaoMatryoshka Representation Learning Embedding Compression
The Chinese University of Hong Kong, Shenzhen
Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval
Aug 30, 2026Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar +2Hierarchical Representation Learning Matryoshka Representation Learning
IIT Delhi · IBM
3D-MRL: Nested Multimodal 3D Representations via Matryoshka Representation Learning
Aug 29, 2026Márcus Lobo, Vitor Matias, Jeová Farias +13D VLMs Multimodal Pretraining
Instituto de Ciências Matemáticas e de Computação Universidade de São Paulo São Carlos, São Paulo, BR · Bowdoin College Brunswick, Maine, USA
CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations
Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteFigurative Language Understanding Arabic NLP
Department of Computer Engineering Autonomous University of Madrid (UAM) School of Science and Technology IE University Madrid, Spain · Department of Computer Science Oviedo University Asturias, Spain
Lychee-Reranker: Fast but Not Late Interaction for Compressed Document Reranking
Jun 22, 2026Xinping Zhao, Jiaxin Xu, Ziqi Dai +9Representation Learning Learning to Rank
Harbin Institute of Technology (Shenzhen) · Shenzhen Loop Area Institute · Soochow University
MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework
Jun 3, 2026Haowen Xiang, Yibo Yan, Jiahao Huo +4Representation Learning Matryoshka Representation Learning
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology
m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
May 19, 2026Yaoxiang Wang, Simiao Zuo, Qingguo Hu +4Language Model Pretraining Representation Learning
Xiamen University Xiamen, China · Microsoft · Microsoft Redmond, USA +2
GraSP-VL: Length as a Semantic Granularity Interface for Vision-Language Representations
May 18, 2026Zesheng Li, Chengchang Pan, Honggang QiVision-Language Models Representation Learning
University of the Chinese Academy of Sciences, China
To MRL or not to MRL: Text Embeddings are Robust to Truncation Without Matryoshka Learning, Except In Heavy Truncation Scenarios
May 15, 2026Sotaro Takeshita, Yurina Takeshita, Simone Paolo Ponzetto +1Text Embeddings Representation Learning
Data and Web Science Group, University of Mannheim, Germany · Independent Researcher · NEC Laboratories Europe, Heidelberg, Germany
ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
May 14, 2026Ziyin Zhang, Zihan Liao, Hang Yu +2Multilingual Language Models Text Embeddings
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Ant Group, Hangzhou, China.
Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learning
May 9, 2026Arghamitra Talukder, Philippe Chlenski, Itsik Pe'erRepresentation Learning Representation Identifiability
Computer Science, Columbia University
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining
Apr 27, 2026Phung Gia Huy, Hai An Vu, Minh-Phuc Truong +4Representation Learning Sentence Embeddings
1Hanoi University of Science and Technology · University of Oregon · 3Monash University
Matryoshka Gaussian Splatting
Mar 19, 2026Zhilin Guo, Boqiao Zhang, Hakan Aktas +113D Gaussian Splatting 3DGS Acceleration
University of Cambridge · Chalmers University of Technology · Google
Reverse Distillation: Consistently Scaling Protein Language Model Representations
Mar 8, 2026Darius Catrina, Christian Bepler, Samuel Sledzieski +1Protein Language Models Protein Representation Learning
Department of Computer Science, Duke University, Durham, NC, USA · Depts of Biostats. and Bioinformatics; and Cell Biology, Duke University, Durham, NC, USA
Cross-lingual Matryoshka Representation Learning across Speech and Text
Feb 23, 2026Yaya Sy, Dioula Doucouré, Christophe Cerisara +1Representation Learning Audio-Text Retrieval
LORIA, CNRS · Soynade Research

Sep 7, 2026J/K move · Enter open · S save

Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval

Peichun Hua, Yunming Xiao

The Chinese University of Hong Kong, Shenzhen

Matryoshka Representation Learning Embedding Compression Dense Retrieval

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) depends on dense retrieval: each document is stored as a learned vector, and a query is answered by finding its nearest neighbors in that vector space. Keeping one full-precision vector per document is the dominant index cost at corpus scale, so retrieval systems replace each vector with a short code of a few bytes---a step called quantization. Standard quantizers such as product quantization (PQ) pick the code that reconstructs the original vector most closely. A single code is even more useful if it serves several byte budgets at once: when its short prefixes are each directly searchable, a deployment can set its efficiency--quality operating point without re-encoding the corpus. But training all prefixes under one objective makes the early bits a compromise across budgets---short codes improve while the full-width code degrades. Quantization to low-bit representation, such as binary codes, further sharpens the conflict. We introduce Matryoshka Hash Representations (MHR), a two-stage procedure that separates full-width training from prefix organization. MHR first learns a longer binary code, then freezes the model and trains additional zero-initialized residual code adaptors for directly searchable prefixes. Documents are stored at one bit per coordinate, while queries keep continuous logits like PQ to attain sufficient expressivity. We implement the search process with FAISS FastScan. Trained on MS MARCO and zero-shot transferred to seven BEIR datasets, MHR reaches .5561 NDCG@10 and .6535 Recall@100 at 32 bytes, surpassing the best baseline of the same budget. The advantage is more pronounced in lower budgets. The same code also strengthens two common pipelines: shortlisting candidates for full-precision reranking, and pruning a low-storage graph index such as LEANN.