Matryoshka Representation Learning

Also known as MRL

Matryoshka Representation Learning organizes a representation so that prefixes of different lengths retain useful information, letting one model serve multiple storage, speed, and accuracy needs. Common approaches train all prefixes with task or contrastive objectives, or use distillation and ordered transformations to coordinate information across dimensions, model layers, and modalities. The approach is used in text and multilingual retrieval, vision-language and document search, and other settings where benchmarks measure quality at several embedding sizes. A central tension is whether specialized training meaningfully improves heavily compressed representations, since ordinary embeddings can often be truncated with little loss at modest compression.

Latest papers

17 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Peeling the Matryoshka: Extracting Ontological Structure from Retrieval Embeddings
Oct 30, 2026Eunchan Kim, Inwhee JoeTaxonomy Induction Hierarchical Representation Learning
Hanyang University
SpecMRL: Redesign Matryoshka Representation Learning within Parametric Spectrum
Oct 19, 2026Zhongxin Chen, Songhao Wu, Heng Cui +3Representation Learning Matryoshka Representation Learning
Gaoling School of Artificial Intelligence, Renmin University of China · School of Computer Science, Wuhan University · Lenovo Group
Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval
Sep 7, 2026Peichun Hua, Yunming XiaoMatryoshka Representation Learning Embedding Compression
The Chinese University of Hong Kong, Shenzhen
Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval
Aug 30, 2026Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar +2Hierarchical Representation Learning Matryoshka Representation Learning
IIT Delhi · IBM
3D-MRL: Nested Multimodal 3D Representations via Matryoshka Representation Learning
Aug 29, 2026Márcus Lobo, Vitor Matias, Jeová Farias +13D VLMs Multimodal Pretraining
Instituto de Ciências Matemáticas e de Computação Universidade de São Paulo São Carlos, São Paulo, BR · Bowdoin College Brunswick, Maine, USA
CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations
Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteRepresentation Geometry in Language Models Arabic NLP
Department of Computer Engineering Autonomous University of Madrid (UAM) School of Science and Technology IE University Madrid, Spain · Department of Computer Science Oviedo University Asturias, Spain
Lychee-Reranker: Fast but Not Late Interaction for Compressed Document Reranking
Jun 22, 2026Xinping Zhao, Jiaxin Xu, Ziqi Dai +9Representation Learning Learning to Rank
Harbin Institute of Technology (Shenzhen) · Shenzhen Loop Area Institute · Soochow University
MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework
Jun 3, 2026Haowen Xiang, Yibo Yan, Jiahao Huo +4Representation Learning Matryoshka Representation Learning
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology
m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
May 19, 2026Yaoxiang Wang, Simiao Zuo, Qingguo Hu +4Language Model Pretraining Representation Learning
Xiamen University Xiamen, China · Microsoft · Microsoft Redmond, USA +2
GraSP-VL: Length as a Semantic Granularity Interface for Vision-Language Representations
May 18, 2026Zesheng Li, Chengchang Pan, Honggang QiVision-Language Models Representation Learning
University of the Chinese Academy of Sciences, China
To MRL or not to MRL: Text Embeddings are Robust to Truncation Without Matryoshka Learning, Except In Heavy Truncation Scenarios
May 15, 2026Sotaro Takeshita, Yurina Takeshita, Simone Paolo Ponzetto +1Text Embeddings Representation Learning
Data and Web Science Group, University of Mannheim, Germany · Independent Researcher · NEC Laboratories Europe, Heidelberg, Germany
ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
May 14, 2026Ziyin Zhang, Zihan Liao, Hang Yu +2Multilingual Language Models Text Embeddings
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Ant Group, Hangzhou, China.
Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learning
May 9, 2026Arghamitra Talukder, Philippe Chlenski, Itsik Pe'erRepresentation Learning Representation Identifiability
Computer Science, Columbia University
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining
Apr 27, 2026Phung Gia Huy, Hai An Vu, Minh-Phuc Truong +4Representation Learning Sentence Embeddings
1Hanoi University of Science and Technology · University of Oregon · 3Monash University
Matryoshka Gaussian Splatting
Mar 19, 2026Zhilin Guo, Boqiao Zhang, Hakan Aktas +113D Gaussian Splatting 3DGS Training
University of Cambridge · Chalmers University of Technology · Google
Reverse Distillation: Consistently Scaling Protein Language Model Representations
Mar 8, 2026Darius Catrina, Christian Bepler, Samuel Sledzieski +1Protein Language Models Protein Representation Learning
Department of Computer Science, Duke University, Durham, NC, USA · Depts of Biostats. and Bioinformatics; and Cell Biology, Duke University, Durham, NC, USA
Cross-lingual Matryoshka Representation Learning across Speech and Text
Feb 23, 2026Yaya Sy, Dioula Doucouré, Christophe Cerisara +1Representation Learning Audio-Text Retrieval
LORIA, CNRS · Soynade Research

Oct 30, 2026J/K move · Enter open · S save

Peeling the Matryoshka: Extracting Ontological Structure from Retrieval Embeddings

Eunchan Kim, Inwhee Joe

Hanyang University

Taxonomy Induction Hierarchical Representation Learning Text Embedding Models Matryoshka Representation Learning

Open paper AskPDF arXiv

Taxonomy construction is essential for knowledge-intensive NLP yet remains costly: clustering-based methods (i) need careful per-corpus tuning and yield unstable hierarchies, while LLM-based pipelines (ii) depend on expensive task-specific training that does not transfer across domains. We report an empirical finding that bears on both: in Matryoshka Representation Learning (MRL) embeddings, where each leading prefix is itself a usable, coarser vector, prefix length aligns with the coarse-to-fine granularity of gold taxonomies; shuffled- and reverse-dimension controls isolate the alignment to the learned early prefix dimensions. The pattern holds across three architecturally distinct MRL models. To test whether this regularity is usable, we introduce PEEL (Peeling Embeddings to Extract Latent ontologies): a minimal probe over these frozen embeddings, trained once with an order-embedding objective that represents hierarchy as vector containment; a minimum spanning arborescence then decodes taxonomic DAGs. Using only a concept vocabulary (no documents), PEEL recovers taxonomic DAGs with high structural fidelity on two OLLM benchmarks, transferring to a new domain with no target-edge supervision. The end-to-end OLLM pipeline, which jointly discovers concepts and induces structure from document corpora, is reported as a cross-task reference. Pre-trained retrieval embeddings encode a decodable granularity gradient; a minimal probe renders it as explicit parent-child (is-a) structure.