Text Embeddings

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

Lend Me Your Eyes: Instruction-Aware Text Embeddings via Attention Relay
Oct 4, 2026Yiyuan Luo, Vaggos ChatziafratisText Embeddings Attention Mechanisms
University of California, Santa Cruz
Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction
Sep 30, 2026Grayson Wycliffe Storer, Julia Witte ZimmermanText Embeddings Language Modeling
Department of Mathematics and Statistics. · Department of Computer Science, University of Vermont, Burlington, 05405, Vermont, United States. · Computational Story Lab. +1
Doc2LoRA Provides Decodable Representations of Scientific Ideas
Sep 29, 2026Chand Sahil Mansuri, Joel Zachariah, Sadamori KojakuText Embeddings Low-Rank Adaptation
School of Systems Science and Industrial Engineering Binghamton University, Binghamton, NY, USA
Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings
Sep 29, 2026Zifeng Cheng, Jie Zheng, Zhiwei Jiang +4Text Embeddings Text Embedding Models
State Key Laboratory for Novel Software Technology, Nanjing University · National University of Singapore · Nanjing University of Posts and Telecommunications
Embedding Models Measure in Peculiar Ways
Sep 17, 2026Juri Opitz, Andrianos MichailText Embeddings Text Embedding Evaluation
Dept. of Computational Linguistics University of Zurich
Beyond frequency measures: Can contextual embeddings capture meaning change in scientific texts?
Sep 16, 2026Jianying Liu, Kim Gerdes, Jean-Marc DeltornText Embeddings Semantic Change Detection
STL, BETA, CEIPI · LISN, Université Paris-Saclay; CEIPI, Université de Strasbourg · LISN, Qatent, STL +1
Multi-Functional Embedding Models for Funder Name Disambiguation in Scientific Publication Records
Sep 9, 2026Kanyao Han, Zhiwen You, Jinseok Kim +1Text Embeddings Entity Resolution
University of Illinois at Urbana-Champaign, Champaign, USA · University of Michigan, Ann Arbor, USA · Technical University of Munich, Munich, Germany +2
Reading a Legal Question Word by Word: Embedding Trajectories of 2,144 Vietnamese Legal Headlines
Sep 8, 2026Tran Minh QuanText Embeddings Word Embeddings
When Can We Work in Embedding Space? What Text Embeddings Preserve
Aug 31, 2026Simon FreyaldenhovenText Embeddings Latent Variable Models
Federal Reserve Bank of Philadelphia
Task-Conditional Flow Matching for Balanced Multilingual Text Embedding Adaptation
Aug 6, 2026Tirth Bhatt, Naren Kumar S, Mayank SinghFlow Matching Text Embeddings
LINGO Research Group, Indian Institute of Technology Gandhinagar, India
Do Static Embeddings Add Value to Hybrid Dutch Retrieval?
Aug 3, 2026António Pereira BarataText Embeddings Word Embeddings
An empirical investigation into the properties of standard word embeddings
Jul 26, 2026Salomon KabongoText Embeddings Word Embeddings
African Institute for Mathematical Sciences (AIMS)
KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback
Jul 17, 2026Yan Zhu, Y. Chen, Rebecca FaustText Embeddings Text Embedding Models
Yan Zhu is with Tulane University. · Y. Chen is with Tulane University. · Rebecca Faust* is with Tulane University.
Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization
Jul 1, 2026Dain Kim, Jinseo Kim, Sungyong BaikDiffusion Model Guidance Text Embeddings
Dept. of Artificial Intelligence, Hanyang University, South Korea · Dept. of Data Science, Hanyang University, South Korea
STEB: Style Text Embedding Benchmark
Jun 30, 2026Rafael Rivera Soto, Anna Wegmann, Cristina AggazzottiText Embeddings Text Embedding Evaluation
1Johns Hopkins University · 2Utrecht University
How Can AI Find My Model? A Model-Finding Experimental Study Considering Data Formats, Embeddings, and Retrieval Strategies
Jun 29, 2026Jhon G. Botello, Jose J. Padilla, Erika Frydenlund +2Text Embeddings Learning to Rank
Dept. of Computer Science, Old Dominion University, Norfolk, VA, USA · Virginia Modeling, Analysis, and Simulation Center, Old Dominion University, Norfolk, VA, USA · Virginia Digital Maritime Center, Old Dominion University, Norfolk, VA, USA +1
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories
Jun 27, 2026Fabio Ciani, Harald Schweiger, Emilia Parada-Cabaleiro +1Text Embeddings Text Embedding Evaluation
Department of Electronics, Information, and Bioengineering, Polytechnic University of Milan · Multimedia Mining and Search Group, Institute of Computational Perception, Johannes Kepler University Linz · Department of Music Pedagogy, Nuremberg University of Music +1
EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory
Jun 19, 2026Chang Nie, Chaoyou Fu, Junlan Feng +1Long-Context Retrieval Text Embeddings
Nanjing University
Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders
Jun 19, 2026Wonseok Shin, Songkuk KimText Embeddings Sentence Embeddings
Yonsei University
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
Jun 11, 2026Marek Šuppa, Andrej Ridzik, Daniel Hládek +2Text Embeddings Word Embeddings
Comenius University in Bratislava, Slovakia · Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · Technical University of Košice, Slovakia +1
STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model
Jun 9, 2026Hailan Zhang, Haipeng Liu, Bo Fu +1Text Embeddings Text-Image Alignment
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · School of Computer and Artificial Intelligence, Liaoning Normal University, Dalian, China
Tensorizing Engram: Sharing Latents Across N-Gram Embeddings is Beneficial in LLMs
Jun 6, 2026Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides +3Text Embeddings LLM Compression
Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom · AIP, RIKEN, Tokyo, Japan.
Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings
Jun 5, 2026Songhao Wu, Zhongxin Chen, Yuxuan Liu +3Text Embeddings Text Embedding Models
Gaoling School of Artificial Intelligence, Renmin University of China Beijing, China · Lenovo Group Limited Beijing, China · Wuhan University Wuhan, China
ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs
Jun 4, 2026Ailiang Lin, Zhuoyun Li, Yusong Wang +3Text Embeddings Decoder-Only Language Models
Institute of Science Tokyo · Tencent
Text-to-Image Models Need Less from Text Encoders Than You Think
Jun 2, 2026Nurit Spingarn, Noa Cohen, Tamar Rott Shaham +1Text Embeddings Conditional Image Generation
Technion – Israel Institute of Technology · MIT CSAIL
SEA-LION-Embedding: Open and Reproducible Text Embeddings for Southeast Asia
Jun 2, 2026Peerat Limkonchotiwat, Raymond Ng, Sarana Nutanong +1Text Embeddings Text Embedding Evaluation
AI Singapore · School of Information Science and Technology, VISTEC
The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness
May 27, 2026Manuel Frank, Haithem AfliText Embeddings Text Embedding Evaluation
Department of Computer Science Munster Technological University
PromptEmbedder: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting
May 27, 2026Yu-Che Tsai, Kuan-Yu Chen, Yuan-Hao Chen +4Text Embeddings Prompt Tuning
Department of Computer Science and Information Engineering, National Taiwan University · National Taiwan University AI Center of Research Excellence Taipei, Taiwan
A graph-based analysis of semantic types and coercion in contextualized word embeddings
May 22, 2026Long Chen, Deniz Ekin YavasText Embeddings Text Embedding Evaluation
Heinrich Heine University Düsseldorf
To MRL or not to MRL: Text Embeddings are Robust to Truncation Without Matryoshka Learning, Except In Heavy Truncation Scenarios
May 15, 2026Sotaro Takeshita, Yurina Takeshita, Simone Paolo Ponzetto +1Text Embeddings Representation Learning
Data and Web Science Group, University of Mannheim, Germany · Independent Researcher · NEC Laboratories Europe, Heidelberg, Germany
ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
May 14, 2026Ziyin Zhang, Zihan Liao, Hang Yu +2Multilingual Language Models Text Embeddings
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Ant Group, Hangzhou, China.
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
May 12, 2026Jingzhou Jiang, Yi Yang, Kar Yan TamText Embeddings LLM Pruning
The Hong Kong University of Science and Technology
Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings
May 11, 2026Benjamin Icard, Lila Sainero, Alice Breton +2LLM Evaluation Text Embeddings
LIP6, Sorbonne University, CNRS, France · Infopro Digital, France
Embeddings for Preferences, Not Semantics
May 8, 2026Carter Blair, Ariel D. Procaccia, Milind TambeText Embeddings Word Embeddings
Harvard University
The Proxy Presumption: From Semantic Embeddings to Valid Social Measures
May 8, 2026Baishi Li, Ta Yu, Kelvin J. L. Koa +1Text Embeddings Text Embedding Evaluation
National University of Singapore · Asian Institute of Digital Finance
LLM-Augmented Semantic Steering of Text Embedding Projection Spaces
May 3, 2026Wei Liu, Eric Krokos, Kirsten Whitley +2Text Embeddings Language Model Steering
Virginia Tech Blacksburg, Virginia, USA · Department of Defense Washington, DC, USA · Tulane University New Orleans, Louisiana, USA
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders
May 2, 2026Ailiang Lin, Zhuoyun Li, Keyu Mao +2Text Embeddings In-Context Learning
Institute of Science Tokyo · Tencent
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings
Apr 30, 2026Tomomasa Hara, Hiroto Kurita, Masaaki Imaizumi +2Text Embeddings Text Embedding Evaluation
Tohoku University · The University of Tokyo · Kyoto University +3
ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models
Apr 27, 2026Orhan Demirci, Sezer Aptourachman, Aydın KayaText Embeddings Word Embeddings
Department of Computer Engineering Hacettepe University Ankara, Turkey
Semantic Entanglement in Vector-Based Retrieval: A Formal Framework and Context-Conditioned Disentanglement Pipeline for Agentic RAG Systems
Apr 20, 2026Nick LoghmaniDocument Chunking Text Embeddings
Salesforce · School of Information Studies, Syracuse University
REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning
Apr 19, 2026Seungmin Lee, Jeonghwan Lee, Hyunkuk Lim +2Text Embeddings Representation Learning
PwC Korea GenAI Team, Seoul, South Korea
Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text
Mar 2, 2026Yixuan Tang, Zhenghong Lin, Yandong Sun +3Text Embeddings Representation Learning
School of Computing, National University of Singapore
NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance
Jul 13, 2025Hanwool Lee, Sara Yu, Yewon Hwang +4Multilingual Language Models Text Embeddings
FinancialNLPLab, MODULABS Shinhan Securities Seoul, Republic of Korea · FinancialNLPLab, MODULABS KT Seoul, Republic of Korea · FinancialNLPLab, MODULABS EMRO Seoul, Republic of Korea +4
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Jun 2, 2025Yimin DuText Embeddings Word Embeddings
Independent Researcher Beijing, China

Oct 4, 2026, cs.CLJ/K move · Enter open · S save

Lend Me Your Eyes: Instruction-Aware Text Embeddings via Attention Relay

Yiyuan Luo, Vaggos Chatziafratis

University of California, Santa Cruz

Text Embeddings Attention Mechanisms Text Embedding Models

Open paper AskPDF arXiv

Text embedding models trained with contrastive learning learn to follow task instructions from instruction-paired data, while instruction-tuned LLMs already know how to follow them. We show that this instruction-following ability can carry over from an LLM to a Transformer-based embedder without any training. We propose Attention Relay, which passes the attention weights an LLM produces to the embedder's own attention. Across six instruction-tuned LLMs from the Qwen3, Llama 3.1 and OLMo 3 families and ten widely used embedding models that differ in tokenizer, size and pooling type, Attention Relay makes nearly every combination instruction-aware. Experiments that break the method down into its parts show that the LLM's attention weights track the instruction in its later layers and come largely from instruction tuning. They also show that relaying these weights selects which content in the text matters: it makes the aspect of the text that the instruction asks about dominant in the embedding, or restores that aspect where averaging had diluted it.

Momentum

Related topics

Latest papers 44