Text Embedding Evaluation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 60

All topics

Your Prompt Should Do More: Effects of Retrieval Instructions in Embedding Models
Oct 7, 2026Amanda Myntti, Jenna Kanerva, Veronika Laippala +1Text Embedding Models Information Retrieval
TurkuNLP University of Turku, Finland · Ellis Institute Finland
A Safe Prototype Is Not a Safety Direction: Reference Dependence and Prompt Confounds in Response-Safety Embeddings
Oct 1, 2026Sahil KadadekarPrototype-Based Classification LLM Evaluation
New York University
Embedding Models Measure in Peculiar Ways
Sep 17, 2026Juri Opitz, Andrianos MichailText Embeddings Text Embedding Evaluation
Dept. of Computational Linguistics University of Zurich
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
Sep 14, 2026Hyojung HanText Embedding Evaluation Weight-Only Quantization
ThakiCloud
ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrieval
Sep 1, 2026Aaryan Kapoor, Md Abdullah Al Hafiz KhanText Embedding Evaluation Information Retrieval
Kennesaw State University
Do General NLP Embeddings Capture Ontological Reasoning?
Aug 31, 2026Hamed Babaei Giglou, Jennifer D'Souza, Sören AuerLogical Reasoning Text Embedding Evaluation
TIB Leibniz Information Centre for Science and Technology · Hannover, Lower Saxony, Germany · L3S Research Center, Leibniz University of Hannover
The Embedder's Dilemma: LLMs Are Better, but at What Cost?
Aug 13, 2026Adnan El Assadi, Niklas Muennighoff, Jinhyuk LeeLLM Inference Efficiency LLM Evaluation
Harvard University · Stanford University · Independent Researcher
Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems
Aug 10, 2026Scott E. FriasSemantic Textual Similarity Text Embedding Evaluation
Eigenforma · Freemind Labs
Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests
Aug 7, 2026Seitaro Ono, Senna Ross, Jun SaikiNeural Representation Geometry Text Embedding Evaluation
Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan · Brock University, St. Catharines, Ontario, Canada
How Far Do Simple Transformations Translate Across Text Embedding Models?
Aug 6, 2026Sid Ali Hamideche, Louis-Adrien Dufrène, Quentin Lampin +1Representation Learning Text Embedding Evaluation
Orange Research · Orange
Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing
Aug 6, 2026Marcin Rozmus, Peter van der PuttenText Embedding Evaluation Text Embedding Models
GenAI Engineering, Pegasystems, Kraków, Poland · AI Lab, Pegasystems, Amsterdam, The Netherlands · LIACS, Leiden University, Leiden, the Netherlands
Do Static Embeddings Add Value to Hybrid Dutch Retrieval?
Aug 3, 2026António Pereira BarataText Embeddings Word Embeddings
Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework
Jul 26, 2026Madhav S BaidyaModel Selection Text Embedding Evaluation
Indian Institute of Technology (BHU) Varanasi
A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset
Jul 23, 2026Katerina Papantoniou, Panagiotis Papadakos, Theodore Patkos +3Text Embedding Evaluation Multilingual IR
ICS-FORTH Heraklion, Greece · Crete University Press Heraklion, Greece · University of Crete, Computer Science Department, Heraklion, Greece
Multilingual Sentence Embeddings for Linguistic-Integrated Reliability Audit
Jul 20, 2026Ummugul Bezirhan, Ji Yoon Jung, Matthias von DavierSentence Embeddings Educational Assessment
Boston College Chestnut Hill, Massachusetts, USA
From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings
Jul 8, 2026Shi-Ting Chen, Jinsong ChenText Embedding Evaluation Benchmark Validity
Faculty of Education, The University of Hong Kong
MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese
Jul 6, 2026Tardelli Ronan Coelho StekelText Embedding Evaluation Text Embedding Models
Federal Institute of S˜ao Paulo (IFSP) S˜ao Paulo, Brazil
Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders
Jul 5, 2026Lucas Hideki Takeuchi Okamura, Alexandre Alcoforado, Anna Helena Reali CostaSentence Embeddings Text Embedding Evaluation
Escola Politécnica, Universidade de São Paulo, São Paulo, Brazil
ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs
Jun 30, 2026Andrianos Michail, Stylianos Psychias, Michelle Wastl +3Text Embedding Evaluation Low-Resource Language Processing
Department of Computational Linguistics University of Zurich
STEB: Style Text Embedding Benchmark
Jun 30, 2026Rafael Rivera Soto, Anna Wegmann, Cristina AggazzottiText Embeddings Text Embedding Evaluation
1Johns Hopkins University · 2Utrecht University
SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics
Jun 29, 2026Nikolay Georgiev, Maria Drencheva, Kseniia Ibragimova +3Text Embedding Evaluation LLM Reranking
INSAIT, Sofia University “St. Kliment Ohridski” · ETH Zurich
Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
Jun 28, 2026V. S. Raghu ParupudiSemantic Textual Similarity Neural Representation Geometry
University of California, San Diego
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories
Jun 27, 2026Fabio Ciani, Harald Schweiger, Emilia Parada-Cabaleiro +1Text Embeddings Text Embedding Evaluation
Department of Electronics, Information, and Bioengineering, Polytechnic University of Milan · Multimedia Mining and Search Group, Institute of Computational Perception, Johannes Kepler University Linz · Department of Music Pedagogy, Nuremberg University of Music +1
Does My Embedding Reflect That $A = B$ ? Evaluating Mathematical Equivalence in Embedding Models
Jun 22, 2026Jiaying Ye, Samarth Rao, Leo Carlin +9Contrastive Learning Text Embedding Evaluation
University of Washington · Axiom Math, University of Washington · Pacific Northwest National Laboratory
Do LLM Embedding Spaces Recover Expert Structure?
Jun 22, 2026Yixuan Zhu, Zhenke Duan, Fanghen LiLLM Fine-Tuning Mental Health
Zhongnan University of Economics and Law
HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions
Jun 22, 2026Yuichi TatenoBenchmark Design Text Embedding Evaluation
Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment
Jun 19, 2026Aaron Marker, Oscar Kjell, Vasudha Varadarajan +1Sentence Embeddings Mental Health
Vanderbilt University · Lund University · Carnegie Mellon University
Examining the Limits of Word2Vec with Toki Pona
Jun 15, 2026Daniel Zhenhan Huang, Hongchen WuText Embedding Evaluation Distributional Semantics
Georgia Institute of Technology Atlanta, GA
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
Jun 11, 2026Marek Šuppa, Andrej Ridzik, Daniel Hládek +2Text Embeddings Word Embeddings
Comenius University in Bratislava, Slovakia · Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · Technical University of Košice, Slovakia +1
Modeling semantic association in self-paced reading with language model embeddings
Jun 5, 2026Sara Møller Østergaard, Kenneth Enevoldsen, Afra Alishahi +1Sentence Embeddings Text Embedding Evaluation
Department of Computational Cognitive Science, Tilburg University · Center for Humanities Computing, Aarhus University

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Your Prompt Should Do More: Effects of Retrieval Instructions in Embedding Models

Amanda Myntti, Jenna Kanerva, Veronika Laippala, Filip Ginter

TurkuNLP University of Turku, Finland · Ellis Institute Finland

Text Embedding Models Information Retrieval Text Embedding Evaluation

Open paper AskPDF arXiv

Prompted embedding models have recently received increasing attention, particularly for retrieval, where detailed retrieval instructions are provided as part of the retrieval prompt. Several new datasets and studies have examined this setting, showing that the current embedding models often struggle to follow such instructions reliably. In this paper, we study the mechanism of how instructions actually affect the representations of retrieval queries in asymmetric retrieval tasks. We show that models can fail to follow even simple task instructions when query-side distractors are included in the evaluation. We hypothesize that this behavior is driven by the training setup of current embedding models and their evaluation, and show that fine-tuning with added query-side distractors leads to substantial improvements, with minimal effect on other tasks.

Momentum

Related topics

Latest papers 60