Text Embedding Evaluation

Ask about this topic

Overview

Text embedding evaluation studies whether vectors produced by language models preserve the distinctions needed for tasks such as semantic similarity, search, clustering, and classification. Researchers use broad suites such as MTEB and BEIR alongside domain- and language-specific benchmarks, controlled contrastive tests, and analyses of embedding geometry and robustness. Results show that model rankings often vary with task, language, prompts, data, and similarity metric, so performance on one benchmark may not predict performance elsewhere. A central challenge is ensuring that benchmark scores reflect meaningful distinctions and real deployment needs, including latency, cost, and the value of combining dense embeddings with lexical retrieval.

Latest papers

CardsList

78 papers7 papers in the last four weeks, up 40% on the four weeks before. 0.0% of all new papers.

  1. Semantically Rich, Metrically Affect-Blind: A Geometric Blind Spot in Modern Text Embedders

    Oct 23, 2026Mahdi Naser MoghadasiSentence EmbeddingsRepresentation Geometry in Language Models

    BrightMind AI Seattle, WA, USA

  2. When PTQ Calibration Helps or Hurts: Dose-Driven Collapse and Validation-Gated Abstention in Quantized Text Embedders

    Oct 19, 2026Hyojung Han, Ryangkyung Kang, Youngeun Kim +1Post-Training QuantizationNeural Network Quantization

    ThakiCloud · Korea University

  3. TRACE: Training-aware Robustness and Challenge Evaluation for Text-Vision Embedding Models

    Oct 19, 2026Pume Tuchinda, Tinnakit Udsa, Peerat LimkonchotiwatVLM EvaluationVLM Robustness

    AI Singapore · School of Information Science and Technology VISTEC

  4. Beyond Black Boxes: Visual Grounding for Text Embeddings

    Oct 13, 2026Slimane LarabiInformation RetrievalText Embedding Models

    Computer Science Faculty, USTHB University

  5. Geometric Structure Predicts Per-Example Neighborhood Loss Under PCA Truncation

    Oct 13, 2026Sriram Loganathan, Gokul Anand, Ragavan ArivazhaganText Embedding EvaluationEmbedding Compression

    Department of Computer Science, San Jose State University, San Jose, CA, USA

  6. Your Prompt Should Do More: Effects of Retrieval Instructions in Embedding Models

    Oct 7, 2026Amanda Myntti, Jenna Kanerva, Veronika Laippala +1Text Embedding ModelsInformation Retrieval

    TurkuNLP University of Turku, Finland · Ellis Institute Finland

  7. A Safe Prototype Is Not a Safety Direction: Reference Dependence and Prompt Confounds in Response-Safety Embeddings

    Oct 1, 2026Sahil KadadekarPrototype-Based ClassificationLLM Evaluation

    New York University

  8. Semantic Candidate-Job Matching: A Comparative Evaluation of Dense Embedding Models in Hybrid Retrieval

    Sep 20, 2026Sai Yashwant, Siddhartha Jain, Anurag Dubey +2Job-Candidate MatchingText Embedding Models

    ManpowerGroup Services India Pvt. Ltd. · Indian Institute of Technology, Hyderabad

  9. Embedding Models Measure in Peculiar Ways

    Sep 17, 2026Juri Opitz, Andrianos MichailText EmbeddingsText Embedding Evaluation

    Dept. of Computational Linguistics University of Zurich

  10. Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families

    Sep 14, 2026Hyojung HanText Embedding EvaluationWeight-Only Quantization

    ThakiCloud

  11. ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrieval

    Sep 1, 2026Aaryan Kapoor, Md Abdullah Al Hafiz KhanText Embedding EvaluationInformation Retrieval

    Kennesaw State University

  12. Do General NLP Embeddings Capture Ontological Reasoning?

    Aug 31, 2026Hamed Babaei Giglou, Jennifer D'Souza, Sören AuerLogical ReasoningText Embedding Evaluation

    TIB Leibniz Information Centre for Science and Technology · Hannover, Lower Saxony, Germany · L3S Research Center, Leibniz University of Hannover

  13. Evaluating Multilingual Sentence Embeddings for Translation Error Detection:An English--Greek Contrastive Study

    Aug 28, 2026Eleftherios Kalogeros, Athanasios Ntalakas, Manolis Gergatsoulis +2MT EvaluationSentence Embeddings

    Laboratory on Digital Libraries and Electronic Publishing, Department of Information Science, Ionian University, Ioannou Theotoki 72, 49100 Corfu, Greece · Centre for Literary Translation and Book Policy, Department of Foreign Languages, Translation and Interpreting, Ionian University, Tsirigoti Sq., 49100 Corfu, Greece

  14. Embedding Models for Stance-Aware Argument Retrieval

    Aug 28, 2026Angelo Sparacino, Francesca Toni, Adam DejlComputational ArgumentationText Embedding Models

    Department of Computing, Imperial College London, UK

  15. KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda via Multi-Stage Curriculum Training

    Aug 27, 2026Ireddi Rakshitha, Devavarapu Yashwanth, Ntakirutimana PierreSentence EmbeddingsAfrican Language Processing

    Barclays · Carnegie Mellon University

  16. Dataset Scarcity Limits Robust Evaluation of Multilingual Embedding Models: A Case Study of Slavic Languages

    Aug 25, 2026Ana Gjorgjevikj, Barbara Koroušić Seljak, Tome EftimovMultilingual Language Model EvaluationText Embedding Evaluation

    Computer Systems Department Jožef Stefan Institute Ljubljana, Slovenia

  17. Comparing Chunking and Embedding Strategies for Turkish RAG Systems

    Aug 24, 2026Mustafa Sertaç Türkel, Fatma Nur Korkmaz, Ahmet Tuğrul BayrakRetrieval-Augmented GenerationRAG Evaluation

    Data Science and Innovation Ata Technology Platforms Istanbul, Turkey

  18. Lexical Coupling in GUI Element Grounding: Sentence Embeddings Track Labels across Mobile and Web

    Aug 22, 2026Qijia Chen, Giulio JacucciGUI GroundingText Embedding Evaluation

    Department of Computer Science University of Helsinki Helsinki, Finland

  19. The Commercial Tax: Rent-vs-Own Blind Spots in Multi-Hop Retrieval Benchmarks

    Aug 17, 2026Luis M. Sanchez, Kosrow DehnadDense RetrievalText Embedding Evaluation

    Toryx

  20. Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach

    Aug 15, 2026Che Shen, Junwei Su, Lingpeng Kong +1Text Embedding ModelsPrototype Learning

    Department of Computer Science The University of Hong Kong, Hong Kong, China · School of Artificial Intelligence and Data Science University of Science and Technology of China, Hefei, Anhui, China

  21. The Embedder's Dilemma: LLMs Are Better, but at What Cost?

    Aug 13, 2026Adnan El Assadi, Niklas Muennighoff, Jinhyuk LeeLLM Inference EfficiencyLLM Evaluation

    Harvard University · Stanford University · Independent Researcher

  22. Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems

    Aug 10, 2026Scott E. FriasSemantic Textual SimilarityText Embedding Evaluation

    Eigenforma · Freemind Labs

  23. Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests

    Aug 7, 2026Seitaro Ono, Senna Ross, Jun SaikiNeural Representation GeometryText Embedding Evaluation

    Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan · Brock University, St. Catharines, Ontario, Canada

  24. How Far Do Simple Transformations Translate Across Text Embedding Models?

    Aug 6, 2026Sid Ali Hamideche, Louis-Adrien Dufrène, Quentin Lampin +1Representation LearningText Embedding Evaluation

    Orange Research · Orange

  25. Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing

    Aug 6, 2026Marcin Rozmus, Peter van der PuttenText Embedding EvaluationText Embedding Models

    GenAI Engineering, Pegasystems, Kraków, Poland · AI Lab, Pegasystems, Amsterdam, The Netherlands · LIACS, Leiden University, Leiden, the Netherlands

  26. Do Static Embeddings Add Value to Hybrid Dutch Retrieval?

    Aug 3, 2026António Pereira BarataText EmbeddingsWord Embeddings

  27. Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework

    Jul 26, 2026Madhav S BaidyaModel SelectionText Embedding Evaluation

    Indian Institute of Technology (BHU) Varanasi

  28. A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

    Jul 23, 2026Katerina Papantoniou, Panagiotis Papadakos, Theodore Patkos +3Text Embedding EvaluationMultilingual IR

    ICS-FORTH Heraklion, Greece · Crete University Press Heraklion, Greece · University of Crete, Computer Science Department, Heraklion, Greece

  29. Multilingual Sentence Embeddings for Linguistic-Integrated Reliability Audit

    Jul 20, 2026Ummugul Bezirhan, Ji Yoon Jung, Matthias von DavierSentence EmbeddingsEducational Assessment

    Boston College Chestnut Hill, Massachusetts, USA

  30. From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings

    Jul 8, 2026Shi-Ting Chen, Jinsong ChenText Embedding EvaluationBenchmark Validity

    Faculty of Education, The University of Hong Kong