Overview

Word embeddings represent words or tokens as vectors, allowing systems to compare meanings, retrieve related text, and use language features in downstream tasks. Static methods such as Word2Vec, GloVe, and FastText learn from word co-occurrences, while contextual transformers produce representations that vary with surrounding text; character and morphology-aware methods help with rare words and complex languages. Researchers study these vectors through analogy, similarity, retrieval, and linguistic-probing benchmarks, as well as their geometry, bias, memory use, and behavior during training. A central tension is that compact or stable vectors can support efficient comparison, but may miss context-dependent meaning or preserve unwanted associations; improvements on one benchmark do not always transfer to other tasks.

Latest papers

CardsList

43 papers3 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. What Does Each Part of Speech Need? Contexts, Subspaces and Dimensions in Static Word Embeddings

    Nov 11, 2026Jintang Xue, Chengwei Wei, Yun-Cheng Wang +1Word EmbeddingsEmbedding Compression

  2. StaRSE: Compact Russian Sentence Embeddings with a Sign-Coded Static Encoder

    Nov 10, 2026Boris Malashenko, Ivan Jarsky, Egor Soldatov +6Sentence EmbeddingsBinary Quantization

  3. A KAN-based Non-linear Factorization for Embedding Layer Compression

    Oct 21, 2026Oliver Giudice, Anna Russo RussoEmbedding CompressionLow-Rank Compression

  4. Scaling and Distilling Text Embeddings for Better Diffusibility

    Oct 1, 2026Zekai Zhang, Yunjie Tian, Yanjin He +4Word EmbeddingsLanguage Model Distillation

  5. Comparative Evaluation of Static Embedding Models for HTTP Request Anomaly Detection

    Sep 22, 2026Amanda Riverol, Gustavo Betarte, Rodrigo Martínez +1Anomaly DetectionUnsupervised Anomaly Detection

  6. Analysing the Linearity of Linguistic Relations in Language Model Embedding Spaces

    Sep 18, 2026Vasudevan Nedumpozhimana, Fathima Thekkekara, John KelleherRepresentation Geometry in Language ModelsWord Embeddings

  7. Generalization through Lexical Abstraction in Transformer Models: The Case of Functional Words

    Sep 17, 2026Giuseppe Samo, Vivi Nastase, Paola MerloTransformer InterpretabilityWord Embeddings

  8. Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization

    Aug 31, 2026Junjie Yao, Liangkai Hang, Zhi-Qin John XuLanguage Model PretrainingWord Embeddings

  9. Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings

    Aug 6, 2026Sriram Loganathan, Gokul Anand, Aung Bo Bo +2Word EmbeddingsDistributional Semantics

  10. An empirical investigation into the properties of standard word embeddings

    Jul 26, 2026Salomon KabongoText EmbeddingsWord Embeddings

  11. CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations

    Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteRepresentation Geometry in Language ModelsArabic NLP

  12. ROMEVA: Geometry-Preserving Vocabulary Expansion for Roman Urdu Language Models

    Jun 21, 2026Mahnoor Khan, Afsheen Asif, Milhan Afzal Khan +2Multilingual Language ModelsWord Embeddings

  13. SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation

    Jun 11, 2026Marek Šuppa, Andrej Ridzik, Daniel Hládek +2Text EmbeddingsWord Embeddings

  14. What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings

    Jun 6, 2026Alexey Kresin, Tchifou M. Dieffi, Tomer CaspiNeural Representation GeometryWord Embeddings

  15. Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence

    May 22, 2026Andres Nava, Matthieu WyartNeural Representation GeometryWord Embeddings

  16. Domain-Independent Game Abstraction using Word Embedding Techniques

    May 15, 2026Juho Kim, Tuomas SandholmWord Embeddings

  17. Embeddings for Preferences, Not Semantics

    May 8, 2026Carter Blair, Ariel D. Procaccia, Milind TambeText EmbeddingsWord Embeddings

  18. FastOmniTMAE: Parallel Clause Learning for Scalable and Hardware-Efficient Tsetlin Embeddings

    May 7, 2026Ahmed K. Kadhim, Lei Jiao, Rishad Shafik +2Word EmbeddingsText Embedding Models

  19. Test-Time Safety Alignment

    Apr 28, 2026Baturay Saglam, Dionysis KalogeriasWord EmbeddingsLLM Safety Alignment

  20. ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models

    Apr 27, 2026Orhan Demirci, Sezer Aptourachman, Aydın KayaText EmbeddingsWord Embeddings

  21. ORPHEAS: A Cross-Lingual Greek-English Embedding Model for Retrieval-Augmented Generation

    Apr 22, 2026Ioannis E. Livieris, Athanasios Koursaris, Alexandra Apostolopoulou +2Retrieval-Augmented GenerationWord Embeddings