Word Embeddings

Follow this topic Ask about this topic

Word embeddings represent words or tokens as vectors, allowing systems to compare meanings, retrieve related text, and use language features in downstream tasks. Static methods such as Word2Vec, GloVe, and FastText learn from word co-occurrences, while contextual transformers produce representations that vary with surrounding text; character and morphology-aware methods help with rare words and complex languages. Researchers study these vectors through analogy, similarity, retrieval, and linguistic-probing benchmarks, as well as their geometry, bias, memory use, and behavior during training. A central tension is that compact or stable vectors can support efficient comparison, but may miss context-dependent meaning or preserve unwanted associations; improvements on one benchmark do not always transfer to other tasks.

Latest papers

40 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

Polish phonology and morphology through the lens of distributional semantics
Mar 31, 2026Paula Orzechowska, R. Harald BaayenDistributional Semantics Word Embeddings
Adam Mickiewicz University · University of Tübingen
A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings
Mar 25, 2026Rami LuistoRepresentation Geometry in Language Models Word Embeddings
Faculty of Information Technology, University of Jyväskylä, Jyväskylä; HUS Helsinki University Hospital, Helsinki; Digital Workforce Services, Helsinki.
LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation
Mar 23, 2026Hailay Teklehaymanot, Dren Fazlija, Wolfgang NejdlLow-Resource Language Processing Word Embeddings
L3S Research Center, Hannover, Germany
Large Language Models provide support for the parallelogram theory of analogy
Mar 19, 2026Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu +2Analogical Reasoning for Language Models Analogical Reasoning
Princeton University · The University of Hong Kong
From Word2Vec to Transformers: Text-Derived Composition Embeddings for Filtering Combinatorial Electrocatalysts
Mar 9, 2026Lei Zhang, Markus StrickerVirtual Screening Compositional Representation Learning
Interdisciplinary Centre for Advanced Materials Simulation, Ruhr University Bochum, Universitätsstraße 150, 44801 Bochum, Germany
World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models
Mar 4, 2026Elan BarenholtzLLM World Models Word Embeddings
Department of Psychology & Center for Complex Systems and Brain Sciences Florida Atlantic University
Tracing the Evolution of Word Embedding Techniques in Natural Language Processing
Feb 27, 2026Minh Anh Nguyen, Kuheli Sai, Minh NguyenWord Embeddings Text Embedding Models
Texas Tech University Lubbock, Texas · University of Pittsburgh Pittsburgh, Pennsylvania · Florida Atlantic University Boca Raton, Florida
Beyond Subtokens: A Rich Character Embedding for Low-resource and Morphologically Complex Languages
Feb 24, 2026Felix Schneider, Maria Gogolev, Sven Sickert +1Word Embeddings Low-Resource Language Processing
Computer Vision Group Friedrich Schiller University Jena
Symmetry in language statistics shapes the geometry of model representations
Feb 16, 2026Dhruva Karkada, Daniel J. Korchinski, Andres Nava +2Representation Learning Neural Representation Geometry
1UC Berkeley · 2EPFL · 3Johns Hopkins +1
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Jun 2, 2025Yimin DuText Embeddings Word Embeddings
Independent Researcher Beijing, China

Mar 31, 2026J/K move · Enter open · S save

Polish phonology and morphology through the lens of distributional semantics

Paula Orzechowska, R. Harald Baayen

Adam Mickiewicz University · University of Tübingen

Distributional Semantics Word Embeddings Computational Psycholinguistics

Open paper AskPDF arXiv

This study investigates the relationship between the phonological and morphological structure of Polish words and their meanings using Distributional Semantics. In the present analysis, we ask whether there is a relationship between the form properties of words containing consonant clusters and their meanings. Is the phonological and morphonological structure of complex words mirrored in semantic space? We address these questions for Polish, a language characterized by non-trivial morphology and an impressive inventory of morphologically-motivated consonant clusters. We use statistical and computational techniques, such as t-SNE, Linear Discriminant Analysis and Linear Discriminative Learning, and demonstrate that -- apart from encoding rich morphosyntactic information (e.g. tense, number, case) -- semantic vectors capture information on sub-lexical linguistic units such as phoneme strings. First, phonotactic complexity, morphotactic transparency, and a wide range of morphosyntactic categories available in Polish (case, gender, aspect, tense, number) can be predicted from embeddings without requiring any information about the forms of words. Second, we argue that computational modelling with the discriminative lexicon model using embeddings can provide highly accurate predictions for comprehension and production, exactly because of the existence of extensive information in semantic space that is to a considerable extent isomorphic with structure in the form space.