Word Embeddings
Word embeddings are dense vector representations of words, capturing semantic meaning and relationships within a numerical space. Current research focuses on improving embedding quality through contextualization (considering surrounding words), addressing biases, and extending their application to low-resource languages and specialized domains like medicine, using architectures such as transformers and graph convolutional networks. These advancements enhance various NLP tasks, including text classification, question answering, and information retrieval, impacting fields ranging from education to healthcare through improved accuracy and interpretability of language models.
Papers
Does mBERT understand Romansh? Evaluating word embeddings using word alignment
Eyal Liron Dolev
"Definition Modeling: To model definitions." Generating Definitions With Little to No Semantics
Vincent Segonne, Timothee Mickus
Contrastive Loss is All You Need to Recover Analogies as Parallel Lines
Narutatsu Ri, Fei-Tzin Lee, Nakul Verma
Detecting and Mitigating Indirect Stereotypes in Word Embeddings
Erin George, Joyce Chew, Deanna Needell
Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation
Di Wu, Christof Monz
Probing Brain Context-Sensitivity with Masked-Attention Generation
Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion, Christophe Pallier