cs.CLJun 15, 2026

The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage

Authors: Afnan Aloraini, Viktor Schlegel, Goran Nenadic, Riza Batista-Navarro

Abstract

Automatic semantic change detection aims to identify how word meanings shift over time, offering insights into both linguistic and societal change. Despite recent progress in computational lexical semantic change (LSC), existing benchmarks and methods struggle to capture bi-directional semantic change, particularly cases where words simultaneously gain and lose senses. This problem is especially challenging for words that have both slang and standard meanings. To address these gaps, we introduce two complementary benchmark datasets. The Bi-Directional Lexical Semantic Change (BD-LSC) dataset captures sense gain, sense loss, and stability across three time periods, enabling the study of complex semantic trajectories. The SlangTrack Word Sense Disambiguation (ST-WSD) dataset provides fine-grained, instance-level sense annotations for words combining slang and standard usages, supporting systematic benchmarking of WSD and semantic change detection models. Using these benchmarks, we systematically evaluate models across different methodological families: unsupervised clustering using contextualised embeddings, supervised machine learning, transformer-based models, and state-of-the-art large language models. Among the evaluated systems, the few-shot GPT-4o model achieved the strongest aggregate performance on Exact Sense Match (ESM) and multi-label accuracy; however, Macro-F1 scores near 0.5 across all systems show that rare slang senses remain difficult, which we identify as the central open challenge.

Explore similar work

CardsList
Jul 23, 2026cs.CL

slang.gr as a Large-Scale Crowdsourced Resource for Non-Standard Greek

Slang is a central component of everyday language, reflecting linguistic creativity, social identity, and cultural change, yet its dy- namic and non-standard nature makes it difficult to model computationally. We present the first large-scale computational study of slang.gr, a crowdsourced lexicon of Greek non-standard language, combining lexical content, user-generated tags, and interaction data. To enable the systematic analysis, we map noisy folksonomic tags to a structured multi-layer taxonomy capturing both semantic categories and sociolinguistic metadata. Using this representation, we analyze the linguistic structure of Greek slang and the behavior of its contributor community. We find that slang is strongly centered on person-related and evaluative language, exhibits high morphological creativity, and is shaped by highly skewed participation with short user lifespans and overlapping communities. Building on these signals, we introduce a community-based confidence score for definitions that integrates user roles, interaction patterns, and moderation signals. Our results show that taxonomy-based representations improve interpretability while retaining meaningful aspects of behavioral structure, enabling a more structured and interpretable analysis of confidence signals. Overall, this work establishes slang.gr as a computational resource for non-standard Greek and provides a foundation for sociolinguistic NLP, bias analysis, and the study of informal language in LLMs.
Panagiotis Papadakos, Katerina Papantoniou, Dimitris Plexousakis
LinguisticsAncient GreekCrowdsourced Framework
AskPDF
Apr 19, 2026cs.CL

Community-Specific Slang and Entity Detection via Semantic Shift in Fine-Tuned Language Models

We propose an unsupervised method of resolving slang, unique entities, and folklore from online communities by isolating words in the lexicon that have the highest magnitude of semantic shift. Semantic shift is defined as the evolution of a word's encoded representation as a result of fine-tuning a pretrained Large Language Model (LLM) on a community-specific text corpus. This value is inversely proportional to the cosine similarity between the base model's encoded representation of a word, and a fine-tuned model's encoded representation. We fine-tune the DistilRoBERTa model on text corpora collected from 3 Reddit subreddits (r/Technology, r/Gaming, r/WorldofWarcraft), model a distribution of cosine similarity over the lexicon, and show that one can successfully resolve words that have unique significance to the community by pulling data in the bottom 10-percentile. In contrast, we show that data in the top 10-percentile consist of words that carry relatively universal semantics.
Julia Kruk, Sanchita Porwal, Amitrajit Bhattacharjee +1
RedditLarge Language Model Fine-TuningCorpus
AskPDF
Sep 17, 2026cs.CL

WiC is Not WSD: A Study on LLMs and Lexical Ambiguity Resolution

Word-in-Context (WiC) remains challenging for language models, despite recent progress on lexical-semantic tasks. We hypothesise that this difficulty arises not only from comparing two contextual uses of a word, but also from the absence of an explicit sense inventory that specifies the relevant level of semantic granularity. We evaluate open LLMs on WiC and traditional Word Sense Disambiguation (WSD) under similar settings. We find that providing candidate senses, similar to what is done in traditional WSD, improves WiC performance in all settings. In general, explicit sense information helps models make more consistent and targeted judgements. Human evaluation further shows that many apparent WiC errors reflect label ambiguity or mismatches between model and annotator sense boundaries rather than simple failures of lexical understanding. In particular, results show that LLMs overthink the sense distinction often leading to errors based on overly fine-grained distinctions.
Yi Zhou, Kiamehr Rezaee, Danushka Bollegala +2
Large Language Models Fail
AskPDF