Span Modeling for Idiomaticity and Figurative Language Detection with Span Contrastive Loss
Organizations: Information Science, Japan Advanced Institute of Science and Technology, Ishikawa, Japan.
Abstract
The category of figurative language contains many varieties, some of which are non-compositional in nature. This type of phrase or multi-word expression (MWE) includes idioms, which represent a single meaning that does not consist of the sum of its words. For language models, this presents a unique problem due to tokenization and adjacent contextual embeddings. Many large language models have overcome this issue with large phrase vocabulary, though immediate recognition frequently fails without one- or few-shot prompting or instruction finetuning. The best results have been achieved with BERT-based or LSTM finetuning approaches. The model in this paper contains one such variety. We propose BERT- and RoBERTa-based models finetuned with a combination of slot loss and span contrastive loss (SCL) with hard negative reweighting to improve idiomaticity detection, attaining state of the art sequence accuracy performance on existing datasets. Comparative ablation studies show the effectiveness of SCL and its generalizability. The geometric mean of F1 and sequence accuracy (SA) is also proposed to assess a model's span awareness and general performance together.
Figures & tables
| Dataset | Task | Primary | Best |
|---|---|---|---|
| EPIE (Formal) | Span tagging (BIO) | Acc. | 98.00 * |
| MAGPIE (random) | Span id. (exact) | Sequence Accuracy | 91.51 Yayavaram et al. (2024) |
| MAGPIE (random) | Span id. | Sequence Accuracy | 92.97 Matheny et al. (2025) |
| MAGPIE (type-aware) | Span id. (unseen types) | Sequence Accuracy | 70.47 Zeng and Bhat (2021) |
| SemEval-2022 2A | Idiomaticity existence | Macro-F1 (All) | 93.85 * |
| FLUTE (Idioms) | Sentence-pair entailment | Accuracy | 79.20 * |
| Model | Hardest ds. | ||||||
|---|---|---|---|---|---|---|---|
| IFL-C4-A-BERT | 0.3762 | 0.3112 | 0.3421 | FLUTE | 0.0968 | 0.0229 | 0.0471 |
| IFL-C4-A-RoBERTa | 0.3513 | 0.2940 | 0.3214 | SEMEVAL2022 | 0.0862 | 0.0000 | 0.0000 |
| FLUTE-BERT | 0.1910 | 0.1960 | 0.1935 | SEMEVAL2022 | 0.0906 | 0.0206 | 0.0432 |
| FLUTE-RoBERTa | 0.1813 | 0.1358 | 0.1569 | PIFL-OSCAR | 0.0508 | 0.0249 | 0.0356 |
| MAGPIE (random)-BERT | 0.4575 | 0.4558 | 0.4566 | SEMEVAL2022 | 0.1446 | 0.0714 | 0.1016 |
| MAGPIE (random)-RoBERTa | 0.4459 | 0.4607 | 0.4532 | SEMEVAL2022 | 0.1366 | 0.0689 | 0.0970 |
| Model | Hardest ds. | ||||||
|---|---|---|---|---|---|---|---|
| IFL-C4-A-BERT 1.0 | 0.3749 | 0.3104 | 0.3202 | FLUTE | 0.0955 | 0.0285 | 0.0522 |
| IFL-C4-A-RoBERTa 1.0 | 0.3595 | 0.2993 | 0.2916 | SEMEVAL2022 | 0.0862 | 0.0000 | 0.0000 |
| FLUTE-BERT 0.1 | 0.2253 | 0.2509 | 0.2372 | SEMEVAL2022 | 0.0381 | 0.0258 | 0.0314 |
| FLUTE-RoBERTa 0.1 | 0.2320 | 0.2257 | 0.2263 | SEMEVAL2022 | 0.0879 | 0.0317 | 0.0528 |
| MAGPIE (random)-BERT 0.1 | 0.4677 | 0.4811 | 0.4676 | FLUTE | 0.1472 | 0.0783 | 0.1074 |
| MAGPIE (random)-RoBERTa 0.5 | 0.4670 | 0.4805 | 0.4674 | SEMEVAL2022 | 0.1525 | 0.0363 | 0.0744 |
| MAGPIE (random) - Over prediction | |
|---|---|
| Sentence | i suppose you get used to it after a while , sleeping in the day . |
| Gold Labels - (none) | i suppose you get used to it after a while , sleeping in the day . |
| BERT baseline - incorrect | I suppose you get used to it after a while , B-idiom[sleeping] I-idiom[in] I-idiom[the] I-idiom[day] . |
| RoBERTa - incorrect | I suppose you get used to it after a while , B-idiom[sleeping] I-idiom[in] I-idiom[the] I-idiom[day] . |
| BERT SCL-neg - incorrect | I suppose you get used to it after a while , B-idiom[sleeping] I-idiom[in] I-idiom[the] I-idiom[day] . |
| RoBERTa SCL-neg - incorrect | I suppose you get used to it after a while , B-idiom[sleeping] I-idiom[in] I-idiom[the] I-idiom[day] . |
Figurative Language Understanding
Figurative language understanding studies how people and AI interpret nonliteral meaning in metaphors, idioms, proverbs, puns, and related forms.
See all 45 papersIdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions
When Meaning Isn't Literal: Exploring Idiomatic Meaning Across Languages and Modalities
grapes are sour''): it encodes denial-driven rationalization, yet naive models latch onto the literal fox-and-grape imagery. Addressing this oversight, we present Mediom,'' a multilingual, multimodal idiom corpus of 3,533 Hindi, Bengali, and Thai idioms, each paired with gold-standard explanations, cross-lingual translations, and carefully aligned text--image representations. We benchmark both large language models (textual reasoning) and vision-language models (figurative disambiguation) on Mediom, exposing systematic failures in metaphor comprehension. To mitigate these gaps, we propose ``HIDE,'' a Hinting-based Idiom Explanation framework that leverages error-feedback retrieval and targeted diagnostic cues for iterative reasoning refinement. Collectively, Mediom and HIDE establish a rigorous test bed and methodology for culturally grounded, multimodal idiom understanding embedded with reasoning hints in next-generation AI systems.