When Meaning Isn't Literal: Exploring Idiomatic Meaning Across Languages and Modalities

Sarmistha Das; Shreyas Guha; Suvrayan Bandyopadhyay; Salisa Phosit; Kitsuchart Pasupa; Sriparna Saha

cs.CLApr 12, 2026

When Meaning Isn't Literal: Exploring Idiomatic Meaning Across Languages and Modalities

Authors: Sarmistha Das, Shreyas Guha, Suvrayan Bandyopadhyay, Salisa Phosit, Kitsuchart Pasupa, Sriparna Saha

Organizations: Department of Computer Science and Engineering, Indian Institute of Technology Patna, Patna, India · School of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand

Abstract

Idiomatic reasoning, deeply intertwined with metaphor and culture, remains a blind spot for contemporary language models, whose progress skews toward surface-level lexical and semantic cues. For instance, the Bengali idiom \foreignlanguage{bengali}{\char"0986\char"0999\char"09CD\char"0997\char"09C1 \char"09B0 \char"09AB\char"09B2 \char"099F\char"0995} (angur fol tok, grapes are sour''): it encodes denial-driven rationalization, yet naive models latch onto the literal fox-and-grape imagery. Addressing this oversight, we present Mediom,'' a multilingual, multimodal idiom corpus of 3,533 Hindi, Bengali, and Thai idioms, each paired with gold-standard explanations, cross-lingual translations, and carefully aligned text--image representations. We benchmark both large language models (textual reasoning) and vision-language models (figurative disambiguation) on Mediom, exposing systematic failures in metaphor comprehension. To mitigate these gaps, we propose ``HIDE,'' a Hinting-based Idiom Explanation framework that leverages error-feedback retrieval and targeted diagnostic cues for iterative reasoning refinement. Collectively, Mediom and HIDE establish a rigorous test bed and methodology for culturally grounded, multimodal idiom understanding embedded with reasoning hints in next-generation AI systems.

Figures & tables

\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishFig. 1: Idiomatic Understanding via HIDE inspired by EFL: The model first interprets the idiom, detects semantic errors, and uses targeted hints to refine and improve idiomatic comprehension.

Corpus Name	Count	Language	Explanations	Multimodality	Idiomatic Part
SemEval-2013 [ \fontspec_if_language:nTFENG\addfontfeatureLanguage=English16 ]	4,350	E	$\times$	$\times$	$\times$
FLUTE [ \fontspec_if_language:nTFENG\addfontfeatureLanguage=English17 ]	8,962	E	$\checkmark$	$\times$	$\checkmark$ (497)
V-FLUTE [ \fontspec_if_language:nTFENG\addfontfeatureLanguage=English18 ]	6,027	E	$\checkmark$	$\checkmark$	$\checkmark$ (370)
Mediom (Proposed)	3,533	ML	$\checkmark$	$\checkmark$	3,533

\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishTable I: Comparative overview of the proposed Mediom against leading resources. Abbreviations: E = English; ML = multilingual set (Thai, Hindi, Bengali); Multimodality = paired text–image resources.

Panel of Figure 3 — \fontspec_if_language:nTF ENG \addfontfeature Language=EnglishFig. 2: (a) Sample instances of our proposed Mediom dataset; (b) Two different candidate images for the same idiom in the Mediom dataset.

\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishFig. 3: Architectural framework for idiom explanation that fuses LLMs and VLMs, augmented by a HIDE module with Hint Generation

Model Type	Model	Metrics
Model Type	Model	R-1	R-2	R-L	B-1	B-2	B-3	B-L	BS	MS	CD	JSD	L2	L1	PS	FRS
LLM	LLaMA2-7B	0.42	0.18	0.34	0.24	0.16	0.13	0.21	0.67	0.37	0.39	0.58	12.17	98.95	74.37	58.11
	Mistral -7B	0.47	0.24	0.41	0.27	0.24	0.20	0.27	0.73	0.45	0.34	0.53	9.82	78.91	70.11	38.05
	GPT 3.5	0.51	0.25	0.42	0.31	0.24	0.20	0.27	0.75	0.46	0.31	0.54	10.14	49.73	68.31	48.91
	Gemma2-9B	0.65	0.55	0.52	0.51	0.46	0.44	0.55	0.79	0.55	0.22	0.39	7.19	44.13	57.69	67.15
	LLaMA2-7B+HIDE	0.42	0.18	0.35	0.24	0.18	0.14	0.21	0.69	0.38	0.38	0.57	11.93	96.02	73.23	59.91

\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishTable II: Performance variance across baseline LLMs and VLM configurations. R-1, R-2, and R-L denote ROUGE-1, ROUGE-2, and ROUGE-L; B-1, B-2, B-3, and B-L denote BLEU-1, BLEU-2, BLEU-3, and BLEU-L; BS denotes BERTScore. MS, CD, JSD, L2, L1, PS, and FRS represent METEOR, Cosine Distance, Jensen–Shannon Divergence, L2 (Euclidean) Distance, L1 (Manhattan) Distance, Perplexity Score, and Flesch–Kincaid Readability Score. Boldface indicates the best results;

\uparrow

higher is better,

\downarrow

lower is better.

Panel of Figure 6 — \fontspec_if_language:nTF ENG \addfontfeature Language=EnglishFig. 4: Head-to-head qualitative and error analysis.

Retrieval-Augmented Reasoning

See all 27 papers

When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models
Jun 1, 2026Sarmistha Das, Vaibhav Vishal, Shreyas Guha +3Mixture-of-Experts Language Models Low-Resource Language Processing
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · School of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Thailand
IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation
Apr 25, 2026Ayman Ali ShararaFigurative Language Understanding Multilingual Language Model Evaluation
MSc Data Science & Machine Learning (SPOC S21) Data ScienceTech Institute (DSTI)
Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages
Jun 1, 2026Saeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida +16Figurative Language Understanding Multilingual Language Models
1Mohamed bin Zayed University of Artificial Intelligence · University of Hamburg · 3Manipal University +4

Jun 1, 2026J/K move · Enter open · S save

When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models

Sarmistha Das, Vaibhav Vishal, Shreyas Guha, Amaan Ali+2

Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · School of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Thailand

Mixture-of-Experts Language Models Low-Resource Language Processing Multimodal Model Evaluation Multimodal Learning

Open paper AskPDF arXiv

In the contemporary epoch of multilingual education, learning idioms provides a fascinating gateway towards creativity, cultural values, historical context, and diverse perspectives inherent to various linguistic traditions. This paper showcases the navigation of retaining figurative and cultural semantics in low-resource Southeast Asian languages such as Hindi, Bengali, and Thai, where culturally rich idioms pose significant obstacles for computational modeling and cross-linguistic transfer due to their deep metaphorical complexity. To tackle such complexity, we present Varnika, a reconstructed multimodal idiom corpus comprising 3,533 multilingual idioms, enriched with seven idiomatic tones aligned with both textual and visual representations. Additionally, to infer informative idiomatic understanding, we introduce a Hybrid Mixture-of-Experts (HybridMoE) framework that embeds multiple idiomatic expert opinions while mitigating expert sparsity by integrating outputs from both selected and unselected experts through controlled hybridization, further augmented with Idiomatic Property Signals via masked multimodal embeddings. To analyze the performance across multiple dimensions, we propose the IDIO-TONE and Idiomatic Validation Score, a three-stage evaluation pipeline measuring (i) literal translation fidelity, (ii) visual-semantic alignment, and (iii) idiomatic meaning retention. Empirical evaluations highlight that HybridMoE achieves 5--6% performance gains across advanced vision language models, demonstrating improved representation of figurative language and culturally embedded meaning in multilingual multimodal settings

Abstract

Retrieval-Augmented Reasoning

Explore similar work

When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models

IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation

Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages