MetFuse: Figurative Fusion between Metonymy and Metaphor

Multimodal metaphor understanding requires identifying cross-domain Target-Source mappings and inferring sentiment from visual and textual evidence. However, existing benchmarks largely evaluate these subtasks separately and lack evidence-grounded explanations, limiting their ability to assess coherent cross-modal reasoning. To enable unified and evidence-grounded evaluation, we introduce M$^3$R-Bench, a benchmark of 1,000 human-annotated image-text instances with joint labels for metaphor occurrence, Target-Source mapping, and sentiment, alongside stage-wise explanations for evidence identification, mapping establishment, and sentiment inference. Evaluations reveal that existing models frequently rely on textual shortcuts and produce Target-Source mappings inconsistent with visual evidence. To address this evidence-mapping mismatch, we propose M$^3$R-Reasoner, which combines curriculum-based reasoning supervision with task-aware reinforcement learning. Built on an 8B-parameter backbone, M$^3$R-Reasoner achieves 37.90% Joint Label Accuracy (JLA) metric, outperforming the strongest proprietary baseline by 19.50 percentage points. It further surpasses GPT-5.5 in image-text evidence and sentiment justification scores by 28.45 and 28.11 points, respectively. The dataset and code are available at https://github.com/hongshi4/m3r-bench-artifact.

Metaphor identification is a foundational task in figurative language processing, yet most computational approaches operate as opaque classifiers offering no insight into why an expression is judged metaphorical. This interpretability gap is especially acute for Chinese, where rich figurative traditions, absent morphological cues, and limited annotated resources compound the challenge. We present an LLM-assisted pipeline that operationalises four metaphor identification protocols--MIP/MIPVU lexical analysis, CMDAG conceptual-mapping annotation, emotion-based detection, and simile-oriented identification--as executable, human-auditable rule scripts. Each protocol is a modular chain of deterministic steps interleaved with controlled LLM calls, producing structured rationales alongside every classification decision. We evaluate on seven Chinese metaphor datasets spanning token-, sentence-, and span-level annotation, establishing the first cross-protocol comparison for Chinese metaphor identification. Within-protocol evaluation shows Protocol A (MIP) achieves an F1 of 0.472 on token-level identification, while cross-protocol analysis reveals striking divergence: pairwise Cohen's kappa between Protocols A and D is merely 0.001, whereas Protocols B and C exhibit near-perfect agreement (kappa = 0.986). An interpretability audit shows all protocols achieve 100% deterministic reproducibility, with rationale correctness from 0.40 to 0.87 and editability from 0.80 to 1.00. Error analysis identifies conceptual-domain mismatch and register sensitivity as dominant failure modes. Our results demonstrate that protocol choice is the single largest source of variation in metaphor identification, exceeding model-level variation, and that rule-script architectures achieve competitive performance while maintaining full transparency.

Metaphorical language remains a major challenge for multilingual natural language processing because successful interpretation and translation require reasoning beyond literal lexical meaning. Existing research has largely investigated metaphor detection, machine translation, and translation evaluation as separate tasks, while little work has explored how these components can be integrated into a unified computational framework. This PhD proposal aims to develop an end-to-end framework for multilingual metaphor processing consisting of three complementary research directions: (1) robust metaphor detection across languages, (2) metaphor-oriented translation evaluation for both human assessment and automatic quality estimation, and (3) joint modelling that connects metaphor detection with translation evaluation. The proposed research will combine linguistic theory with recent advances in large language models to develop new datasets, annotation methodologies, evaluation benchmarks, and automatic evaluation approaches for metaphor-aware machine translation. The expected outcome is a unified framework that improves both the development and evaluation of multilingual NLP systems when processing figurative language.