Multimodal MT

MT: Machine Translation

Follow this topic Ask about this topic

Latest papers

11 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation
Oct 13, 2026Subaru Kimura, Jun SuzukiMultimodal MT MT Evaluation
Tohoku University
Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting
Sep 25, 2026Paweł Mąka, Piotr Andruszkiewicz, Yusuf Can Semerci +2Multimodal MT Machine Translation
Department of Advanced Computing Sciences, Maastricht University · Warsaw University of Technology · IDEAS Research Institute
Last Translation Benchmark
Sep 3, 2026Vilém Zouhar, Niyati Bafna, Mukund Choudhary +257Multimodal MT MT Evaluation
There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation
Aug 28, 2026Gabe Guo, Elon Litman, Thanawat Sornwanee +2Multimodal MT Multimodal Generation
Stanford University
Direct Image-to-Modern Vietnamese Translation of Han-Nom Manuscripts via Multimodal RLHF Preference Alignment
Jul 13, 2026Thi Kim Trang Vo, Nghia Hieu Nguyen, Ha Minh TanDirect Preference Optimization Preference Optimization
University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation
Jun 23, 2026Jiahao Lyu, Pei Fu, Zhenhang Li +6Image Text Translation Unified Multimodal Models
Institute of Information Engineering, Chinese Academy of Science, China · MiLM Plus, Xiaomi Inc, China · School of Cyber Security, University of Chinese Academy of Science, China +2
ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation
May 15, 2026Michał Ciesiółka, Dawid Wiśniewski, Adrian Charkiewicz +1Multimodal MT Machine Translation
Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland · Pozna´n University of Technology
Towards Visually-Guided Movie Subtitle Translation for Indic Languages
May 12, 2026Tarun Chintada, Kshetrimayum Boynao Singh, Asif EkbalMultimodal MT Low-Resource MT
Department of Computer Science and Engineering Indian Institute of Technology Patna, India
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Large Vision-Language Models Multimodal MT
Department of Informatics, Universität Hamburg · Alibaba Group
IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation
Mar 11, 2026Jiahao Lyu, Pei Fu, Zhenhang Li +7Image Text Translation Multimodal MT
Institute of Information Engineering, Chinese Academy of Science, China · MiLM Plus, Xiaomi Inc, China · School of Cyber Security, University of Chinese Academy of Science, China +3
Scalable Multilingual Multimodal Machine Translation with Speech-Text Fusion
Feb 25, 2026Yexing Du, Youcheng Pan, Zekun Wang +7Multimodal MT Machine Translation
Harbin Institute of Technology · Pengcheng Laboratory

Oct 13, 2026J/K move · Enter open · S save

Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation

Subaru Kimura, Jun Suzuki

Tohoku University

Multimodal MT MT Evaluation Vision-Language Model-as-a-Judge VLM Evaluation

Open paper AskPDF arXiv

In manga translation, visual context outside speech-bubble text, including speaker identities, referents, and scene tone, is depicted in page images, making it natural to refer to images during translation evaluation. However, most conventional manga translation evaluations rely solely on text-only automatic metrics, leaving the impact of image input insufficiently explored. We propose a practical validation framework that employs the same vision-language model (VLM) as both a text-only judge (TOJ) and an image-aware judge (IAJ) to investigate evaluation divergences associated with image-aware evaluation procedures. We use a Japanese-English manga dataset of official translations aligned at the page and speech-bubble levels. First, adding page images to TOJ improved the primary VLM's detection of deliberately injected, visually dependent degradations. Second, we analyzed evaluation divergences between TOJ and IAJ, finding different error counts and error-category distributions. Human verification did not reveal a consistent preference between TOJ and IAJ.