MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

Gengluo Li; Chengquan Zhang; Yupu Liang; Huawen Shen; Yaping Zhang; Pengyuan Lyu; Weinong Wang; Xingyu Wan

cs.CVMar 25, 2026

MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

Authors: Gengluo Li, Chengquan Zhang, Yupu Liang, Huawen Shen, Yaping Zhang, Pengyuan Lyu, Weinong Wang, Xingyu Wan,

Organizations: Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent · Institute of Automation, Chinese Academy of Sciences ·

Abstract

End-to-end text-image machine translation (TIMT), which directly translates textual content in images across languages, is crucial for real-world multilingual scene understanding. Despite advances in vision-language large models (VLLMs), robustness across diverse visual scenes and low-resource languages remains underexplored due to limited evaluation resources. We present MMTIT-Bench, a human-verified multilingual and multi-scenario benchmark with 1,400 images spanning fourteen non-English and non-Chinese languages and diverse settings such as documents, scenes, and web images, enabling rigorous assessment of end-to-end TIMT. Beyond benchmarking, we study how reasoning-oriented data design improves translation. Although recent VLLMs have begun to incorporate long Chain-of-Thought (CoT) reasoning, effective thinking paradigms for TIMT are still immature: existing designs either cascade parsing and translation in a sequential manner or focus on language-only reasoning, overlooking the visual cognition central to VLLMs. We propose Cognition-Perception-Reasoning for Translation (CPR-Trans), a data paradigm that integrates scene cognition, text perception, and translation reasoning within a unified reasoning process. Using a VLLM-driven data generation pipeline, CPR-Trans provides structured, interpretable supervision that aligns perception with reasoning. Experiments on 3B and 7B models show consistent gains in accuracy and interpretability. We will release MMTIT-Bench to promote the multilingual and multi-scenario TIMT research upon acceptance.

Figures & tables

Figure 1: MMTIT-Bench and the CPR-Trans Data Paradigm.

Dataset	Sample	Avg. Words	Languages	Scenarios			Annotation
Dataset	Nums	Per Image	Languages	Web	Document	Street	Model	Human
PATIMT [ 39 ]	1200	507	En $\to$ Zh, Zh $\to$ En	✓				✓
DITrans [ 36 , 35 ]	1651	458	En $\to$ Zh		✓			✓
DoTA [ 14 ]	1003	447	En $\to$ Zh		✓		✓	✓
OCRMT [ 11 ]	1000	5	Zh $\to$ En			✓	✓	✓
MTIT6 [ 22 ]	1200	7	(En&Ko&Ja) $\to$ Zh, Zh $\to$ (En&Ko&Ja)	✓		✓	✓	✓

Table 1: Comparison of existing Text-Machine Translation benchmarks.

Figure 2: Overview of the MMTIT-Bench construction pipeline, which consists of four stages: image collection, parsing annotation, translation annotation, and benchmark filtering with human verification.

#	Model	Param.	Think	Gemini-flash Judge		Qwen3VL Judge		COMET
#	Model	Param.	Think	other2en	other2zh	other2en	other2zh	other2en	other2zh
Cascade Pipeline	MinerU2.5 [ 20 ] +Qwen3	-	-	48.32	49.70	45.13	46.26	63.43	65.06
Cascade Pipeline	Dots.ocr [ 23 ] +Qwen3	-	-	66.24	65.27	61.20	60.98	73.62	73.42
General VLMs	MiMo-VL [ 32 ]	7B	✓	67.72	69.52	64.81	65.24	74.96	76.15
	dots.vlm1 [ 23 ]	671B-A37B	✓	79.23	76.73	73.82	72.14	76.84	78.45
	Qwen3-VL-Instruct [ 1 ]	235B-A22B	-	64.39	69.67	61.75	65.00	73.67	77.20

Table 2: Comparison of translation performance across different VLLMs under various evaluation settings. The “think” column indicates whether the model employs explicit reasoning.

Figure 3: Overview of the CPR-Trans data generation pipeline.

Model Param.	Method	Gemini-Flash Judge		Qwen3VL Judge		COMET
Model Param.	Method	other2en	other2zh	other2en	other2zh	other2en	other2zh
3B	Origin	38.32	33.04	47.01	43.54	68.80	69.96
	Direct	64.98	58.83	66.67	63.86	73.98	74.29
	Simple CoT	72.31	67.18	74.06	71.51	77.62	78.21
	Distillation(VLLM)	69.15	66.76	68.01	66.12	75.38	76.12
	Distillation(LLM)	67.92	65.22	67.30	64.51	75.21	75.09

Table 3: Comparison of different data paradigms under 3B and 7B model settings, evaluated by Gemini-Flash, Qwen3VL, and COMET.

#	Component			Gemini2.5 (o2e)		Gemini2.5 (o2z)		Qwen3VL (o2e)		Qwen3VL (o2z)
#	Cognition	Perception	Trans	3B	7B	3B	7B	3B	7B	3B	7B
Baseline				72.31	74.65	67.18	71.03	74.06	76.30	71.51	73.00
1	✓			75.23	76.91	72.21	74.12	76.82	79.08	72.88	75.69
2		✓		72.40	74.43	67.88	71.21	73.53	75.69	71.56	72.36
3			✓	79.47	80.73	78.92	80.21	78.49	80.51	76.92	80.32
4	✓	✓		76.98	77.82	73.90	75.50	77.03	79.73	73.45	77.26

Table 4: Ablation study of reasoning components on 3B and 7B models under two evaluation settings.

Figure 4: Qualitative example of CPR-Trans on an MMTIT-Bench sample, showing how the model sequentially conducts scene cognition, text perception, and translation reasoning to produce the final translation.

Inference Strategy	Gemini2.5 Judge		Qwen3VL Judge
Inference Strategy	other2en	other2zh	other2en	other2zh
One-step	64.39	69.67	61.75	65.00
Multi-step	70.54	73.21	68.72	70.54

Table 5: Training-free Validation of the CPR-Trans Paradigm on Qwen3-VL-235B-A22B-Instruct.

Image Text Translation

See all 8 papers

Mar 11, 2026

IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation

End-to-end In-Image Machine Translation (IIMT) aims to convert text embedded within an image into a target language while preserving the original visual context, layout, and rendering style. However, existing IIMT benchmarks are largely synthetic and thus fail to reflect real-world complexity, while current evaluation protocols focus on single-modality metrics and overlook cross-modal faithfulness between rendered text and model outputs. To address these shortcomings, we present In-image Machine Translation Benchmark (IMTBench), a new benchmark of 2,500 image translation samples covering four practical scenarios and nine languages. IMTBench supports multi-aspect evaluation, including translation quality, background preservation, overall image quality, and a cross-modal alignment score that measures consistency between the translated text produced by the model and the text rendered in the translated image. We benchmark strong commercial cascade systems, and both closed- and open-source unified multi-modal models, and observe large performance gaps across scenarios and languages, especially on natural scenes and resource-limited languages, highlighting substantial headroom for end-to-end image text translation. We hope IMTBench establishes a standardized benchmark to accelerate progress in this emerging task.

Jiahao Lyu, Pei Fu, Zhenhang Li +7

Institute of Information Engineering, Chinese Academy of Science, China · MiLM Plus, Xiaomi Inc, China · School of Cyber Security, University of Chinese Academy of Science, China +3

Image Text Translation Multimodal MT Machine Translation MT Evaluation

Ask PDF

Feb 25, 2026

Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation

Text Image Machine Translation (TIMT) aims to translate text embedded in images in the source-language into target-language, requiring synergistic integration of visual perception and linguistic understanding. Existing TIMT methods, whether cascaded pipelines or end-to-end multimodal large language models (MLLMs),struggle with high-resolution text-rich images due to cluttered layouts, diverse fonts, and non-textual distractions, resulting in text omission, semantic drift, and contextual inconsistency. To address these challenges, we propose GLoTran, a global-local dual visual perception framework for MLLM-based TIMT. GLoTran integrates a low-resolution global image with multi-scale region-level text image slices under an instruction-guided alignment strategy, conditioning MLLMs to maintain scene-level contextual consistency while faithfully capturing fine-grained textual details. Moreover, to realize this dual-perception paradigm, we construct GLoD, a large-scale text-rich TIMT dataset comprising 510K high-resolution global-local image-text pairs covering diverse real-world scenarios. Extensive experiments demonstrate that GLoTran substantially improves translation completeness and accuracy over state-of-the-art MLLMs, offering a new paradigm for fine-grained TIMT under high-resolution and text-rich conditions.

Junxin Lu, Tengfei Song, Zhanglin Wu +9

School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China · Huawei Translation Services Center, China · School of Computer Science and Technology, East China Normal University, Shanghai 200062, China +1

Image Text Translation Machine Translation Multimodal Large Language Models

Ask PDF

Sep 29, 2026

VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preservation across languages. However, existing benchmarks have limited language coverage and often lack explicit image-specific evaluation criteria, making it difficult to comprehensively assess this capability. To systematically evaluate this capability, we introduce VISTA-Bench, covering 22 languages and 10 domains, and develop an image-specific rubric evaluation protocol. The benchmark combines sampling for language and scenario coverage with model-assisted, human-verified annotations that group related text into coherent semantic units and provide multilingual reference translations. The rubrics specify essential content, semantic relations, and acceptable translation variants, yielding separate output-based scores for translation quality and the preservation of visual and knowledge-dependent information. We conduct extensive evaluations of 16 mainstream models, including 12 multimodal models and four text-input models, and provide systematic analyses across languages, domains, and evaluation dimensions.

Bo Lv, Mao Zheng, Zheng Li +3

Foundation Model Department, Tencent

Multilingual VLM Evaluation Rubric-Based Evaluation MT Evaluation

Ask PDF

Abstract

Image Text Translation

Explore similar work

IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation

Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation

VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics