Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation

Junxin Lu; Tengfei Song; Zhanglin Wu; Pengfei Li; Xiaowei Liang; Hui Yang; Kun Chen; Ning Xie

cs.CVFeb 25, 2026

Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation

Authors: Junxin Lu, Tengfei Song, Zhanglin Wu, Pengfei Li, Xiaowei Liang, Hui Yang, Kun Chen, Ning Xie,

Organizations: School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China · Huawei Translation Services Center, China · School of Computer Science and Technology, East China Normal University, Shanghai 200062, China · State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China

Abstract

Text Image Machine Translation (TIMT) aims to translate text embedded in images in the source-language into target-language, requiring synergistic integration of visual perception and linguistic understanding. Existing TIMT methods, whether cascaded pipelines or end-to-end multimodal large language models (MLLMs),struggle with high-resolution text-rich images due to cluttered layouts, diverse fonts, and non-textual distractions, resulting in text omission, semantic drift, and contextual inconsistency. To address these challenges, we propose GLoTran, a global-local dual visual perception framework for MLLM-based TIMT. GLoTran integrates a low-resolution global image with multi-scale region-level text image slices under an instruction-guided alignment strategy, conditioning MLLMs to maintain scene-level contextual consistency while faithfully capturing fine-grained textual details. Moreover, to realize this dual-perception paradigm, we construct GLoD, a large-scale text-rich TIMT dataset comprising 510K high-resolution global-local image-text pairs covering diverse real-world scenarios. Extensive experiments demonstrate that GLoTran substantially improves translation completeness and accuracy over state-of-the-art MLLMs, offering a new paradigm for fine-grained TIMT under high-resolution and text-rich conditions.

Figures & tables

Figure 1: Comparison of (a) cascade methods, (b) traditional end-to-end models, (c) MLLMs-based models, and (d) our proposed GLoTran for TIMT. Through a dual visual perception strategy integrating global contextual understanding and local textual focus, we enable more complete and accurate TIMT.

Figure 2: Overview of the proposed GLoTran framework. (a) The high-resolution input image is processed by a text region detector to identify candidate textual regions, which are subsequently sorted, merged, and cropped into localized slices. (b) The global image and the local slices are fed into GLoTran with a structured prompt, enabling global contextual understanding and local textual focus for TIMT.

Figure 3: An illustration of the structured prompt design in GLoTran.

Dataset	Text-Rich	Global-Local	Language	Images	Image-Text
ECOIT [ 65 ]	✗	✗	2	479,490	479,490
OCRMT30K [ 25 ]	✗	✗	2	30,000	30,000
DoTA [ 28 ]	✓	✗	2	126,345	126,000
IIMT [ 24 ]	✗	✗	1	89,033	89,033
MIT-10M [ 26 ]	✗	✗	14	840,855	10,931,115
GLoD	✓	✓	5	517,354	517,354

Table 1: The statistics of GLoD and other TIMT dataset.

Figure 4: Overview of GLoD curation pipeline. The construction pipeline systematically generates high-quality global and local translation pairs through five core stages: conceptualization, data collection and pre-filtering, text region detection and grouping, global-local translation, and quality control.

		Document			Scene			Poster
Method	Size	Paper $\uparrow$	News $\uparrow$	Novel $\uparrow$	Title $\uparrow$	Sign $\uparrow$	Introduction $\uparrow$	Cover $\uparrow$	Leaflet $\uparrow$	Avg $\uparrow$
Close-Source MLLMs
GPT-4o [ 23 ]	$\thicksim$ 200B	62.1 / 86.3	51.6 / 85.6	44.2 / 83.4	51.3 / 83.4	46.1 / 83.3	40.1 / 75.3	34.9 / 69.1	43.9 / 71.7	46.8 / 79.9
Qwen-VL-Max [ 1 ]	$\thicksim$ 72B	60.5 / 85.7	58.3 / 85.7	44.8 / 83.5	50.1 / 81.6	43.2 / 80.5	37.8 / 74.6	34.2 / 67.3	43.7 / 69.9	46.5 / 78.6
Open-Source MLLMs
Qwen2.5-VL-Instruct [ 2 ]	7B	60.7 / 85.8	53.4 / 85.5	27.4 / 78.1	47.9 / 80.1	44.0 / 82.2	36.2 / 72.8	30.3 / 64.4	43.6 / 72.4	42.9 / 77.7

Table 2: Performance comparison of translating English into Chinese with open-source and close-source MLLMs on the MCiTon [ 66 ] test set. We report BLEU/COMET for translation quality of each real-world scenarios.

\uparrow

indicates higher is better. The bold represents the best results, and the underline represents the second best results.

Figure 5: (a)-(b) performance comparison of multilingual translating task with open-source MLLMs on MTIT6 [ 40 ] . (c)-(d) performance analysis with respect to global image resolution and the pre-region translation replay window size $\eta$ .

Figure 6: Qualitative comparison. The results in $\square$ indicates omissions, those in $\square$ indicate mistranslations, and those in $\square$ indicate hallucinations caused by contextual misunderstanding.

Method	zh $\rightarrow$ en	zh $\rightarrow$ ko	ko $\rightarrow$ zh	jp $\rightarrow$ zh	Novel	Title	Introduction	Leaflet
InternVL2.5 4B	4.02 / 70.78	1.19 / 60.44	2.57 / 48.20	28.37 / 76.66	18.48 / 75.67	31.44 / 76.10	26.16 / 71.09	21.22 / 63.92
GLoTran (InternVL2.5 4B)	6.68 / 71.46	2.26 / 60.86	5.97 / 58.89	29.93 / 78.33	25.80 / 77.51	38.20 / 79.82	29.47 / 71.52	23.27 / 65.50
Gain $\bigtriangleup$	+66.17 / +0.96	+89.92 / +0.69	+132.29 / +22.18	+5.50 / +2.18	+39.61 / +2.43	+21.56 / +4.88	+12.65 / +0.61	+9.66 / +2.47
Qwen2.5-VL 7B	9.54 / 74.45	4.13 / 66.76	20.23 / 70.27	35.83 / 79.09	27.43 / 78.09	47.91 / 80.09	36.18 / 72.78	43.66 / 72.40
GLoTran (Qwen2.5-VL 7B)	10.99 / 76.78	5.45 / 68.49	22.39 / 70.99	43.81 / 81.84	44.59 / 83.60	53.15 / 82.92	38.93 / 74.12	45.64 / 73.74
Gain $\bigtriangleup$	+15.20 / +3.13	+31.96 / +2.59	+10.68 / +1.02	+22.27 / +3.48	+62.59 / +7.06	+10.93 / +3.53	+7.60 / +1.84	+4.54 / +1.85

Table 3: Ablation study on the effect of GLoTran across different backbone models. We report BLEU and COMET scores for each translation task. The Gain row denotes the relative improvement (%) of GLoTran over its baseline, calculated as

(\text{GLoTran}-\text{Baseline})/\text{Baseline}\times 100\%

		Document			Scene			Poster
Method	Size	Paper $\uparrow$	News $\uparrow$	Novel $\uparrow$	Title $\uparrow$	Sign $\uparrow$	Introduction $\uparrow$	Cover $\uparrow$	Leaflet $\uparrow$
InternVL3	8B	59.8/86.0	48.9/83.9	31.2/79.8	50.5/82.1	41.8/81.6	32.8/72.6	29.5/66.1	35.8/70.9
InternVL3 ♢	8B	60.5/86.0	45.0/83.7	36.8/81.2	48.1/77.9	36.8/75.6	37.4/73.4	27.8/63.7	46.9/72.9
Qwen2.5-VL	7B	60.7/85.8	53.4/85.5	27.4/78.1	47.9/80.1	44.0/82.2	36.2/72.8	30.3/64.4	43.6/72.4
Qwen2.5-VL ♢	7B	63.5/86.5	46.3/84.0	37.6/81.9	42.6/78.8	37.5/75.2	40.3/73.4	32.9/65.8	44.6/72.9
Qwen3-VL	8B	60.9/86.1	54.3/86.2	43.6/83.1	52.3/81.8	44.2/83.0	33.6/75.8	36.2/70.4	48.2/75.1

Table 4: Components analysis results (BLEU/COMET) of GLoTran on the MCiTon dataset.

Method	Size	Token V $\downarrow$	Novel			zh $\rightarrow$ jp
Method	Size	Token V $\downarrow$	FTL s $\downarrow$	FPS $\uparrow$	BLEU $\uparrow$	FTL s $\downarrow$	FPS $\uparrow$	BLEU $\uparrow$
Qwen2.5-VL ♡	7B	6.1K	0.098	0.061	27.36	0.098	3.755	11.85
InternVL3 ♡	8B	10K	0.033	0.065	31.74	0.149	2.016	15.02
Qwen3-VL ♡	8B	6.1K	0.099	0.028	9.98	0.093	2.572	18.79
Qwen3-VL ♠	8B	164K	1.081	0.041	43.56	3.297	0.275	22.06
GLoTran ♡	8B	8.4K	3.002	0.035	48.12	0.253	2.285	26.74

Table 5: Comparison of MLLMs in terms of efficiency and accuracy for TIMI. The superscripts

\clubsuit

\heartsuit

, and

\spadesuit

denote input resolutions of 224

\times

224, 448

\times

448, and the original size, respectively.

Image Text Translation

See all 8 papers

MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation
Mar 25, 2026Gengluo Li, Chengquan Zhang, Yupu Liang +9Image Text Translation Machine Translation
Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent +3
Comparative Evaluation of Machine Translation Systems on Images with Text
May 28, 2026Blai Puchol, Sergio Gómez González, Miguel Domingo +1VLM Evaluation Machine Translation
PRHLT Research Center - Universitat Polit`ecnica de Val`encia, Camino de Vera, s/n - Valencia · ValgrAI - Valencian Graduate School and Research Network for Artificial Intelligence, Camino de Vera s/n
UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation
Jun 23, 2026Jiahao Lyu, Pei Fu, Zhenhang Li +6Image Text Translation Unified Multimodal Models
Institute of Information Engineering, Chinese Academy of Science, China · MiLM Plus, Xiaomi Inc, China · School of Cyber Security, University of Chinese Academy of Science, China +2

Mar 25, 2026J/K move · Enter open · S save

MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

Gengluo Li, Chengquan Zhang, Yupu Liang, Huawen Shen+8

Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent+3

Image Text Translation Machine Translation Multilingual VLM Evaluation MT Evaluation Multimodal CoT Reasoning

Open paper AskPDF arXiv

End-to-end text-image machine translation (TIMT), which directly translates textual content in images across languages, is crucial for real-world multilingual scene understanding. Despite advances in vision-language large models (VLLMs), robustness across diverse visual scenes and low-resource languages remains underexplored due to limited evaluation resources. We present MMTIT-Bench, a human-verified multilingual and multi-scenario benchmark with 1,400 images spanning fourteen non-English and non-Chinese languages and diverse settings such as documents, scenes, and web images, enabling rigorous assessment of end-to-end TIMT. Beyond benchmarking, we study how reasoning-oriented data design improves translation. Although recent VLLMs have begun to incorporate long Chain-of-Thought (CoT) reasoning, effective thinking paradigms for TIMT are still immature: existing designs either cascade parsing and translation in a sequential manner or focus on language-only reasoning, overlooking the visual cognition central to VLLMs. We propose Cognition-Perception-Reasoning for Translation (CPR-Trans), a data paradigm that integrates scene cognition, text perception, and translation reasoning within a unified reasoning process. Using a VLLM-driven data generation pipeline, CPR-Trans provides structured, interpretable supervision that aligns perception with reasoning. Experiments on 3B and 7B models show consistent gains in accuracy and interpretability. We will release MMTIT-Bench to promote the multilingual and multi-scenario TIMT research upon acceptance.

Abstract

Image Text Translation

Explore similar work

MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

Comparative Evaluation of Machine Translation Systems on Images with Text

UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation