ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction

Bojun Zhang; Junhong Liang; Feifei Zhai; Fengxian Ji; Yu Zhou

cs.CVAug 27, 2026

ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction

Authors: Bojun Zhang, Junhong Liang, Feifei Zhai, Fengxian Ji, Yu Zhou

Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Mohamed bin Zayed University of Artificial Intelligence · Fanyu AI Laboratory, Zhongke Fanyu Technology Co., Ltd, Beijing, China

Abstract

Vision Language Models (VLMs) have shown great success in general visual tasks, yet they still struggle to deeply understand text within images. In this paper, we introduce ReViCo (Real Visual Correction), a benchmark designed to evaluate VLM text understanding through a novel task of visual text error correction. ReViCo challenges models to identify and fix text errors in real-world images, which requires a profound understanding of the interplay between visual text and its surrounding visual context. We benchmark various VLMs using two distinct paradigms: prompt-based strategy and targeted model training, both aimed at pushing the limits of current models. Our experiments reveal a striking performance gap between even the best VLMs and human, and further analysis also shows that most models struggle to accurately perceive the visual text, resulting in frequent correction errors. By highlighting these gaps, ReViCo provides a new benchmark foundation for developing more robust and text-aware VLMs.

Figures & tables

Figure 1: Examples of visual text error correction. Only when a model truly understands the textual content within an image can it perform the correction accurately, both for the Chinese and English example.

Figure 2: Overview of the ReViCo Benchmark pipeline, illustrated with one English and one Chinese example from the dataset. The evaluation procedure is shown with a single English example.

Figure 3: Error statistics of the ReViCo benchmark. For Chinese and English separately, the inner pie charts show the counts of erroneous vs. error-free images, while the outer pie charts depict the distribution of error types across the erroneous images.

Models	I mage- D etection			T oken- D etection			I mage- C orrection			T oken- C orrection
Models	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$
Oracle OCR Text + Large language Models
Qwen3 8B	70.7	44.1	54.3	6.7	27.5	10.8	30.3	8.0	12.6	34.3	15.4	21.3
Qwen3 32B	63.7	57.0	60.2	9.8	25.7	14.2	31.4	14.5	19.8	36.7	20.5	26.3
Qwen3 235B	77.4	58.6	66.7	24.4	33.2	28.1	55.1	21.0	30.4	45.6	24.1	30.2
Qwen3VL 8B	75.5	44.2	55.8	12.3	24.8	16.5	44.5	11.5	18.2	38.9	16.9	23.6

Table 1: Performance comparison of OCR+LLMs and VLMs on visual text error detection and correction, evaluated at both the image (I) level and token (T) level across detection (D) and correction (C) tasks. The best overall results in each column are highlighted in bold , while the strongest results within the OCR+LLM, QwenVL and InternVL families are underlined .Please note that Qwen3VL models are used as LLM in OCR+LLM pipeline(w.o. image input).

Model	I-D		T-D		I-C		T-C
Model	${F_{1d}}$	${F_{1b}}$	${F_{1d}}$	${F_{1b}}$	${F_{1d}}$	${F_{1b}}$	${F_{1d}}$	${F_{1b}}$
Qwen2.5VL 3B	58.5	51.1 -7.4	8.3	2.8 -5.5	5.7	0.6 -5.1	6.0	1.5 -4.5
$\hookrightarrow$ GRPO	68.2 +9.7	63.4 +4.9	9.5 +1.2	8.4 +0.1	6.9 +1.2	5.9 +0.2	14.0 +8.0	13.9 +7.9
Qwen2.5VL 7B	37.1	62.0 +24.9	17.7	21.6 +3.9	14.2	20.4 +6.2	14.7	18.5 +3.8
$\hookrightarrow$ GRPO	68.7 +31.6	73.1 +36.0	17.3 -0.4	27.2 +9.5	16.4 +2.2	23.3 +9.1	18.6 +3.9	22.1 +7.4
Qwen2.5VL 32B	64.3	66.5 +2.2	41.1	47.2 +6.1	39.5	44.5 +5.0	36.5	43.6 +7.1

Table 2: Performance comparison across prompting paradigms and GRPO training.

F_{1d}

and

F_{1b}

denote direct and background-enhanced prompts, respectively. Blue / red values indicate increases/decreases relative to the original model’s

F_{1d}

. Detection (D) and Correction (C) are reported at Image (I) and Token (T) levels.

Figure 4: Case studies of attention visualization with QwenVL-7B.Each row shows the original image alongside attention maps from the 22nd and 25th layers. The corresponding golden labels and model predictions are displayed below, where incorrect tokens are highlighted in red and corrected tokens in blue .

Figure 5: Performance of humans and VLMs on randomly selected images. Numbers indicate the counts of correctly classified images in image detection and fully corrected images in wrong image correction.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Language	Error Type	Explanation
Chinese	Phonetic Error	Substitution involving characters with similar pronunciation.
	Graphic Error	Substitution involving characters with similar visual appearance.
	Phonetic-Graphic Error	Substitution involving both phonetic and graphic similarity.
	Others	Instances that do not fall into the preceding categories.
English (Letter Level)	Letter Omission	A letter is missing from a word.
	Letter Insertion	An extra letter is incorrectly added to a word.

Appendix

Table 3: Hierarchical categorization of text error types in the ReViCo benchmark.

Figure 6: Examples from our ReViCo Benchmark, where the error types for Chinese and English are displayed. Wrong tokens are highlighted in red and the corresponding correct tokens are highlighted in blue .

Error Type	Count
Phonetic	649
Graphic	238
Phonetic + Graphic	409
Other Error	417
Total	1713

Appendix

Table 4: Distribution of error types in the Chinese reinforcement learning data.

Error Type	Count
Letter Substitution	1043
Letter Omission	652
Letter Insertion	766
Phonetic Confusion	348
Split Word	9
Total	2818

Appendix

Table 5: Distribution of error types in the English reinforcement learning data.

Background Information Enhancement
(Chinese) 请先对图片的内容（包括文本、环境和背景信息等）进行概括	(English) Please first summarize the content of the image (including text, environment and background information et al.)

Appendix

Table 6: Additional prompts Background Information Enhancement, where the prompts are added to the direct prompt (Appendix C ) as the first step.

Models	I mage- D etection			T oken- D etection			I mage- C orrection			T oken- C orrection
Models	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$	$P$	$R$	$F_{1}$
Qwen3 8B	70.7	44.1	54.3	6.7	27.5	10.8	30.3	8.0	12.6	34.3	15.4	21.3
$\hookrightarrow$ GLM $--$ OCR	60.9	40.0	48.3	4.3	19.1	7.1	17.4	5.4	8.2	27.8	11.0	15.8
$\hookrightarrow$ HunyuanOCR	74.0	17.0	27.7	4.9	8.2	6.1	20.8	1.6	2.9	29.3	5.2	8.8
$\hookrightarrow$ Qwen2.5 $--$ VL $--$ 72B	63.8	41.9	50.6	4.5	19.4	7.3	20.3	6.1	9.3	28.7	11.9	16.8
$\hookrightarrow$ Qwen3 $--$ VL $--$ 235B	63.4	42.0	50.5	3.9	19.6	6.5	19.0	5.7	8.8	28.4	11.8	16.7

Appendix

Table 7: Performance comparison of OCR+LLMs with Oracle OCR Text and OCR tools on visual text error detection and correction, evaluated at both the image (I) level and token (T) level across detection (D) and correction (C) tasks. The best overall results in each column are highlighted in bold , while the strongest results within each LLM’s results are underlined . Please note that the first line of each LLM’s results represents Oracle OCR Text+LLM.

Hyper-parameters	Value
Training steps	100
Batch size	512
Warmup ratio	0.03
Learning rate scheduler	Cosine
Learning rate	$1\times 10^{-6}$
Rollout	6

Appendix

Table 8: The hyper-parameters used during GRPO of Qwen2.5-VL-7B using the training set in ReViCo benchmark.

Figure 7: Examples of visual error correction by GPT-4o and QwenVL-32B under background information enhancement. The relevant background information is highlighted in blue , while incorrect or vague descriptions are highlighted in red .

Model	I-D		T-D		I-C		T-C
	$F_{1d}$	$F_{1b}$	$F_{1d}$	$F_{1b}$	$F_{1d}$	$F_{1b}$	$F_{1d}$	$F_{1b}$
Oracle OCR Text + Large language Models
Qwen3VL 4B	57.6	–	14.4	–	16.0	–	21.2	–
$\hookrightarrow$ GRPO	59.3 +1.7	–	17.5 +3.1	–	18.1 +2.1	–	23.0 +1.8	–
Qwen3VL 8B	55.8	–	16.5	–	18.2	–	23.6	–
$\hookrightarrow$ GRPO	60.2 +4.4	–	19.2 +2.7	–	19.6 +1.4	–	26.1 +2.5	–

Appendix

Table 9: Performance comparison of Qwen3-VL-4B and Qwen3-VL-8B under OCR+LLM and Image+VLM paradigms.

F_{1d}

and

F_{1b}

denote direct and background-enhanced prompts, respectively. Results marked with

\hookrightarrow

GRPO represent the model trained via GRPO. For the OCR+LLM pipeline, only

F_{1d}

is reported as background enhancement is not applicable to pure text. Blue / red values indicate increases/decreases relative to the base model’s performance without GRPO. Thinking mode is activated for the OCR+LLM pipeline.

Figure 8: Chinese visual text error correction for Qwen2.5VL-3B without(up) and with targeted model training(bottom), original VLM response is in Chinese, English translation is given.

Figure 9: English visual text error correction for Qwen3VL-8B without(up) and with targeted model training(bottom).

Explore similar work

Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
May 21, 2026Zixuan Lan, Luzhe Sun, Matthew R. Walter +1VLM Evaluation VLM Interpretability
University of Chicago · Toyota Technological Institute at Chicago · Stony Brook University
Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark
Jun 9, 2026Pratham Singla, Shivank Garg, Vihan Singh +1VLM Evaluation Vision-Language Models
Lossfunk · Indian Institute of Technology Roorkee · Raeth AI
Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
Apr 17, 2026Yige Xu, Yongjie Wang, Zizhuo Wu +3Vision-Language Models VLM Adaptation
College of Computing and Data Science, Nanyang Technological University, Singapore · Alibaba-NTU Global e-Sustainability CorpLab (ANGEL) · Tongyi Lab, Alibaba Group, China

May 21, 2026J/K move · Enter open · S save

Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?

Zixuan Lan, Luzhe Sun, Matthew R. Walter, Jiawei Zhou

University of Chicago · Toyota Technological Institute at Chicago · Stony Brook University

VLM Evaluation VLM Interpretability Vision-Language Grounding Visually Grounded Reasoning Fine-Grained Visual Perception

Open paper AskPDF arXiv

Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image tokens only degrades model performance very slightly on a widely used hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs. Our analysis spans multiple levels of granularity, spanning global visual degradation, localized occlusion, question reformulation, answer-space expansion, and decision-level analyses beyond standard accuracy. We further complement these behavioral results with a layer-wise analysis of vision-token geometry. Throughout the experiments, we find that although VLMs do incorporate visual input, their predictions are less sensitive to the loss of fine-grained visual evidence that standard accuracy should have suggested. Even when the final prediction remains unchanged, the model's internal support for the correct answer may already be weakened. We further complement a representation-level analysis, which shows increasing similarity among visual tokens in deeper layers, providing a possible explanation for our findings. Together, these results suggest that current benchmarks are not sufficient to reliably evaluate fine-grained visual grounding in VLMs.

Abstract

Explore similar work

Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?

Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark

Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap