ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

Jinlong Li; Jiaming Ding; Dingfu Lu; Malcolm Hsiu; Chuang Ke; Kangning Yang; Bochen Guan; Lan Fu

cs.CVAug 28, 2026

ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

Authors: Jinlong Li, Jiaming Ding, Dingfu Lu, Malcolm Hsiu, Chuang Ke, Kangning Yang, Bochen Guan, Lan Fu, +3 more

Organizations: OPPO US AI Center, USA · University of Wisconsin–Madison, USA · University of California San Diego, USA

Abstract

Multimodal Large Language Models (MLLMs) have recently made strong progress in visual--linguistic understanding. However, their performance on text-centric video reasoning remains highly sensitive to input quality. Real-world user-provided videos often contain motion blur, compression artifacts, noise, and low-resolution text, which impair reliable text reading and downstream reasoning. Whether MLLMs can robustly read and reason about real-world scene text under diverse quality conditions remains a fundamental open question. We introduce ClearText-Video (CTVid), a large-scale, scene-text-aware benchmark for studying text-centric video understanding under controlled quality variation. CTVid contains 4,639 real-world text-rich egocentric videos, 550K+ frames, 1.6M human-verified scene-text annotations, and 220K+ spatial/temporal question--answer pairs in Chinese and English. For each high-quality video, CTVid provides content-matched Degraded-Quality and Restored-Quality variants, supporting two task families: Text-Centric Video Restoration and Multi-Quality VideoQA. We evaluate 18 representative restoration methods and 16 state-of-the-art MLLMs on CTVid. The results show that visual enhancement does not guarantee textual fidelity or downstream reasoning gains: blur is more damaging than low resolution, restored videos can alter the textual evidence used by MLLMs, and OCR-only pipelines remain far below direct multimodal reasoning. CTVid exposes the gap between video restoration and text-grounded understanding, providing a rigorous foundation for restoration-aware, quality-robust text-centric video systems.

Figures & tables

Figure 1: Overview of the ClearText-Video (CTVid) Benchmark . CTVid is built from bilingual (Zh/En) egocentric videos with scene-text boxes, transcripts, captions, and spatial/temporal QA pairs. Its key design is a quality-controlled video triplet: each content instance is evaluated as High-Quality (HQ), Degraded-Quality (DQ), and Restored-Quality (RQ), allowing direct analysis of how degradation and restoration change the textual evidence available to MLLMs. Unlike restoration-only datasets or single-quality TextVQA benchmarks, CTVid links Text-Centric Video Restoration (image SR, video SR, and deblurring) with Multi-Quality VideoQA, testing whether models can preserve, recover, and reason over the same scene text across changing video quality.

Task	Dataset	Venue	GT Resolution	Caption	Text in Video	Content Language	Videos/Img	Video Source
Video Super-Resolution	VideoLQ [ 6 ]	CVPR 2022	640 $\times$ 480	✗	✗	✗	0.05K/4.9K	Online
	SPMCS [ 47 ]	ICCV 2017	1920 $\times$ 1080	✗	✗	✗	0.03K/0.93K	Manual
	Vimeo-90k [ 62 ]	IJCV 2019	448 $\times$ 256	✗	✗	✗	4.28K/0.6M	Online
	RealVSR [ 65 ]	ICCV 2021	1024 $\times$ 512	✗	✗	✗	0.5K/2.5K	Manual
	YouHQ [ 75 ]	CVPR 2024	1920 $\times$ 1080	✗	✗	✗	37K/-	Online
	MVSR4 $\times$ [ 51 ]	CVPRW 2023	1920 $\times$ 1080	✗	✗	✗	0.2K/2K	Manual

Table 1: Comparison of ClearText-Video with representative video datasets. “Text in Video” indicates whether textual content appears in video frames, and “Content Language” reports the languages of embedded text.

Figure 2: Example videos and their annotated questions from the ClearText-Video benchmark. Note: [BBOX] denotes bounding-box coordinates, visualized as red detection boxes in the images.

Figure 3: Overview of the annotation pipeline.

Input Type	Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	CLIPIQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	FID $\downarrow$	P det $\uparrow$	R det $\uparrow$	F1 det $\uparrow$	Acc rec $\uparrow$	NED rec $\downarrow$
Image-based	Real-ESRGAN [ 52 ]	27.85	0.8748	0.1638	0.1032	0.4790	4.877	64.65	0.5718	35.00	89.06	57.78	70.09	34.82	53.51
	SwinIR [ 28 ]	28.18	0.8788	0.1613	0.1012	0.5029	4.877	65.73	0.5705	32.15	88.78	58.47	70.50	36.76	54.96
	SeeSR [ 57 ]	27.89	0.8529	0.1824	0.1161	0.6471	5.121	69.92	0.6196	27.66	87.27	68.72	76.89	35.08	54.95
	OSEDiff [ 56 ]	25.67	0.8289	0.1946	0.1151	0.6219	4.852	69.11	0.6168	32.67	87.59	66.35	75.50	34.25	54.06
	S3Diff [ 68 ]	26.05	0.8033	0.1617	0.09611	0.5764	4.517	64.52	0.5799	29.16	86.72	69.52	77.17	38.42	58.03
	AdcSR [ 7 ]	25.84	0.8239	0.2024	0.124	0.6448	4.941	69.82	0.6083	37.52	88.84	57.72	69.98	32.02	51.26

Table 2: Text-Centric Visual Super-Resolution Benchmark. We report both general perceptual metrics and text-aware metrics. Text-aware metrics include precision (P

{}_{det}\%

), recall (R

{}_{det}\%

), and H-mean (F1

{}_{det}\%

) for detection, and accuracy (Acc

{}_{rec}\%

) and normalized edit distance (NED

{}_{rec}\%

) for recognition. The best and second-best results for each metric are highlighted in red and blue , respectively.

Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	CLIPIQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	FID $\downarrow$	P det $\uparrow$	R det $\uparrow$	F1 det $\uparrow$	Acc rec $\uparrow$	NED rec $\downarrow$
MIMO-UNet+ [ 11 ]	29.26	0.8657	0.2053	0.1393	0.3324	6.483	46.02	0.4760	47.15	88.58	65.02	74.99	51.75	69.54
NAFNet [ 8 ]	28.21	0.8716	0.2461	0.1749	0.3487	7.614	48.35	0.4253	48.65	90.81	66.37	76.69	50.77	68.30
Restormer [ 67 ]	31.38	0.8863	0.1904	0.1349	0.3404	6.552	47.31	0.4864	41.50	88.28	67.08	76.23	54.99	71.67
Stripformer(GoPro) [ 49 ]	31.13	0.8832	0.1754	0.1191	0.3348	6.386	48.65	0.4955	38.78	88.42	66.69	76.03	55.05	72.26
Stripformer(RealBlur-J) [ 49 ]	27.04	0.8655	0.1837	0.1182	0.3657	6.240	54.44	0.4922	32.36	89.18	69.99	78.43	58.93	76.51
Stripformer(RealBlur-R) [ 49 ]	27.36	0.8667	0.1905	0.1166	0.3396	6.490	50.41	0.4668	33.46	89.45	69.71	78.36	58.57	75.71

Table 3: Text-Centric Video Deblurring Benchmark. We use the same metrics as the Super-Resolution Benchmark above. Best and second-best results are highlighted in red and blue , respectively.

Figure 4: Statistics of the CTVid dataset. The left panels summarize text carrier types and scene categories, while the right panels show word-frequency visualizations for video captions and frame-level text annotations.

Panel of Figure 5 — Figure 5: Qualitative results on the Text-Centric Video Restoration benchmarks on the CTVid test set. Results are shown for both English and Chinese text instances. The examples include video super-resolution and deblurring outputs, highlighting differences in text fidelity and legibility across methods.

	HQ				DQ-Low_res [ 34 ]				DQ-Blur [ 34 ]				RQ-DOVE [ 10 ]				RQ-MIMO [ 11 ]				RQ-S3DIFF [ 68 ]
Model	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs	Acc $\uparrow$	OC $\downarrow$	UAcc $\uparrow$	Abs
GPT-5.4 [ 38 ]	60.00	15.00	55.00	0.00	53.33	10.00	58.33	6.70	56.67	8.33	60.00	18.30	48.33	15.00	55.00	3.30	55.00	10.00	58.33	6.70	53.33	13.33	53.33	0.00
GPT-5.4-mini [ 37 ]	46.67	6.67	65.00	0.00	48.33	18.33	51.67	0.00	43.33	5.00	63.33	3.30	46.67	3.33	66.67	0.00	41.67	11.67	56.67	0.00	45.00	8.33	56.67	0.00
Claude-Sonnet-4.6 [ 2 ]	70.00	21.67	66.67	0.00	63.33	20.00	70.00	0.00	60.00	20.00	70.00	13.30	65.00	18.33	70.00	0.00	70.00	20.00	70.00	3.30	61.67	16.67	71.67	0.00
Gemini-2.5-pro [ 12 ]	71.67	16.67	65.00	0.00	65.00	8.33	71.67	1.70	60.00	13.33	66.67	23.30	65.00	13.33	68.33	1.70	63.33	20.00	56.67	10.00	70.00	11.67	65.00	0.00
Gemini-2.5-flash [ 12 ]	60.00	21.67	61.67	1.70	65.00	10.00	61.67	1.70	48.33	10.00	76.67	30.00	66.67	10.00	66.67	3.30	60.00	16.67	61.67	11.70	61.67	13.33	58.33	1.70

Table 4: Text-Centric VideoQA for Spatial Understanding. We evaluate accuracy (Acc), uncertainty-aware accuracy (UAcc), overconfidence ratio (OC), and abstention rate (Abs) across six input-quality conditions: HQ (High-Quality), DQ-Low_res [ 34 ] (low-resolution degradation), DQ-Blur [ 34 ] (blur degradation), RQ-DOVE [ 10 ] (video-based SR), RQ-MIMO [ 11 ] (blur restoration), and RQ-S3DIFF [ 68 ] (image-based SR). “Qwen2.5-VL-7B-SFT” denotes Qwen2.5-VL-7B instruction-tuned on the ClearText-Video training set. For Acc, OC, and UAcc, the best and second-best results in each column are highlighted in red and blue , respectively. All reported values are percentages (%).

Figure 6: Accuracy (Acc) and uncertainty-aware accuracy (UAcc) radar plots for multiple MLLMs across six video-quality conditions. The top plot reports Acc, and the bottom plot reports UAcc.

Accuracy%( $\uparrow$ )
	Models	HQ	DQ-Blur [ 34 ]	RQ-MIMO [ 11 ]	RQ-S3DIFF [ 68 ]	RQ-DOVE [ 10 ]
	Gemini-2.5-flash [ 12 ]	51.55	51.20	52.12	51.32	53.04
	Gemini-2.5-pro [ 12 ]	45.83	50.00	51.39	50.00	48.61
	GPT-4o-mini [ 36 ]	50.06	50.52	50.29	50.13	49.94
	Claude-Sonnet-4.6 [ 2 ]	60.02	59.45	59.34	58.88	60.37
	Qwen2.5-VL-7B [ 3 ]	50.86	50.06	50.40	50.29	50.63

Table 5: Text-Centric Video Question Answering for Temporal Understanding. We report accuracy across five video-quality conditions. The best and second-best results in each quality condition are highlighted in red and blue , respectively, across all models.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Sunburst visualization of the ClearText-Video test set composition. From the center outward, the rings represent video source (offline vs. online), language (EN vs. CHN), and lighting condition (artificial vs. daylight). Offline videos constitute 74% of the test set, with a balanced distribution of English and Chinese clips and a mix of artificial and natural lighting. The online portion (26%) is evenly split between English and Chinese and is almost entirely captured under artificial lighting.

Dataset	#QA	#Boxes / Trans.	Spatial QA	Temporal QA	Bilingual/ Multilingual	Dense Frame-Text	Paired HQ/DQ/RQ	Rest. Eval.
TextVQA [ 44 ]	45,336	OCR tokens #boxes: NA	✗	✗	✗	✗	✗	✗
NewsVideoQA [ 19 ]	8,672	OCR tokens + subtitles	✗	✓	✗	✗	✗	✗
EgoTextVQA [ 76 ]	7,064	OCR stats only #boxes: NA	Partial	✓	✗	✗	✗	✗
ViTXT-GQA [ 77 ]	2,055	52,494 boxes 2,227 segments	✓	✓	✗	Partial	✗	✗
M4-ViteVQA [ 73 ]	25,123	OCR tokens avg. 56.92/video	Partial	✓	✗	✗	✗	✗
MME-VideoOCR [ 43 ]	2,000	NA	✓	✓	Partial	✗	✗	✗

Appendix

Table 6: Comparison with text-centric video benchmarks. “#Boxes / Trans.” denotes text bounding boxes, OCR tokens, or transcripts when explicitly provided. “Dense Frame-Text” indicates dense frame-level text annotations rather than sparse or question-grounded evidence. “Partial” indicates related but incomplete support.

Figure 8: Example videos and their annotated questions from the ClearText-Video benchmark. Note: [BBOX] denotes bounding-box coordinates, visualized as the red detection boxes in the images. The examples span transportation, commercial, and public-service scenes in both English and Chinese environments, illustrating the multilingual and text-centric diversity of our benchmark. This diversity enables the evaluation of model generalization across varied real-world video scenarios.

Figure 9: An example text instance showing its trajectory, speed, and acceleration over time.

Figure 10: Temporal variation of the bounding-box area for the same text instance.

Figure 11: Failure modes on RQ inputs. Restoration may hallucinate or distort text, while MLLMs may further fail through localization bias, reasoning errors, or temporal aggregation errors.

	HQ	LR	Blur	DOVE	MIMO	S3Diff
Avg. Acc $\uparrow$	47.73	44.59	41.69	45.21	44.02	44.79
Avg. $\Delta$ HQ	–	-3.14	-6.05	-2.52	-3.72	-2.95
Qwen3-VL-8B Acc $\uparrow$	50.56	45.43	44.46	47.70	46.91	46.64
Qwen3-VL-8B $\Delta$ HQ	–	-5.13	-6.10	-2.86	-3.65	-3.92

Appendix

Table 7: Quality-type comparison on spatial QA. Avg. Acc is averaged over the 16 MLLMs in the main spatial VideoQA table.

Input quality	OCR+Qwen2.5-7B			Qwen2.5-VL-7B
Input quality	Acc $\uparrow$	UAcc $\uparrow$	OC $\downarrow$	Acc $\uparrow$	UAcc $\uparrow$	OC $\downarrow$
HQ	23.3	66.0	14.6	51.21	58.69	10.89
DQ-Low_res	22.7	66.7	13.5	40.23	61.67	3.99
DQ-Blur	22.3	66.6	13.7	41.72	62.66	9.19
RQ-DOVE	22.9	64.5	17.7	46.58	59.97	10.79
RQ-MIMO	22.2	61.9	20.1	46.10	59.89	10.79

Appendix

Table 8: OCR+LLM vs. direct MLLM on spatial QA. OCR+Qwen2.5-7B uses OCR text only, without image/video input.

Figure 12: Result visualization on ClearText-Video for text-centric spatial VideoQA. This example shows a Chinese sign in which the question asks the model to select the option that correctly matches the text inside a specified bounding box. Most higher-performing models identify the correct phrase consistently across quality conditions, while weaker models confuse visually similar Chinese characters or repeatedly select an incorrect option. This example illustrates that even when the text is visually prominent, robust spatial text understanding still depends on precise character-level discrimination rather than coarse recognition.

Figure 13: Result visualization on ClearText-Video for text-centric spatial VideoQA. This example shows an outdoor transit sign in which the question asks whether a set of words, including “Food”, “Do Not”, “on bus”, and “Bring”, appears inside the specified bounding box. Many models correctly verify the presence of the target phrases across quality conditions, but some models consistently reject the correct answer or become unstable under degraded and restored inputs. This case illustrates that spatial VideoQA requires not only text recognition, but also accurate grounding of multiple words within the queried region.

Figure 14: Result visualization on ClearText-Video for text-centric spatial VideoQA. This example shows an English warning sign where the question requires completing the missing substring in “HIGH ____ TAGE”. Most proprietary MLLMs consistently recover the correct missing text across different quality conditions, while several open-source models either output the full word instead of the missing span, predict only a single character, or hallucinate visually plausible but incorrect fragments. The results highlight the difficulty of fine-grained text completion under blur, low resolution, and restoration artifacts.

Panel of Figure 15 — Figure 15: Qualitative results on the Text-Centric Video Restoration benchmark on the CTVid test set. Results are shown for both English and Chinese text instances. The examples include video super-resolution and deblurring outputs, highlighting differences in text fidelity and legibility across methods.

Figure 16: Result visualization on ClearText-Video for text-centric temporal VideoQA. This example asks for the overall motion direction of the text “ $\downarrow$ 363-378Gates” across the video clip. Although the ground-truth motion is consistently leftward under all quality conditions, most models incorrectly predict that the text stays roughly in place, suggesting substantial reliance on static keyframe cues in this example. Only a few models predict the correct motion direction, while their predictions still become unstable under degraded or restored inputs, showing that robust temporal motion reasoning over scene text remains challenging for current MLLMs.

Figure 17: Result visualization on ClearText-Video for text-centric temporal VideoQA. This example asks where the text “NO SUPRISES!” appears for most of the clip. The ground-truth answer is consistently the bottom-left region, but many models confuse it with the bottom-right or center regions, and some predictions vary across different quality conditions. This example illustrates that reliably estimating the dominant spatial position of a text instance over time requires more than recognizing the text in a single frame, as models must aggregate its location across the full temporal sequence.

Figure 18: Result visualization on ClearText-Video for text-centric temporal VideoQA. This example asks about the relation between the text “PHARMACY” and the frame boundary. Although the correct answer is that the text touches or runs off an edge, most models incorrectly classify it as fully inside or merely close to an edge, especially under degraded and restored inputs. This case highlights the difficulty of boundary-aware temporal text reasoning, where models must precisely track the relation between a text instance and the frame boundary rather than relying on coarse recognition alone.

System Prompt

Fill-in Question Prompt

Your task is to answer fill-in questions based on image input. Input Format: An image with {size} and a fill-in question string, e.g., “C____T”. Output Format: A single-line JSON object: {"answer": "<only the missing characters>", "reasoning": "<a brief explanation, 1 sentence max>"} Rules: Only output the missing characters, not the full word. Keep the reasoning short and image-based. Do not infer from common sense or spelling patterns, because the answer may not follow normal spelling. Only answer based on what is actually visible in the image. The blank, represented as “____”, can correspond to any number or type of characters, including letters, digits, Chinese characters, or symbols. Example: Question: What letter is missing in “C__T”? Answer: {"answer": "a", "reasoning": "C-a-T forms CaT as shown in the image."} Respond only with a JSON object. Do not include anything else.

Multi-choice Question Prompt

You are a vision-language assistant specialized in text recognition. Your task is to answer multiple-choice questions based on image content. Input Format: An image with {size} and a question string with options, e.g., “Which letter is missing in G__FT? Choices: A. A B. B C. I D. H”. Output Format: A single-line JSON object: {"answer": "<only the correct letter(s)>", "reasoning": "<a brief explanation, 1 sentence max>"} Rules: Only output the letter(s) corresponding to the correct option. Do not repeat the full option. Keep the reasoning short and image-based. The answer can contain multiple letters, e.g., “A,C”, if multiple options are correct. Do not infer from common sense or spelling patterns. Only answer based on what is actually visible in the image. Example: Question: Which word(s) is located within the area […]? Choose the correct option. A. Apple B. App C. Able D. Aple Answer: {"answer": "A,B", "reasoning": "I can see both ‘Apple’ and ‘App’ in the bounding box area."} Respond only with a JSON object. Do not include anything else.

True-False Question Prompt

You are a vision-language assistant specialized in text recognition. Your task is to answer true/false questions based on the content of the image. Input Format: An image with {size} and a true/false question string, e.g., “The word in the image is DOG. True or False?”. Output Format: A single-line JSON object: {"answer": "True" or "False", "reasoning": "<a brief explanation, 1 sentence max>"} Rules: The answer must be strictly “True” or “False”. Keep the reasoning short and image-based. Do not infer from common sense or spelling patterns. Only answer based on what is actually visible in the image. Example: Question: The word in the image is DOG. True or False? Answer: {"answer": "False", "reasoning": "The image shows DoG, not DOG."} Respond only with a JSON object. Do not include anything else.

Appendix

Table 9: System prompt used for evaluating spatial video question answering.

System Prompt

You are a helpful assistant that can answer questions about a video. Video context: The original video has {total_frames} frames in total{fps_clause}. Original frame resolution: {original_size}. Frames are resized to {resized_size} before being shown to you. Any coordinates or bounding boxes mentioned in the questions are based on the original resolution {original_size}. You are shown {n_sampled} frames uniformly sampled from this video. The frames are given in chronological order. The k-th frame you see, 1-based, corresponds to original frame index sampled_indices[k-1], where sampled_indices = {sampled_indices_str}. When the question refers to a specific frame index or timestamp, reason about which of your sampled frames is closest to it. You only have access to the listed sampled frames. Answer vocabulary — geometric question types: Each question about a specific target text uses one of the fixed option sets below. Pick the option that best matches what you observe across the sampled frames. Position regions: “center (middle of the frame)”: the central

\sim

1/3 of the frame on both the horizontal and vertical axes; “top-left (upper-left area)”: upper half of the frame, left of centre; “top-right (upper-right area)”: upper half of the frame, right of centre; “bottom-left (lower-left area)”: lower half of the frame, left of centre; “bottom-right (lower-right area)”: lower half of the frame, right of centre. Position change amount: “stays in roughly the same place (barely shifts)”: the text barely moves; displacement is negligible; “moves a moderate amount (shifts across part of the frame)”: noticeable shift — text crosses a portion of the frame; “moves a large distance (shifts across much of the frame)”: large displacement — text traverses most of the frame. Overall motion direction: “stays roughly in place”: negligible net displacement throughout; “moves left”: predominantly leftward drift; “moves right”: predominantly rightward drift; “moves up”: predominantly upward drift; “moves down”: predominantly downward drift; “moves diagonally”: significant movement on both horizontal and vertical axes simultaneously; “changes direction partway”: reverses along its dominant axis mid-clip, e.g., moves right then left. Text size level: “small (a minor part of the frame)”: text occupies a small fraction of the frame, roughly

<2\%

; “medium (a noticeable block of the frame)”: clearly visible, moderate fraction, roughly

2

–

6\%

; “large (a major part of the frame)”: covers a substantial portion of the frame, roughly

6

–

15\%

; “very large (dominates the frame)”: fills most of the frame, roughly

>15\%

. Scale change: “stays about the same size”: size is roughly constant throughout the clip; “gets larger”: text grows noticeably from the start to the end; “gets smaller”: text shrinks noticeably from the start to the end; “size fluctuates (goes up and down)”: size oscillates non-monotonically during the clip. Edge relation: “fully inside, away from the edges”: text stays well clear of all frame borders throughout; “comes close to an edge”: text approaches a border but does not touch it; “touches or runs off an edge”: text reaches or extends beyond a frame border. Answer the question based on the video. Directly return the answer with no extra text.

Appendix

Table 10: System prompt used for evaluating temporal video question answering.

Explore similar work

Watch Before You Answer: Learning from Visually Grounded Post-Training
Apr 6, 2026Yuxuan Zhang, EunJeong Hwang, Huaisong Zhang +8Vision-Language Models Vision-Language Grounding
University of British Columbia · Vector Institute · Etude AI +4
ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
Jun 23, 2026Zhentao Guo, Chen Duan, Tongkun Guan +3Multimodal Large Language Models Video QA
Meituan · Shanghai Jiao Tong University
VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA
May 6, 2026Haibin He, Maoyuan Ye, Jing Zhang +2Video QA Video Frame Selection
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, Wuhan, Hubei, China

Apr 6, 2026J/K move · Enter open · S save

Watch Before You Answer: Learning from Visually Grounded Post-Training

Yuxuan Zhang, EunJeong Hwang, Huaisong Zhang, Penghui Du+7

University of British Columbia · Vector Institute · Etude AI+4

Vision-Language Models Vision-Language Grounding Temporal Video Understanding VLM Evaluation

Open paper AskPDF arXiv

It is critical for vision-language models (VLMs) to comprehensively understand visual, temporal, and textual cues. However, despite rapid progress in multimodal modeling, video understanding performance still lags behind text-based reasoning. In this work, we find that progress is even worse than previously assumed: commonly reported long video understanding benchmarks contain 40-60% of questions that can be answered using text cues alone. Furthermore, we find that these issues are also pervasive in widely used post-training datasets, potentially undercutting the ability of post-training to improve VLM video understanding performance. Guided by this observation, we introduce VidGround as a simple yet effective solution: using only the actual visually grounded questions without any linguistic biases for post-training. When used in tandem with RL-based post-training algorithms, this simple technique improves performance by up to 6.2 points relative to using the full dataset, while using only 69.1% of the original post-training data. Moreover, we show that data curation with a simple post-training algorithm outperforms several more complex post-training techniques, highlighting that data quality is a major bottleneck for improving video understanding in VLMs. These results underscore the importance of curating post-training data and evaluation benchmarks that truly require visual grounding to advance the development of more capable VLMs. Project page: http://vidground.etuagi.com.

Abstract

Explore similar work

Watch Before You Answer: Learning from Visually Grounded Post-Training

ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering

VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA