Organizations: Indian Institute of Technology Mandi Mandi, India · DIT University Dehradun, India · Indian Institute of Technology Kharagpur Kharagpur, India
Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images suffer from real-world degradation such as motion blur, low resolution, or compression artifacts. While modular OCR-based pipelines and end-to-end vision-language models are both widely used for this task, their comparative robustness under degraded conditions remains underexplored. We present an empirical study comparing two modular pipelines with SA-DBNet, a custom detector architecture combining ResNet-18 with self-attention spatial modeling and deformable convolutions against an end-to-end vision-language baseline, evaluated on 4013 degraded images with 7000 question-answer pairs. Fine-tuned modular pipelines achieve up to 57.50% exact-match accuracy versus 38.00% for the end-to-end baseline, with domain-specific fine-tuning yielding a gain of up to 29.50 percentage points. Critically, we find that conventional OCR error metrics like Character Error Rate and Word Error Rate are unreliable predictors of downstream VQA performance, as semantic reasoning can compensate for recognition failures when contextual cues are present. These findings highlight the importance of task-aware evaluation for text-centric VQA systems under realistic visual conditions. Codes are available here
Figures & tables
Fig. 1: Conceptual illustration of text-centric VQA from a drone perspective. The system must recognize and reason over text (e.g., labeling) under motion blur degradation.
Fig. 2: Proposed modular pipeline: SA-DBNet + TrOCR + Qwen2-VL. Color coding: blue = pretrained components, purple = proposed custom architecture (self-attention bottleneck, DCN-v2 FPN, DB head), pink/red = fine-tuned components, green = pretrained reasoning module.
Fig. 3: Baseline PaddleOCR + Qwen pipeline architecture showing the standard DBNet+SVTR modular approach for benchmark comparison.
Fig. 4: Examples of visual degradation applied to text images in the experimental setup, including Gaussian blur, motion blur, median filtering, and downscaling.
Pipeline
Acc (%)
ANLS
Semantic
CER
FPS
PaddleOCR+Qwen (FT)
42.83
0.59
0.61
0.42
0.72
SA-DBNet+TrOCR+Qwen (FT)
57.50
0.54
0.50
0.46
0.59
Qwen2-VL (End-to-End)
38.00
0.57
0.65
0.58
0.76
TABLE I: Main Performance Comparison (Blur σ=5.0 , n=7000 )
Pipeline
Pretrained
Fine-tuned
Gain
PaddleOCR+Qwen
32.50%
42.83%
+10.33%
SA-DBNet+TrOCR+Qwen
28.00%
57.50%
+29.50%
TABLE II: Impact of Fine-Tuning on Blur Robustness