Recent advances in Vision-Language Models (VLMs) have significantly improved Visual Question Answering (VQA) performance, yet their reasoning over structured content such as tables remains inconsistent, often plagued by spatial misalignment and logical hallucinations. Existing approaches for improving table VQA typically rely on architectural modifications or expensive supervised fine-tuning (SFT), which impose high barriers to entry due to the need for large-scale, expert-annotated datasets and significant computational resources. In this work, we propose Self-RefineVQA, a training-free, inference-time self-correction framework that improves table VQA performance without additional labeled data or model updates. Inspired by human revision behavior, our method enables a single VLM to iteratively act as a generator, critic, and refiner, forming a closed self-feedback loop that corrects common reasoning errors through prompt-guided re-evaluation. Experiments on tabular VQA benchmarks demonstrate that Self-RefineVQA yields consistent accuracy gains in monolingual and cross-lingual understanding, with a particularly significant boost for small models, enabling lightweight models (e.g., 2B-7B parameters) to rival the performance of significantly larger counterparts. Furthermore, we provide a comparative complexity analysis demonstrating that while inference cost increases linearly, our framework offers superior data efficiency and a lower cumulative cost by eliminating the prohibitive annotation bottleneck of human labeling required for fine-tuning. Our results highlight the potential of prompt-based self-improvement as a scalable, accessible paradigm for reliable table reasoning in VLMs.
Figures & tables
Figure 1: The Challenge of Spatial Drift in Table VQA. Examples from (a) TableVQA-Bench, (b) DocVQA, and (c) WTQ. These samples illustrate how minor shifts in attention (spatial drift) lead to reasoning failures, highlighting the need for iterative re-evaluation provided by Self-RefineVQA.
Figure 2: Minor shifts in visual attention cause the model to attend to semantically adjacent but incorrect cells. The proposed Generator–Critic–Refiner cycle detects row-column misalignment and iteratively re-grounds attention to the correct table intersection.
Figure 3: The Cycle-of-Thought Framework : The structured Critic prompt changes the model’s conditioning context, prompting re-examination of spatial evidence rather than reinforcement of the initial output. The structured prompting demonstrably reduces error persistence, consistent with a visual re-grounding effect.
Dataset
Domain/Source
# Images
# QA Pairs
WikiTableQuestions (Image)
Wikipedia tables
2062
2062
DocVQA - Table
Real documents
1500
1500
TableVQA-Bench
Financial reports, scientific articles
1758
1758
IndotabVQA
Real Documents
1043
4172
Table 1: Dataset statistics used for evaluating Self-RefineVQA.
Model
Zero-shot
Self-consistency
Self-refine (Ours)
Improvement
WikiTableQuestions
TAPAS
24.10
24.10
26.96
+ 2.86
Qwen2-VL [2B]
53.54
58.58
61.06
+ 7.52
Qwen2.5-VL [3B]
58.97
56.45
71.97
+ 13.00
Qwen3-VL [2B]
65.86
44.40
69.59
+ 3.73
Qwen2-VL [7B]
63.29
65.91
74.59
+ 11.30
Table 2: Performance of Models on three different dataset.
Figure 4: Iteration-wise accuracy progression across WikiTableQuestions, DocVQA-Table, and TableVQA-Bench. Most gains occur in the first 1-2 iterations, after which performance saturates, supporting our choice of Tmax = 3.
Lang.
Model
Zero-shot
Self-Cons.
Self-refine
Δ
Arabic
Qwen2-VL
26.75
–
30.49
+3.74
Qwen2.5-VL
26.94
–
33.08
+6.14
English
Qwen2-VL
39.41
–
44.49
+5.08
Qwen2.5-VL
40.08
–
45.45
+5.37
Bahasa
Qwen2-VL
69.51
–
75.26
+5.75
Qwen2.5-VL
68.84
–
76.80
+7.96
Table 3: Performance of Qwen-VL models on IndotabVQA across different languages. Self-refine denotes our proposed approach.
Dataset
Model
Stage 1 Only (Zero-shot)
Stage (1+2) (No Refine)
Stage (1+3) (No Critique)
Full Pipeline (Ours)
WikiTableQuestions
Qwen2.5-VL [7B]
63.48
63.48
68.57
72.55
Qwen3-VL [8B]
73.62
73.62
74.35
75.90
DocVQA-Table
Qwen2.5-VL [7B]
89.48
89.48
90.27
93.31
Qwen3-VL [8B]
93.63
93.63
94.37
95.02
TableVQA-Bench
Qwen2.5-VL [7B]
72.40
72.40
75.53
79.87
Qwen3-VL [8B]
82.07
82.07
84.07
86.73
Table 4: Ablation study.
Metric
Supervised Fine-Tuning (SFT)
Self-RefineVQA (Ours)
Data
High (Requires 1k+ labeled pairs)
Zero (Training-free)
Compute
High (Backprop, Optimizer states)
Zero (Inference only)
Inference Cost
Low ( 1×Tgen )
Moderate ( ≈4×Tgen )
Memory
High (Weights + Gradients + Optimizer)
Low (Weights only)
Adaptability
Low (Requires retraining for new schemas)
High (Prompt adjustment)
Access
Open Weights (Required)
Black Box API (Compatible)
Table 5: Operational Efficiency Comparison
Figure 5: Failure taxonomy before and after Self-RefineVQA : We illustrate representative failure modes of zero-shot table VQA models (top), including spatial drift, numerical reasoning errors, and entity mismatches. Self-RefineVQA corrects these errors via iterative self-critique and refinement (bottom), re-grounding predictions in verified row–column intersections without additional training.
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · Artificial Intelligence Research Institute, iFLYTEK Co., Ltd
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China