Recent advancements in Vision-Language Models (VLMs) have significantly pushed the boundaries of Visual Question Answering (VQA).However,high-resolution details can sometimes become noise that leads to hallucinations or reasoning errors. In this paper,we propose Degradation-Driven Prompting (DDP), a novel framework that improves VQA performance by strategically reducing image fidelity to force models to focus on essential structural information. We evaluate DDP across two distinct tasks. Physical attributes targets images prone to human misjudgment, where DDP employs a combination of 80p downsampling, structural visual aids (white background masks and orthometric lines), and In-Context Learning (ICL) to calibrate the model's focus. Perceptual phenomena addresses various machine-susceptible visual anomalies and illusions, including Visual Anomaly (VA), Color (CI), Motion(MI),Gestalt (GI), Geometric (GSI), and Visual Illusions (VI).For this task, DDP integrates a task-classification stage with specialized tools such as blur masks and contrast enhancement alongside downsampling. Our experimental results demonstrate that less is more: by intentionally degrading visual inputs and providing targeted structural prompts, DDP enables VLMs to bypass distracting textures and achieve superior reasoning accuracy on challenging visual benchmarks.
Figures & tables
Figure 1: Less is More in Visual Perception. Comparison between a standard high-resolution pipeline and our proposed DDP strategy. While high-resolution inputs ( 500×400 p) can paradoxically lead to misinterpretation (e.g., identifying ”18” instead of ”73”), our DDP leverages lower resolution ( 80×64 p) to eliminate background noise. This approach achieves a 50% reduction in response time and a 50% improvement in accuracy for basic physical attribute tasks by focusing the model on essential structural information.
Figure 2: Overcoming visual reasoning bottlenecks via the DDP framework. Standard VLMs are easily deceived by optical illusions or occlusions (e.g., a dog seemingly split by a tree). Our DDP approach introduces a ”divide-and-conquer” strategy: the classifier categorizes the image type, the tool manager invokes specialized visual tools (e.g., draw_rectangle and crop ) to highlight suspicious regions, and the Critic synthesizes these visual cues. The Critic detects that the front and back portions mismatch the trunk width, thereby correcting the initial misconception and providing the right answer ( Option C: 2 ).
Figure 3: Overview of the DDP-based VLM enhancement framework. The workflow consists of three primary stages: (1) classifier : The input image and choice question are categorized into preset domains (e.g., motion illustration, colorblindness, or color attributes) to guide subsequent tool selection. (2) tool manager : After an initial noise-removal downsampling, a DDP agent performs iterative function calls to select specialized visual tools from magazines, such as Polar/Cartesian auxiliary lines , crops , and masks , to highlight key features. (3) target prompting : Despite the extreme downsampling (structural bottleneck < 80p), the processed image is fed into the Critic module. By leveraging task-specific prompts and auxiliary tips based on the ClassID, the system generates a structured Chain-of-Thought (CoT) reasoning process to produce the final option.
Figure 4: A case study demonstrating how DDP leverages external tools to solve visual perception bottlenecks. The pipeline iteratively classifies the task, invokes specific image-processing tools (blurring and contrast enhancement), and utilizes the resulting ”degraded” yet cleaner visual features to perform robust reasoning on perception-intensive tasks.
Category
Sub-tasks
Assigned Tools
Physical Attributes
color, size, length
crop, white mask, cartesian mask, red box
Perceptual Phenomena
counting,difference locating, color-blind, motion, geometry, real picture, size, others
Table 1: Task classification and corresponding tool-sets. For physical attributes (e.g., color, size, length), isolation and highlighting tools such as crop, masks, and red boxes are utilized to eliminate background interference . Conversely, for perceptual phenomena tasks (e.g., counting, geometry, difference locating), structural and visibility-enhancing tools like auxiliary lines (cartesian/polar) and contrast enhancement are employed to provide spatial coordinate references and emphasize subtle visual patterns.
Method
MMBench [ 9 ]
SEED-Bench [ 21 ]
SciQA [ 13 ]
VQAv2 [ 26 ]
LLaVA-v1.5 [ 24 ]
64.3
66.1
66.8
78.5
Qwen-VL [ 1 ]
38.2
62.3
67.1
78.8
GPT-4o [ 28 , 6 ]
83.4
77.1
94.3
82.3
Gemini-1.5-pro [ 12 ]
82.3
77.1
94.8
73.2
Gemini-3-pro
88.4
87.2
98.1
85.6
DDP (w/ Gemini-3-Pro)
92.1
94.5
99.1
89.4
Table 2: Performance comparison on common public benchmarks. Accuracy (%) is reported for all metrics. GPT-4o results are cited from its official technical report.
Model
Attribute (%)
Spatial (%)
Overall (%)
Human [ 40 ]
98.3
100.0
99.0
Bard [ 40 ]
31.3
46.1
37.2
Gemini-1.5-Pro [ 40 ]
40.9
59.2
48.2
GPT-4V [ 40 ]
51.3
60.5
55.0
InstructBLIP [ 40 ]
25.2
47.4
34.0
LLaVA-1.5-7B [ 40 ]
43.5
56.6
48.7
Table 3: Performance comparison on the V*Bench [ 40 ] . Our proposed DDP consistently outperforms both state-of-the-art proprietary models and open-source models across all evaluation metrics, including Attribute, Spatial, and Overall accuracy.
Models / Tasks
ColorBlind (Pass@1)
OpenAI o1
0.00%
Claude-3-Sonnet
0.00%
Gemini-2.5-Pro
0.00%
Qwen2.5-VL-72B
0.00%
InternVL3-78B
0.00%
MiniCPM-V-2.6
0.00%
Table 4: Comparison of Pass@1 accuracy on TeT [ 10 ] . DDP significantly outperforms state-of-the-art general-purpose VLMs.
Method
Original Images
Perturbed Images
Gemini-3-Pro
89.52%
66.19%
DDP (w/ Gemini-3-Pro)
95.71%
86.19%
Improvement
+6.19%
+20.00%
Table 5: Performance comparison on Workshop task1 [ 34 ] . We evaluate the zero-shot accuracy ( ↑ ) using Gemini-3-Pro as the backbone model across task1 challenging domain.
Variant
Attribute (%)
Spatial (%)
Overall (%)
Δ
Full Pipeline
89.2
89.5
89.3
-
w/o Visual Tools
82.6
85.5
83.8
-5.5
w/o Prompt Engineering
85.2
86.8
85.9
-3.4
w/o Degradation
83.8
76.3
80.6
-8.7
Gemini-3-Pro
52.2
68.4
58.6
-30.7
Table 6: Ablation study of pipeline components on the V*Bench [ 40 ] . We evaluate the contribution of the Tool Manager, Prompt Engineering, and Task Classifier across different reasoning dimensions. Bold indicates the best performance.
Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images suffer from real-world degradation such as motion blur, low resolution, or compression artifacts. While modular OCR-based pipelines and end-to-end vision-language models are both widely used for this task, their comparative robustness under degraded conditions remains underexplored. We present an empirical study comparing two modular pipelines with SA-DBNet, a custom detector architecture combining ResNet-18 with self-attention spatial modeling and deformable convolutions against an end-to-end vision-language baseline, evaluated on 4013 degraded images with 7000 question-answer pairs. Fine-tuned modular pipelines achieve up to 57.50% exact-match accuracy versus 38.00% for the end-to-end baseline, with domain-specific fine-tuning yielding a gain of up to 29.50 percentage points. Critically, we find that conventional OCR error metrics like Character Error Rate and Word Error Rate are unreliable predictors of downstream VQA performance, as semantic reasoning can compensate for recognition failures when contextual cues are present. These findings highlight the importance of task-aware evaluation for text-centric VQA systems under realistic visual conditions. Codes are available here
Visual Question Answering (VQA) is the task of answering questions about images, requiring the integration of multimodal input and reasoning. Modular approaches that incorporate logic-based representations into the reasoning component offer clear advantages over end-to-end trained systems, particularly in terms of interpretability. However, adapting or extending these representations when task requirements change can place a significant burden on developers. To address this challenge, we present an approach for distilling rules from Large Language Models (LLMs). Our method prompts an LLM to extend an initial VQA reasoning theory, expressed as an answer-set program, to meet new requirements of the task. Examples from VQA datasets guide the LLM, validate the results, and help correct erroneous rules by leveraging feedback from the ASP solver. We demonstrate that our approach is effective across diverse VQA datasets. Notably, only a few examples are needed to elicit correct rules from LLMs. Our experiments suggest that rule distillation from LLMs is a promising alternative to traditional data-driven rule learning approaches. Under consideration in Theory and Practice of Logic Programming (TPLP).
Thomas Eiter, Nelson Higuera Ruiz, Johannes Oetsch
Vienna University of Technology (TU Wien), Favoritenstrasse 9–11, Vienna, 1040, Austria · Jönköping University, Gjuterigatan 5, 551 11 Jönköping, Sweden
Manual annotation of high-quality visual question answering with grounding (VQA-G) datasets, which pair visual questions with evidential grounding, is crucial for advancing vision-language models (VLMs), but remains unscalable. Existing automated methods are often hindered by two key issues: (1) inconsistent data fidelity due to model hallucinations; (2) brittle verification mechanisms based on simple heuristics. To address these limitations, we introduce AutoVQA-G, a self-improving agentic framework for automated VQA-G annotation. AutoVQA-G employs an iterative refinement loop where a Consistency Evaluation module uses Chain-of-Thought (CoT) reasoning for fine-grained visual verification. Based on this feedback, a memory-augmented Prompt Optimization agent analyzes critiques from failed samples to progressively refine generation prompts. Our experiments show that AutoVQA-G generates VQA-G datasets with superior visual grounding accuracy compared to leading multimodal LLMs, offering a promising approach for creating high-fidelity data to facilitate more robust VLM training and evaluation. Code: https://github.com/rohnson1999/AutoVQA-G
Rongsheng Hu, Runwei Guan, Yicheng Di +2
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China · Thrust of Artificial Intelligence, HKUST(GZ), Guangzhou, China