DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering

Zihan Huang; Shihang Wu; Junle Liu; Peirong Zhang; Yongxin Shi; Xuhan Zheng; Lianwen Jin

cs.CVAug 29, 2026

DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering

Authors: Zihan Huang, Shihang Wu, Junle Liu, Peirong Zhang, Yongxin Shi, Xuhan Zheng, Lianwen Jin

Organizations: South China University of Technology

Abstract

Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimodal Large Language Models (MLLMs). Applying restoration tools before Visual Question Answering (VQA) is an intuitive solution. However, existing restoration approaches remain limited, as manually designing and executing restoration strategies is labor-intensive and requires domain expertise. Agentic restoration offers new possibilities for automation, yet existing frameworks primarily target natural images and pursue perceptual quality, overlooking that restoration should serve downstream tasks rather than optimize generic image quality metrics. To this end, we explore the value of agentic restoration for real-world degraded document VQA and propose DocIntent, a training-free Answerability-Guided Agentic Restoration framework. DocIntent first assesses question answerability, then identifies task-relevant degradations and selectively invokes restoration tools. A Comparison-Based Rollback mechanism validates each restoration step and reverts it when question-relevant evidence becomes less decipherable. The entire process requires no additional pretrained degradation classifier or image quality assessment model. Extensive experiments on the WildDoc benchmark show that DocIntent consistently improves the average score and consistency of different open- and closed-source MLLMs. The code and experimental data will be publicly available.

Figures & tables

Figure 1: Comparison between existing real-world document VQA methods and the proposed DocIntent (bottom). The ground truth answer to the question is 212.

Figure 2: Overview of the DocIntent framework with two-phase cyclic mechanism: Phase A (Answerability-Guided Restoration) and Phase B (Comparison-Based Review). The figure illustrates an example where deshadowing triggers rollback (bottom-left), while appearance enhancement succeeds and continues (bottom-right).

Figure 3: Cases where models fail on degraded images but succeed with DocIntent. Left two columns: Qwen3.5-9B Qwen Team (2026a) . Right two columns: Gemini-3.1-Pro Google DeepMind (2026) .

Figure 4: Representative cases where questions can be correctly answered despite visible degradations. D denotes degradation type.

Model	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	ANLS	Consistency	Acc.	Consistency	Acc.	Consistency	Score	Consistency
MiniMonkey-2B Huang et al. (2024)	54.3	22.8	32.3	12.0	31.3	13.4	39.3	16.1
Monkey Li et al. (2024b)	31.0	9.9	22.4	9.8	23.0	11.7	25.5	10.5
TextHarmony Zhao et al. (2024)	37.1	16.0	21.9	10.2	14.7	6.5	24.6	10.9
Llava-Onevision-7B Li et al. (2024a)	52.9	23.7	49.4	20.2	33.9	13.3	45.4	19.1
mPLUG-DocOwl2 Hu et al. (2025)	44.2	19.2	23.3	11.6	22.1	4.8	29.9	11.9

Table 1: Evaluation on the WildDoc Wang et al. (2025a) benchmark. "ANLS" denotes Average Normalized Levenshtein Similarity. Top results are bolded and second-best underlined .

Figure 5: Comparison of restoration strategies. Pipeline blindly applies all tools, MLLM-Judge applies tools to address MLLM-identified degradations, Human Expert uses manually selected tools, and DocIntent performs answerability-guided selective restoration. See Section 4.3 for detailed definitions.

Model	Method	Avg. Tools	Avg. Score	Avg. Cons.
Qwen3.5-9B	No restoration	0.00	68.5	49.7
	Pipeline-random	5.00	63.2	45.7
	Pipeline-fixed	5.00	63.5	47.6
	MLLM-Judge-random	2.78	67.7	48.8
	MLLM-Judge-fixed	2.78	68.5	50.7
	Human Expert	2.35	71.8	56.3

Table 2: Comparison of different restoration strategies on WildDoc benchmark. "Avg." indicates the average results across all three subtasks. Top results are bolded and second-best underlined . Complete results for each subtask are provided in Appendix H .

Model	AG Prompt	AGRes.	CBRb.	Avg. Score	Avg. Cons.
Qwen3.5-9B	✗	✗	✗	68.5	49.7
	✓	✗	✗	67.8	49.2
	✓	✓	✗	69.2	50.5
	✓	✓	✓	71.6	55.6
Gemini-3.1-Pro	✗	✗	✗	83.0	75.5
	✓	✗	✗	82.9	75.7

Table 3: Ablation study on DocIntent components on WildDoc benchmark. "AG Prompt" denotes answerability-guided prompting. The Prompt-only setting enables AG Prompt while disabling AGRes and CBRb. "Avg." indicates the average results across all three subtasks. Complete results for each subtask are provided in Appendix H .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Model	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	ANLS	Consistency	Acc.	Consistency	Acc.	Consistency	Score	Consistency
Qwen3.5-27B Qwen Team (2026a)	83.3	72.8	53.2	43.6	76.6	49.2	71.0	55.2
w/ DocIntent	84.1 +0.8	74.4 +1.6	54.9 +1.7	50.8 +7.2	86.3 +9.7	58.8 +9.6	75.1 +4.1	61.3 +6.1
GPT-4o Hurst et al. (2024)	63.2	41.6	31.9	23.2	59.6	30.8	51.6	31.9
w/ DocIntent	73.7 +10.5	58.4 +16.8	35.8 +3.9	34.4 +11.2	64.2 +4.6	38.4 +7.6	57.9 +6.3	43.7 +11.8
Qwen3.5-Plus Qwen Team (2026a)	84.4	77.0	62.5	52.4	81.5	59.6	76.1	63.0

Appendix

Table 4: Performance of additional MLLMs w/ and w/o DocIntent on WildDoc benchmark Wang et al. (2025a) . "ANLS" denotes Average Normalized Levenshtein Similarity.

Model	Context Strategy	WildDocVQA			WildChartQA			WildTableVQA			Average
Model	Context Strategy	ANLS	Consistency	Tokens	Acc.	Consistency	Tokens	Acc.	Consistency	Tokens	Score	Consistency	Tokens
Qwen3.5-9B	No Context	84.6	75.6	15880	40.7	32.8	29792	71.5	43.4	25826	65.6	50.6	23833
	Full Context	86.1	76.0	27967	54.9	41.6	30735	73.5	46.0	29615	71.5	54.5	29439
	Text-only (Ours)	85.4	76.4	17967	53.6	42.0	19611	75.7	48.4	19518	71.6	55.6	19032
Gemini-3.1-Pro	No Context	92.5	88.0	4904	61.6	56.4	5322	90.7	81.2	4322	81.6	75.2	4849
	Full Context	94.0	92.0	7352	66.3	62.8	7625	92.4	85.7	7147	84.2	80.2	7375

Appendix

Table 5: Comparison of different context management strategies. "ANLS" denotes Average Normalized Levenshtein Similarity. Within each model, top results are bolded and second-best underlined .

Model	Category	WildDocVQA	WildChartQA	WildTableVQA	Average
		ANLS	Acc.	Acc.	Score
Qwen3.5-9B	Baseline	82.5	51.9	71.1	68.5
	Confident	84.6 +2.1	54.4 +2.5	83.7 +12.6	74.2 +5.7
	Forced	78.6 -3.9	48.8 -3.1	57.6 -13.5	61.7 -6.8
Gemini-3.1-Pro	Baseline	92.8	66.7	89.5	83.0
	Confident	95.6 +2.8	70.9 +4.2	91.1 +1.6	85.9 +2.9

Appendix

Table 6: Analysis of tool invocation decisions where all categories employ direct answering w/o DocIntent. "ANLS" denotes Average Normalized Levenshtein Similarity.

Model	Method	WildDocVQA	WildChartQA	WildTableVQA	Average
Model	Method	ANLS	Acc.	Acc.	Score
Qwen3.5-9B	Direct Answer	78.6	48.8	57.6	61.7
Qwen3.5-9B	DocIntent	86.9 +8.3	52.6 +3.8	67.1 +9.5	68.9 +7.2
Gemini-3.1-Pro	Direct Answer	90.3	60.7	87.6	79.5
Gemini-3.1-Pro	DocIntent	92.8 +2.5	63.1 +2.4	91.3 +3.7	82.4 +2.9
Qwen3.5-27B	Direct Answer	72.5	41.6	65.7	59.9

Appendix

Table 7: Comparison on the Forced subset, where DocIntent decides to invoke restoration tools. "ANLS" denotes Average Normalized Levenshtein Similarity.

Model	Method	Avg. Tools	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	Method	Avg. Tools	ANLS	Consistency	Acc.	Consistency	Acc.	Consistency	Score	Consistency
Qwen3.5-9B	No restoration	0.00	82.5	70.8	51.9	33.2	71.1	45.2	68.5	49.7
	Pipeline-random	5.00	78.2	62.4	51.4	32.8	60.0	42.0	63.2	45.7
	Pipeline-fixed	5.00	80.6	69.6	51.0	31.6	58.9	41.6	63.5	47.6
	MLLM-Judge-random	2.78	82.9	68.8	50.1	32.0	70.1	45.6	67.7	48.8
	MLLM-Judge-fixed	2.78	82.7	70.6	51.7	34.8	71.2	46.8	68.5	50.7

Appendix

Table 8: Complete results of restoration strategy comparison on WildDoc benchmark. "ANLS" denotes Average Normalized Levenshtein Similarity. "Avg. Tools" denotes the average number of tool invocations per sample. Top results are bolded and second-best underlined .

Model	AG Prompt	AGRes.	CBRb.	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	AG Prompt	AGRes.	CBRb.	ANLS	Consistency	Acc.	Consistency	Acc.	Consistency	Score	Consistency
Qwen3.5-9B	✗	✗	✗	82.5	70.8	51.9	33.2	71.1	45.2	68.5	49.7
	✓	✗	✗	82.0	70.3	51.0	33.7	70.4	43.7	67.8	49.2
	✓	✓	✗	84.0	69.6	52.1	35.6	71.5	46.4	69.2	50.5
	✓	✓	✓	85.4	76.4	53.6	42.0	75.7	48.4	71.6	55.6
Gemini-3.1-Pro	✗	✗	✗	92.8	88.0	66.7	59.6	89.5	78.8	83.0	75.5

Appendix

Table 9: Complete results of ablation study on DocIntent components. "AG Prompt" denotes answerability-guided prompting. The Prompt-only setting enables AG Prompt while disabling AGRes and CBRb. "ANLS" denotes Average Normalized Levenshtein Similarity. Top results are bolded and second-best underlined .

Model	API Version	Price ($/1M, In/Out)	Avg. Tokens (In/Out)	Cost/Sample ($)
Gemini-3.1-Pro	gemini-3.1-pro-preview	4.00/18.00	5,033/1,073	0.0394
GPT-4o	gpt-4o-2024-08-06	2.50/10.00	12,004/2,468	0.0547
GPT-5.2	gpt-5.2-2025-12-11	1.75/14.00	16,758/2,444	0.0635
Claude Sonnet 4.6	claude-sonnet-4-6	5.00/25.00	11,124/1,504	0.0932
Qwen3.5-Plus	qwen3.5-plus-2026-02-15	0.57/3.43	16,814/2,220	0.0172
Doubao-2.0-Pro	doubao-seed-2-0-pro-260215	1.37/6.86	6,992/933	0.0160

Appendix

Table 10: API versions, token usage, and per-sample inference costs of DocIntent for closed-source MLLMs. Prices are official input/output token prices at the time of evaluation.

Figure 6: Rollback mechanism demonstration. Failed restoration triggers rollback and alternative attempt.

Model	Tools	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	Tools	ANLS	Consistency	Acc.	Consistency	Acc.	Consistency	Score	Consistency
Qwen3.5-9B	Group 1	85.4	76.4	53.6	42.0	75.7	48.4	71.6	55.6
Qwen3.5-9B	Group 2	86.3	76.8	52.5	39.6	73.9	46.8	70.9	54.4
Gemini-3.1-Pro	Group 1	94.1	91.6	67.7	63.6	91.2	84.8	84.3	80.0
Gemini-3.1-Pro	Group 2	94.4	91.2	67.3	62.8	90.7	82.8	84.1	78.9

Appendix

Table 11: Performance comparison of DocIntent with different tools. "ANLS" denotes Average Normalized Levenshtein Similarity. Better results are bolded .

Model	Method	ANLS
Qwen3.5-9B	Direct Answer	95.3
Qwen3.5-9B	DocIntent	94.7
Gemini-3.1-Pro	Direct Answer	95.6
Gemini-3.1-Pro	DocIntent	95.4

Appendix

Table 12: Performance on the clean DocVQA Mathew et al. (2021) test set. Better results within each model are bolded .

Model	Method	WildDocVQA		WildChartQA		WildTableVQA		Average
Model	Method	ANLS	Cons.	Acc.	Cons.	Acc.	Cons.	Score	Cons.
Qwen3.5-9B	Direct Answer	$82.5\pm\text{0.1}$	$70.8\pm\text{0.1}$	$51.9\pm\text{0.3}$	$33.2\pm\text{0.2}$	$71.1\pm\text{0.1}$	$45.2\pm\text{0.2}$	$68.5\pm 0.1$	$49.7\pm 0.1$
Qwen3.5-9B	DocIntent	$\mathbf{85.4\pm\text{0.2}}$	$\mathbf{76.4\pm\text{0.1}}$	$\mathbf{53.6\pm\text{0.4}}$	$\mathbf{42.0\pm\text{0.4}}$	$\mathbf{75.7\pm\text{0.2}}$	$\mathbf{48.4\pm\text{0.1}}$	$\mathbf{71.6\pm 0.3}$	$\mathbf{55.6\pm 0.2}$
Gemini-3.1-Pro	Direct Answer	$92.8\pm\text{0.3}$	$88.0\pm\text{0.2}$	$66.7\pm\text{0.5}$	$59.6\pm\text{0.6}$	$89.5\pm\text{0.2}$	$78.8\pm\text{0.4}$	$83.0\pm 0.3$	$75.5\pm 0.5$
Gemini-3.1-Pro	DocIntent	$\mathbf{94.1\pm\text{0.3}}$	$\mathbf{91.6\pm\text{0.2}}$	$\mathbf{67.7\pm\text{0.6}}$	$\mathbf{63.6\pm\text{0.6}}$	$\mathbf{91.2\pm\text{0.2}}$	$\mathbf{84.8\pm\text{0.2}}$	$\mathbf{84.3\pm 0.4}$	$\mathbf{80.0\pm 0.4}$

Appendix

Table 13: Mean and standard deviation over three runs on the WildDoc Wang et al. (2025a) benchmark. Better results within each model are bolded .

Model	Avg. Tools	WildDocVQA	WildChartQA	WildTableVQA	Average
Qwen3.5-9B	0.00	4943	4858	6137	5313
w/ DocIntent	1.69	17967	19611	19518	19032
Gemini-3.1-Pro	0.00	2019	2226	2882	2376
w/ DocIntent	1.16	5817	6791	5711	6106
Qwen3.5-27B	0.00	4839	4658	5756	5084
w/ DocIntent	1.42	17836	16523	16528	16962

Appendix

Table 14: Average token consumption per sample w/ and w/o DocIntent across different models and tasks.

Figure 7: Prompt for DocIntent on WildDocVQA task.

Figure 8: Prompt for DocIntent on WildChartQA task.

Figure 9: Prompt for DocIntent on WildTableVQA task.

Figure 10: Prompt for Comparison-Based Rollback.

Figure 11: Tools descriptions.

Figure 12: Prompt for Baseline on WildDocVQA task.

Figure 13: Prompt for Baseline on WildChartQA task.

Figure 14: Prompt for Baseline on WildTableVQA task.

Explore similar work

Apr 15, 2026

Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA

Multi-page Document Visual Question Answering requires reasoning over semantics, layouts, and visual elements in long, visually dense documents. Existing OCR-free methods face a trade-off between capacity and precision: end-to-end models scale poorly with document length, while visual retrieval-based pipelines are brittle and passive. We propose Doc-

V^*

, an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation. Doc-

V^*

begins with a thumbnail overview, then actively navigates via semantic retrieval and targeted page fetching, and aggregates evidence in a structured working memory for grounded reasoning. Trained by imitation learning from expert trajectories and further optimized with Group Relative Policy Optimization, Doc-

V^*

balances answer accuracy with evidence-seeking efficiency. Across five benchmarks, Doc-

V^*

outperforms open-source baselines and approaches proprietary models, improving out-of-domain performance by up to 47.9% over RAG baseline. Other results reveal effective evidence aggregation with selective attention, not increased input pages.

Yuanlei Zheng, Pei Fu, Hang Li +9

School of Software Engineering, Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc. · School of Computer Science and Technology, Huazhong University of Science and Technology +1

Document QA VLM Reasoning

Ask PDF

Sep 17, 2026

DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering

Answer grounding in document visual question answering remains an open challenge: most benchmarks lack grounding annotations or provide limited-quality labels, while constructing grounded datasets still requires costly manual effort. We introduce DocAttriBench (DAB), a large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding answers to specific layout elements such as text blocks, tables, and images. To build DAB, we propose a Mask-based Perplexity-Derived Attribution method (MAPPET) that combines document layout and language modeling to identify the most informative element for each answer. MAPPET measures the increase in perplexity after masking candidate elements and attributes the answer to the element contributing most to model confidence. Applying MAPPET to multiple existing Document VQA datasets yields DAB, with 237k documents and 296k question-answer pairs with element-level grounding. We benchmark grounding-capable multimodal LLMs on DAB, evaluating answer accuracy, attribution accuracy, and overall answer quality. Results show that while larger models generally achieve higher answer accuracy, even the strongest models often fail to localize the supporting elements. DAB provides a scalable benchmark for developing grounded, verifiable, and trustworthy Document VQA models. Dataset and code are available at https://aimagelab.github.io/DocAttriBench/.

Luca De Grandis, Silvia Cappelletti, William Raccagni +3

University of Modena and Reggio Emilia, Modena, Italy · University of Modena and Reggio Emilia Modena, Italy · University of Pisa, Pisa, Italy +1

Visual Question Answering Vision-Language Grounding

Ask PDF

Dec 4, 2025

Counterfeit Answers: Adversarial Forgery against OCR-Free Document Visual Question Answering

Document Visual Question Answering (DocVQA) enables end-to-end reasoning grounded on information present in a document input. While recent models have shown impressive capabilities, they remain vulnerable to adversarial attacks. In this work, we introduce a novel attack scenario that aims to forge document content in a visually imperceptible yet semantically targeted manner, allowing an adversary to induce specific or generally incorrect answers from a DocVQA model. We develop specialized attack algorithms that can produce adversarially forged documents tailored to different attackers' goals, ranging from targeted misinformation to systematic model failure scenarios. We demonstrate the effectiveness of our approach against two end-to-end state-of-the-art models: Pix2Struct, a vision-language transformer that jointly processes image and text through sequence-to-sequence modeling, and Donut, a transformer-based model that directly extracts text and answers questions from document images. Our findings highlight critical vulnerabilities in current DocVQA systems and call for the development of more robust defenses. We release our open source code at https://github.com/pralab/adv-docVQA.

Marco Pintore, Maura Pintor, Dimosthenis Karatzas +1

University of Cagliari, Italy · CINI, Italy · Computer Vision Center, UAB, Spain

Visual Question Answering VLM Robustness Adversarial Attacks on VLMs Adversarial Perturbations Document Image Understanding

Ask PDF