AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction

Jiashu Yang; Chi Zhang; Abudukelimu Wuerkaixi; Xuxin Cheng; Cao Liu; Ke Zeng; Xu Jia; Xunliang Cai

cs.IRMar 5, 2026

AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction

Authors: Jiashu Yang, Chi Zhang, Abudukelimu Wuerkaixi, Xuxin Cheng, Cao Liu, Ke Zeng, Xu Jia, Xunliang Cai

Organizations: Dalian University of Technology · Meituan LongCat Interaction Team · Tsinghua University

Abstract

Multimodal document question answering requires retrieving dispersed evidence from visually rich long documents and performing reliable reasoning over heterogeneous information. Existing multimodal RAG systems remain limited by two bottlenecks: static retrieval that ignores query complexity, and end-to-end Vision-Language Models (VLMs) that couple visual perception with logical reasoning, leading to inefficient computation and unstable answer generation. We propose AutoThinkRAG, a complexity-aware inference architecture for multimodal document QA. It has two components: (1) a Query Complexity Router that analyzes query difficulty and structure to adaptively select retrieval and reasoning paths; and (2) a Perception--Reasoning Decoupling architecture that uses a lightweight VLM as a high-fidelity visual interpreter to convert query-relevant visual cues into textual representations, which are then passed to an LLM for logical reasoning and answer synthesis. This design improves both efficiency and robustness, especially on long-document and unanswerable queries. Experiments on DocBench and MMLongBench show that AutoThinkRAG achieves 82.13% and 51.29% overall accuracy, respectively, while reducing per-query token consumption by 18.9% and monetary cost by 18.2%. Further analyses show that the gains are most pronounced on complex queries requiring adaptive retrieval and multi-step reasoning.

Figures & tables

Figure 1: Compared with traditional RAG, AutothinkRAG enables more effective and robust long-document question answering through adaptive retrieval and decoupled perception-and-reasoning.

Figure 2: Overview of our AutothinkRAG. The framework consists of four primary stages: (1) Information Extraction : Heterogeneous multimodal data are ingested. (2) Data Processing : Through layout analysis, the system partitions data into text chunks and image indices, followed by an iterative “Extract–Judge” loop to construct a Graph Knowledge Base. (3) Data Linker : A hybrid retrieval mechanism integrates traditional Top–K vector search with graph–based merging to capture complex entity relationships. (4) Reasoning & Response : The Query Router decomposes the original query into sub-queries ( $Q_{1},Q_{2},Q_{3}$ ), which are then processed through a decoupled perception reasoning architecture leveraging both “related text” and visual cues to synthesize the final response.

Doc	Types					Domains			All
Doc	Aca.	Fin.	Gov.	Law.	News	Txt.	Mm.	Una.	All
LLM	41.3	16.3	50.7	49.7	77.3	53.9	20.1	75.8	44.8
MLLM	19.8	16.3	28.4	31.4	46.5	35.7	15.9	39.5	27.7

Table 1: Comparative experimental results of VLM and LLM on DocBench and MMLongBench benchmarks.

Method	Domains					Types			Overall
Method	Aca.	Fin.	Gov.	Law.	News.	Txt.	Mm.	Una.	Overall
AutothinkRAG	80.16	75.40	85.71	87.09	92.50	86.43	77.31	81.25	82.13
Nothink	77.34	73.46	80.59	85.29	88.78	84.08	76.12	77.84	79.37
VLM only	77.03	74.00	81.54	84.39	89.26	85.32	75.09	76.42	79.47
baseline	76.34	72.93	79.77	84.07	84.52	84.65	74.47	75.83	78.02
RAG-Anything	74.90	70.68	77.41	83.06	81.67	84.21	76.36	52.80	75.47

Table 2: Accuracy (%) on DocBench Dataset. Domain categories include Academia (Aca.), Finance (Fin.), Government (Gov.), Legal Documents (Law), and News Articles (News). Document types are categorized as Text-only (Txt.), Multimodal (Mm.), and Unanswerable queries (Una.).

Method	Domains							Overall
Method	Res.	Tut.	Acad.	Guid.	Broch.	Admin.	Fin.	Overall
AutothinkRAG	44.86	52.20	52.26	52.34	51.00	62.19	56.14	51.29
Nothink	43.15	48.55	43.21	52.25	46.00	53.08	50.42	47.21
VLM (direct)	41.43	49.27	43.43	52.25	41.00	54.32	48.72	45.55
VLM (unguided)	42.07	51.12	45.24	52.34	43.00	55.78	50.13	47.21
baseline	40.41	47.82	42.42	50.96	42.00	51.85	46.15	44.86

Table 3: Accuracy (%) on MMLongBench across different domains and overall performance. Domain categories include Research Reports/Introductions (Res.), Tutorials/Workshops (Tut.), Academic Papers (Acad.), Guidebooks (Guid.), Brochures (Broch.), Administration/Industry Files (Admin.), and Financial Reports (Fin.).

Figure 3: Comparison of routing strategy distributions across different document lengths. This figure illustrates the selection proportions of graphs and hypergraphs within length intervals under Think and No-Think modes.

Figure 4: The approach of converting VLM outputs to LLM leads to a marked boost in accuracy when processing multi-page long documents.

Pipeline	Image Description	Summary	Total
Hypergraph	2.7k	60.3k	63.0k
Routing-Enhanced	2.7k	48.3k	51.4k
Savings (Abs.)	-	12.0k	11.6k
Savings (Rel.)	-	19.9%	18.9%

Table 4: Token comparison across stages.

Pipeline	Image Description	Summary	Total
Hypergraph	0.0084	0.1239	0.132
Routing-Enhanced	0.0084	0.0984	0.108
Savings (Abs.)	-	0.0255	0.024
Savings (Rel.)	-	20.6%	18.2%

Table 5: Monetary cost comparison across stages.

Multimodal RAG

Multimodal retrieval-augmented generation grounds AI answers in retrieved text, images, video, audio, and structured visual evidence.

See all 155 papers

Jul 21, 2026

TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering

Long-document multimodal question answering requires more than retrieving relevant chunks from a large document. Different queries require different evidence behavior. Existing multimodal RAG systems improve evidence access through text chunks, page images, graph links, or heterogeneous document elements, but they often apply a largely query-agnostic evidence-use strategy. We present TAP-RAG, a task-aware policy-controlled RAG framework for long-document multimodal QA. TAP-RAG contains a main controller, the Task-Aware Policy Controller (TAPC), and two policy-guided evidence executors: Task-Aware Query-Guided Flow Diffusion (TA-QFD) and Task-Aware Visual Enhancement (TAVE). For each query, TAPC predicts the task prior, estimates visual/local/global evidence signals, and produces an executable policy. TA-QFD then expands textual and structural evidence over the multimodal document graph, while TAVE selectively inspects page images when visual or layout evidence is needed. A guarded synthesis stage fuses text, visual, and structural evidence and abstains when support is insufficient. On DocBench and MMLongBench-Doc, TAP-RAG achieves the best overall accuracy among the compared systems, improving over a matched multimodal-RAG baseline by +9.1 points (61.1 to 70.2) and +4.5 points (42.2 to 46.7), respectively.

Zhong Ji, Keqi Jin, Yan Zhang +1

School of Electrical and Information Engineering, Tianjin University · The International Joint Institute of Tianjin University, Fuzhou, Tianjin University · Department of Automation, Tsinghua University

Multimodal RAG Retrieval-Augmented Generation Long-Context QA Adaptive RAG Multimodal QA Document QA

Ask PDF

Aug 31, 2026

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios. Moreover, processing numerous retrieved images incurs substantial computational overhead from irrelevant visual tokens. To address these challenges, we introduce DocLongRAG, a large-scale dataset of 343K question--answer pairs, each associated with an average of 37.4 retrieved images to reflect authentic RAG workflows. Building on this dataset, we propose Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands question-relevant ones via a lightweight RL-based selector. Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines, achieving state-of-the-art accuracy with significantly lower inference latency. Our resources are available at https://github.com/Collab-Gen/Doc-REFRAG.

Ruofan Hu, Shengyang Xu, Minjie Hong +5

Zhejiang University

Multimodal RAG Efficient Multimodal Inference Multimodal QA Visual Token Compression

Ask PDF

Sep 16, 2026

Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion

Graph-based retrieval-augmented generation (RAG) is widely used for multimodal, cross-document question answering. However, building corpus-level graphs is expensive, slow to query, and difficult to maintain. We present TrioRAG, a graph-free multimodal framework that integrates evidence from three complementary signals: the question, the anchor image, and a VLM-enhanced query generated from both. Each signal retrieves independently over a shared multi-vector index of page text and page images, and the results are combined through late fusion. Further, we introduce AutoQA, a multimodal automotive benchmark whose questions are grounded in noisy, web-sourced images rather than clean document-sourced figures. Its questions require reasoning across manuals. We position it as a model-curated testbed rather than a human-validated gold standard. Across three benchmarks, TrioRAG matches or outperforms graph-based systems while reducing total cost and accelerating per-query inference by 1.6-2.3 times. By construction, AutoQA grounds its questions in out-of-corpus web images. In this setting image retrieval reaches only 19.3% document-level recall, while text-derived signals, especially the VLM-enhanced query, keep retrieval robust.

Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1

University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany

Late Fusion Multimodal RAG Multimodal IR Question Answering Multimodal QA Multimodal Fusion

Ask PDF

Abstract

Multimodal RAG

Explore similar work

TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion