Towards Knowledgeable Deep Research: Framework and Benchmark
Authors: Wenxuan Liu, Zixuan Li, Long Bai, Chunmao Zhang, Fenghui Zhang, Zhuo Chen, Wei Li, Yuxin Zuo, +8 more
Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Also with University of Chinese Academy of Sciences. · National University of Singapore, Singapore, Singapore
Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive reports. In contrast to existing studies that mainly focus on unstructured web content, a more challenging DR task should additionally utilize structured knowledge to provide a solid data foundation, facilitate quantitative computation, and lead to in-depth analyses. In this paper, we refer to this novel task as Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge. Furthermore, we propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that reasons over both kinds of knowledge and integrates the texts, figures, and tables into coherent multimodal reports. The key design is the Structured Knowledge Analyzer, which utilizes both coding and vision-language models to produce figures, tables, and corresponding insights. To support systematic evaluation, we construct KDR-Bench, which covers 9 domains, includes 41 expert-level questions, and incorporates a large number of structured knowledge resources (e.g., 1,252 tables). We further annotate the main conclusions and key points for each question and propose three categories of evaluation metrics including general-purpose, knowledge-centric, and vision-enhanced ones. Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics, and even surpasses the Gemini DR agent on vision-enhanced metrics, highlighting its effectiveness in deep, structure-aware knowledge analysis. Finally, we hope this work can serve as a new foundation for structured knowledge analysis in DR agents and facilitate future multimodal DR studies.
Figures & tables
Figure 1. Demonstration of the proposed HKA framework.
Figure 2. The construction procedure of KDR-Bench: (a) Dataset construction process; (b) Evaluation framework.
Figure 3. The statistics on tables in KDR-Bench.
General-Purpose
Knowledge-Centric
Methods
Avg.
Comp.
Depth
Coher.
Read.
Main.
Key.
Support.
LLM with search tools
Hunyuan2.0
40.6
40.9
38.8
41.4
41.2
61.0
46.7
-
GLM4.6
40.8
40.8
36.4
42.6
43.5
59.1
40.1
-
Qwen3-Max
44.6
43.9
40.5
46.6
47.4
64.2
43.5
-
Minimax-M2
44.7
44.8
41.1
46.8
46.0
58.2
50.5
-
Table 1. Results on General-purpose and Knowledge-centric metrics. The best results are in boldface, and the second-best results are underlined.
Methods
Agr.
P&E
E&E
F&I
M&E
Soc.
Art
Tech.
Trans.
Avg.
GLM4.6
91.7
100.0
100.0
100.0
100.0
100.0
83.3
100.0
100.0
97.6
Minimax-M2
100.0
80.0
100.0
83.3
100.0
83.3
100.0
100.0
100.0
92.2
Perplexity
66.7
91.7
60.0
66.7
70.0
83.3
66.7
50.0
100.0
72.0
Grok
66.7
100.0
100.0
58.3
80.0
83.3
66.7
100.0
100.0
81.7
Gemini
50.0
66.6
60.0
41.7
60.0
50.0
66.7
50.0
100.0
56.1
Enterprise
66.7
100.0
90.0
66.7
100.0
83.3
83.3
66.7
100.0
82.9
Table 2. Results on Vision-Enhanced metrics. Greener cells indicate a larger advantage for HKA, while redder cells indicate a larger disadvantage for HKA.