Anagent For Enhancing Scientific Table & Figure Analysis

Xuehang Guo; Zhiyong Lu; Tom Hope; Qingyun Wang

cs.CLFeb 10, 2026

Anagent For Enhancing Scientific Table & Figure Analysis

Authors: Xuehang Guo, Zhiyong Lu, Tom Hope, Qingyun Wang

Organizations: College of William & Mary · NIH - National Library of Medicine · The Allen Institute for AI (AI2)

Abstract

In scientific research, analysis requires accurately interpreting complex multimodal knowledge, integrating evidence from different sources, and drawing inferences grounded in domain-specific knowledge. However, current artificial intelligence (AI) systems struggle to consistently demonstrate such capabilities. The complexity and variability of scientific tables and figures, combined with heterogeneous structures and long-context requirements, pose fundamental obstacles to scientific table & figure analysis. To quantify these challenges, we introduce AnaBench, a large-scale benchmark featuring $63,178$ instances from nine scientific domains, systematically categorized along seven complexity dimensions. To tackle these challenges, we propose Anagent, a multi-agent framework for enhanced scientific table & figure analysis through four specialized agents: Planner decomposes tasks into actionable subtasks, Expert retrieves task-specific information through targeted tool execution, Solver synthesizes information to generate coherent analysis, and Critic performs iterative refinement through five-dimensional quality assessment. We further develop modular training strategies that leverage supervised finetuning and specialized reinforcement learning to optimize individual capabilities while maintaining effective collaboration. Comprehensive evaluation across 170 subdomains demonstrates that Anagent achieves substantial improvements, up to $\uparrow 13.43\%$ in training-free settings and $\uparrow 42.12\%$ with finetuning, while revealing that task-oriented reasoning and context-aware problem-solving are essential for high-quality scientific table & figure analysis. Our project page: https://xhguo7.github.io/Anagent/.

Figures & tables

Figure 1: Challenges In Scientific Table & Figure Analysis. The heterogeneity of scientific literature presents great challenges for high-quality analysis of scientific tables and figures (Fig. 6 ).

Figure 2: AnaBench For Evaluating Autonomous Scientific Analysis. We implement four-stage benchmark construction method to build AnaBench , with multi-level filtering to enhance data quality.

Figure 3: Multi-Agent Coordinative Scientific Analysis. Our multi-Agent scientific analysis framework, Anagent , is developed to cover various stages to analyze scientific tables and figures through four collaborative agents: Planner , Expert , Solver , and Critic . Some example details are omitted as [...] for clarity.

Model	Size	Semantic Accuracy (%)			Lexical Accuracy (%)			Overall Accuracy (%)
Model	Size	Cosine	BERT	Meteor	Rouge-L	Bleu	Word	$\bm{S}_{\textbf{{Sem}}}$	$\bm{S}_{\textbf{{Lex}}}$	$\bm{S}_{\textbf{{Avg}}}$
Baselines
GPT-4.1-mini	-	56.34	59.74	19.47	16.74	3.39	11.49	45.18	10.54	27.86
Gemini-2.5-Flash	-	52.41	55.99	19.01	14.90	2.76	9.95	42.47	9.20	25.84
InternVL-3.5	4B	54.38	58.19	18.76	15.67	2.66	9.80	43.78	9.37	26.58
InternVL-3.5	8B	55.73	59.10	19.30	16.80	2.86	10.28	44.71	9.98	27.34

Table 1: Evaluation of Training-Free Agents. Comparing between baselines and training-free Anagent (

M_{e}=5

) on AnaBench (§ C.4 ), relative differences (Eq. 29 ) are shown as positive

\uparrow\Delta_{\textit{rel}}\%

or negative

\downarrow\Delta_{\textit{rel}}\%

Model	Size	Semantic Accuracy (%)			Lexical Accuracy (%)			Overall Accuracy (%)
Model	Size	Cosine	BERT	Meteor	Rouge-L	Bleu	Word	$\bm{S}_{\textbf{{Sem}}}$	$\bm{S}_{\textbf{{Lex}}}$	$\bm{S}_{\textbf{{Avg}}}$
Anagent \| SFT (Zero-Shot)
InternVL-3.5	4B	62.38	64.78	28.38	24.77	13.10	18.90	↑18.43%	↑101.92%	↑33.15%
InternVL-3.5	8B	64.16	65.82	29.62	25.26	14.27	20.56	↑18.99%	↑100.70%	↑33.91%
Qwen2.5-VL	3B	60.84	65.77	25.09	24.27	11.19	17.58	↑15.77%	↑86.30%	↑28.32%
Qwen2.5-VL	7B	63.21	65.97	27.63	24.91	13.96	20.21	↑16.83%	↑97.29%	↑31.75%

Table 2: Evaluation of Finetuned Agents. Comparing baselines (Tab. 1 ) with finetuned Anagent (

M_{e}=5

) on AnaBench (§ C.4 ), relative performance differences (Eq. 29 ) are shown as positive

\uparrow\Delta_{\textit{rel}}\%

or negative

\downarrow\Delta_{\textit{rel}}\%

Figure 4: Few-Shot Learning Optimization (§ 3.3 )

Figure 5: Agent-Level Ability Augmentation (§ 3.3 )

Model	Size	$\bm{S}_{\textbf{{Sem}}}$	$\bm{S}_{\textbf{{Lex}}}$	$\bm{S}_{\textbf{{Avg}}}$
Anagent (Training-Free)
Qwen2.5-VL	3B	46.18	10.91	↑7.37%
Qwen3-VL	4B	46.95	10.50	↑7.36%
Anagent (End-to-End)
Qwen2.5-VL	3B	48.32	14.56	↑18.24%
Qwen3-VL	4B	49.17	14.87	↑19.66%

Table 3: End-to-End Training vs. Modular Training.

Appendix figures & tables54 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Key Error Patterns. Through our case study on 120 samples, we identify seven key error patterns that correspond to the analysis challenges shown in Fig. 1 .

Figure 7: Preliminary Analysis On Failure Patterns. Analyzing on two different data types, table & figure , the agent shows its lack of accurate visual perception and its incapability of writing with proper analysis depth and width.

Figure 8: Preliminary Analysis On Failure Patterns. Analyzing on two data types with different data sources, formats, and domains, the agent generates analysis with significant hallucinated contents, meanwhile incapable of writing with proper analysis depth and width.

Figure 9: Scientific Analysis Workflow. Motivated by how human researchers perform scientific analysis, we decompose the scientific analysis workflow into dedicated stages, which leads to Anagent (Fig. 3 ).

Figure 10: Ablations On Training Data. Performance visualization of ablation studies (§ 5.4 ), respectively on: (a) data size , (b) data domain , (c) data format , (d) data type (§ C.2.1 ).

Figure 11: Year Distribution of AnaBench . Visualization of year distribution, with 2025 comprising the largest proportion to mitigate data contamination (§ C ).

Benchmark

Task

Source

Data Complexity

Reasoning Complexity

M-mod.

M-Layout

M-Doc.

M-Src.

M-Fmt.

M-Dom.

Long-Ctx.

M-Width

M-Depth

M-Obj.

M3SciQA Li et al. (2024)

Partial

\checkmark

\checkmark

\checkmark

\times

\times

\times

\times

\times

\times

\times

SCIDQA Singh et al. (2024)

Full

\times

\times

\checkmark

\times

\times

\times

\checkmark

\times

\times

\times

SCITAB Lu et al. (2023)

Claim Verification

Partial

\times

\times

\times

\times

\times

\times

\times

\times

\times

\times

SCITAT Zhang et al. (2025)

Partial

\times

\times

\times

\times

\times

\times

\times

\times

\times

\times

SPIQA Pramanick et al. (2024)

Full

\checkmark

\checkmark

\times

\checkmark

\times

\times

\checkmark

\times

\times

\times

Appendix

Table 4: Benchmark Comparison. We compare AnaBench with existing scientific benchmarks across multiple dimensions of data complexity (M-mod.: multimodal, M-Layout: multi-layout, M-Doc.: multi-document, M-Src.: multi-source, M-Fmt.: multi-format, M-Dom.: multi-domain) and reasoning complexity (Long-Ctx.: long-context reasoning, M-Width: multi-width, M-Depth: multi-depth, M-ObJ.: multi-objective), highlighting the comprehensiveness of our approach to evaluating autonomous scientific analysis capabilities of MLLM-powered scientific agents. Note: 9/26 denotes 9 domains and 26 subdomains; 9/170 denotes 9 domains and 170 subdomains. Compared benchmarks are listed in alphabetical order.

Curriculum	Broad	Fine-Grained	Category Definition
		Table	Tabular-structured data that organize information into rows and columns to systematically present structured contents, such as method characteristics, parameter configurations, benchmark comparisons, categorical classifications, experimental results, quantitative or qualitative summaries, and analytical breakdowns, enabling precise lookup, comparison, and reference.
	Type	Figure	Visual representations of scientific information that employ graphical, diagrammatic, or illustrative elements, such as plots, charts, schematics, diagrams, images, framework architectures, etc., to show relationships, patterns, processes, structures, conceptual approaches, and other critical information that may be complex or inefficient to express in textual form
		LaTeX	Scientific data obtained from documents in LaTeX, which is a structured format widely used in scholarly publishing to encode mathematical expressions, tables, figures, algorithms, and document structures
Data Complexity	Format	XML	Scientific data obtained from documents encoded in XML, which is a structured, machine-readable format that represents document content, metadata, and hierarchical relationships through tagged elements, commonly used for standardized archival and data exchange in scholarly publishing
		General	Scientific literature that primarily focuses on the development, analysis, or evaluation of a specific method, model, framework, or benchmark, typically proposing novel techniques or reporting empirical results within a defined research problem.
	Source	Review	Scientific literature that systematically examines, summarizes, and synthesizes existing research within a specific domain or research direction, aiming to provide an overview of prior work, identify trends, compare approaches, and highlight open challenges for future research.

Appendix

Table 5: AnaBench Complexity Curriculum. This table summarizes the task classification stage of our benchmark construction method for the complexity curriculum of AnaBench , which defines 23 task complexity categories across seven data complexity and analysis complexity dimensions.

Figure 12: AnaBench Domain Distribution. Domain distribution of AnaBench across 9 broad domain categories (Fig. 13 .a) and 170 fine-grained subdomain categories (Fig. 12 ).

Figure 13: Broad Domain Distribution. Broad domain distribution of (a) AnaBench and (b) the evaluation set of AnaBench across nine broad domain categories.

Figure 14: AnaBench Evaluation Domain Distribution. Besides the domain distribution of AnaBench (Fig. 12 & Fig. 13 .a), we additionally visualize that of the downsampled evaluation set across 9 broad domains (Fig. 13 .b) and 170 subdomains (Fig. 14 ).

Figure 15: AnaBench : Benchmark Statistics. We quantitatively analyze our benchmark in the dimensions of seven challenges (Fig. 1 ) across varying data and analysis complexities.

Broad Domain	Fine-Grained Subdomain
Computer Science (40 Subdomains)	Artificial Intelligence; Hardware Architecture; Computational Complexity; Computational Engineering, Finance, and Science; Computational Geometry; Computation and Language; Cryptography and Security; Computer Vision and Pattern Recognition; Computers and Society; Databases; Distributed, Parallel, and Cluster Computing; Digital Libraries; Discrete Mathematics; Data Structures and Algorithms; Emerging Technologies; Formal Languages and Automata Theory; General Literature; Graphics; Computer Science and Game Theory; Human-Computer Interaction; Information Retrieval; Information Theory; Machine Learning; Logic in Computer Science; Multiagent Systems; Multimedia; Mathematical Software; Numerical Analysis; Neural and Evolutionary Computing; Networking and Internet Architecture; Other Computer Science; Operating Systems; Performance; Programming Languages; Robotics; Symbolic Computation; Sound; Software Engineering; Social and Information Networks; Systems and Control
Economics (3 Subdomains)	Econometrics; General Economics; Theoretical Economics
Electrical Engineering (4 Subdomains)	Audio and Speech Processing; Image and Video Processing; Signal Processing; Systems and Control
Physics (51 Subdomains)	Astrophysics (Cosmology and Nongalactic Astrophysics; Earth and Planetary Astrophysics; Astrophysics of Galaxies; High Energy Astrophysical Phenomena; Instrumentation and Methods for Astrophysics; Solar and Stellar Astrophysics); Condensed Matter (Disordered Systems and Neural Networks; Mesoscale and Nanoscale Physics; Materials Science; Other Condensed Matter; Quantum Gases; Soft Condensed Matter; Statistical Mechanics; Strongly Correlated Electrons; Superconductivity); General Relativity and Quantum Cosmology (General Relativity and Quantum Cosmology); High Energy Physics - Experiment; High Energy Physics - Lattice; High Energy Physics - Phenomenology; High Energy Physics - Theory; Mathematical Physics; Nonlinear Sciences; Nuclear Experiment; Nuclear Theory; Physics (Accelerator Physics; Atmospheric and Oceanic Physics; Applied Physics; Biological Physics; Chemical Physics; Classical Physics; Computational Physics; Data Analysis, Statistics and Probability; Physics Education; Fluid Dynamics; General Physics; Geophysics; History and Philosophy of Physics; Instrumentation and Detectors; Medical Physics; Optics; Plasma Physics; Popular Physics; Physics and Society; Space Physics); Quantum Physics

Appendix

Table 6: AnaBench Domains. This table summarizes the 9 broad domains and 170 subdomains that AnaBench covers and collects data from. Directly summing subdomain categories is inaccurate due to subdomain overlapping.

Broad Domain	Fine-Grained Subdomain
Mathematics (32 Subdomains)	Commutative Algebra; Algebraic Geometry; Analysis of PDEs; Algebraic Topology; Classical Analysis and ODEs; Combinatorics; Category Theory; Complex Variables; Differential Geometry; Dynamical Systems; Functional Analysis; General Mathematics; General Topology; Group Theory; Geometric Topology; History and Overview; Information Theory; K-Theory and Homology; Logic; Metric Geometry; Mathematical Physics; Numerical Analysis; Number Theory; Operator Algebras; Optimization and Control; Probability; Quantum Algebra; Rings and Algebras; Representation Theory; Symplectic Geometry; Spectral Theory; Statistics Theory
Quantitative Biology (10 Subdomains)	Biomolecules; Cell Behavior; Genomics; Molecular Networks; Neurons and Cognition; Other Quantitative Biology; Populations and Evolution; Quantitative Methods; Subcellular Processes; Tissues and Organs
Quantitative Finance (9 Subdomains)	Computational Finance; Economics; General Finance; Mathematical Finance; Portfolio Management; Pricing of Securities; Risk Management; Statistical Finance; Trading and Market Microstructure
Statistics (6 Subdomains)	Applications; Computation; Methodology; Machine Learning; Other Statistics; Statistics Theory
Biomedicine (15 Subdomains)	General Pathology; Infectious Disease; Neurological Disease; Endocrine & Metabolic Disease; Psychiatry; Oncology; Cardiovascular System; Cell Biology; Genetics; Endocrinology; Immunology; Biochemistry; Metabolism; Histology; Virology

Appendix

Table 22

Data Size	Data Type		Data Format		Data Source		Domain
Data Size	Figure	Table	LaTeX	XML	General	Review & Survey	Domain
Single-Format
20,000	16,743	3,258	20,000	0	10,000	10,000	8
24,210	11,778	12,432	0	24,210	22,860	1,350	1
42,804	33,485	9,319	42,804	0	20,000	11,350	8
Multi-Format

Appendix

Table 7: SFT Data Distribution. This table summarizes the data size, data type, data format, and domain categories of different downsampled datasets for SFT.

Agent	Data Type		Data Format		Data Source		Domain
Agent	Figure	Table	LaTeX	XML	General	Review & Survey	Domain
Train
Planner	7,870	2,584	7,632	2,822	5,829	4,625	9
Expert	10,628	2,894	8,058	5,464	7,883	5,639	9
Solver	18,737	6,444	16,331	8,850	16,868	8,313	9
Critic	8,761	4,065	7,655	5,171	7,650	5,176	9

Appendix

Table 8: RL Data Distribution. This table summarizes the data size, data type, data format, and domain categories of the downsampled datasests for agent-wise RL training.

Figure 16: Evaluation Set Data Distribution.

Figure 17: Ablation Studies On Anagent Variants.

Anagent Variants		Tools	Agent Component
Anagent Variants			Planner	Expert	Solver	Critic
Baselines	–	$\times$	$\times$	$\times$	$\times$	$\times$
Omnion	–	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$
Symnion	–	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\times$
Anagent	–	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$
	w/ Critic	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

Appendix

Table 9: Anagent Variants. Overview of Anagent variants.

	$\bm{n_{MLLM}}$
	$\bm{1}$	$\bm{2}$	$\bm{3}$	$\bm{4}$
Evaluation	$A100$ $80GB\times 1$	$A100$ $80GB\times 2$	$A100$ $80GB\times 4$	$A100$ $80GB\times 4$
SFT	$A100$ $80GB\times 1$	–	–	–
RL	$A100$ $80GB\times 8$	–	–	–

Appendix

Table 10: Computation Overhead. This table summarizes the computation overhead during training and evaluation.

n_{MLLM}

denotes the number of different backbone MLLMs.

	Agent				Configuration
	Planner	Expert	Solver	Critic	(All Agents)
	$\bm{M_{p}}$	$\bm{M_{e}}$	$\bm{M_{s}}$	$\bm{M_{c}}$	$\bm{k}$	$\bm{max_{\textit{pixel}}}$	$\bm{min_{\textit{pixel}}}$	$\bm{lr}$	$\bm{n_{\textit{rollout}}}$	$\bm{n_{\textit{epoch}}}$
Evaluation	1	5	2	1	1	$1024\times 1024$	$128\times 128$	–	–	–
SFT	1	1	1	1	1	$512\times 512$	$128\times 128$	$1\times 10^{-4}$	–	$1$
RL	1	1	1	1	1	$512\times 512$	$128\times 128$	$1\times 10^{-6}$	4	$1$

Appendix

Table 11: Implementation Details. This table summarizes the key configuration settings during training and evaluation.

M_{p}

M_{e}

M_{s}

, and

M_{c}

denote the maximum number of turns allowed for Planner, Expert, Solver, and Critic, respectively. The maximum depth of context searching tools is set to

k=1

across all settings. The maximum and minimum pixel values are denoted by

max_{\textit{pixel}}

and

min_{\textit{pixel}}

. The learning rate is represented by

lr

n_{\textit{rollout}}

denotes the number of rollouts during RL, and

n_{\textit{epoch}}

denotes the number of training epochs.

Model	Size	Five-Dimensional Evaluation (%)					Overall Accuracy (%)
Model	Size	$\bm{S}_{\textbf{{Acc}}}$	$\bm{S}_{\textbf{{Complete}}}$	$\bm{S}_{\textbf{{Format}}}$	$\bm{S}_{\textbf{{Clarity}}}$	$\bm{S}_{\textbf{{Faith}}}$	$\bm{S}_{\textbf{{Mllm}}}$
Baselines
InternVL-3.5	4B	42.00	27.17	69.67	64.33	55.33	51.70
InternVL-3.5	8B	52.67	30.50	71.67	68.50	60.83	56.83
Qwen2.5-VL	3B	43.17	24.88	64.50	64.83	56.00	50.68
Qwen2.5-VL	7B	50.83	32.17	73.50	67.50	60.33	56.87

Appendix

Table 12: MLLM-As-Judge Evaluation. Evaluation of scientific table & figure analysis across baselines, training-free Anagent , and finetuned Anagent , through five-dimensional evaluation protocol (§ E.2 ). Compared with baselines, relative performance differences (Eq. 29 ) are shown as positive

\uparrow\Delta_{\textit{rel}}\%

or negative

\downarrow\Delta_{\textit{rel}}\%

Figure 18: Length Distribution. Visualization of Data Length in AnaBench . (1) Left: Input data length. (2) Center: Core context length. (3) Right: Ground-truth analysis length.

Figure 19: Length Distribution of GPT-4.1-mini. Using GPT-4.1-mini as the base model, this figure visualizes the length distribution of 150 randomly sampled agent-generated analysis versus ground-truth analysis length.

Figure 20: Length Distribution of Qwen3-VL-8B. Leveraging Qwen3-VL-8B, this figure visualizes the length distribution of 150 randomly sampled agent-generated analysis versus ground-truth analysis length.

Specialized Tools	Supported Inputs & Formats	Functions
Document Toolkit
Online Fetcher	URL (arXiv, Semantic Scholar, PubMed, general web), Source Type & Preferred Format (HTML, LaTeX, XML, PDF), DOI, Paper ID, Search Query	Fetch and parse documents from online sources, including arXiv papers (LaTeX/PDF), Semantic Scholar (search/metadata), PubMed Central (XML), and general URLs (HTML/PDF/XML)
PDF Parser	URL, Local Path, Bytes	Parse PDF documents to extract text, metadata, hierarchical contents with optional image saving
XML Parser	URL, Local Path, XML Strings, Bytes	Parse XML documents and extract metadata, hierarchical contents with configurable detail levels
Knowledge Toolkit
Abstract Collector	Title, Author, Year, URL, DOI, arXiv ID, PMID, Local Path	Search and extract paper abstracts from multiple academic sources (e.g., arXiv, PubMed, Semantic Scholar, CrossRef)

Appendix

Table 13: Five Specialized Toolkits of Anagent . To enhance specialized scientific research capabilities, Anagent is equipped with five complementary toolkits that respectively support document comprehension, knowledge retrieval, targeted searching, multimodal understanding, and autonomous coding exploration.

Figure 21: Ablation Studies On Additional Contexts & Domain Knowledge. Comparison between with ("w/" in the figure) and without gold contextual information and domain-specific knowledge .

Figure 22: When AI Scientists Meet Stated Analysis Expectations. Our experiments investigate AI agents’ capabilities in properly comprehending and implementing scientific analysis with explicit expectations.

Figure 23: Ablation Study on Data Type. Evaluation of agent performance across different data types (§ C.2.1 ).

Figure 24: Ablation Study on Data Format. Evaluation of agent performance across different data formats (§ C.2.1 ).

Figure 25: Ablation Study on Data Source. Evaluation of agent performance across different data sources (§ C.2.1 ).

Figure 26: Ablation Study on Data Domain. Evaluation of agent performance across nine broad domains (§ C.2.1 ).

Figure 27: Ablation Study on Input Coverage. Evaluation of agent performance across six different input coverage settings. S-T represents single-table input, S-F represents single-figure input, M-T represents multi-table input, and M-F represents multi-figure input.

Figure 28: Ablation Study on Analysis Width. Evaluation of agent performance across four analysis width curriculum (§ C.2.2 ).

Figure 29: Ablation Study on Analysis Depth. Evaluation of agent performance across two analysis depth: shallow analysis and in-depth analysis (§ C.2.2 ).

Figure 30: Ablation Study on Analysis Objective. Evaluation of agent performance across two analysis objectives: methodology-oriented analysis and experiment-oriented analysis (§ C.2.2 ).

Figure 31: Domain Evaluation. Assessing 100 samples manually for each model on each domain, we compare the evaluation results between baseline and Anagent using the same base model (Qwen3-VL-8B).

Dimension	MLLM Judge (%)		Human Expert (%)			Agreement & Significance
Dimension	Baseline	Anagent	Baseline	Anagent	Gold	Krippendorff’s $\bm{\alpha}$	Wilcoxon $\bm{p}$
Five-Dimensional Evaluation Protocol
Analysis Consistency	48.23	59.34	52.57	64.52	92.78	0.92	$<0.001$
Query-Analysis Alignment	29.32	33.03	32.63	37.42	90.28	0.87	$<0.001$
Knowledge Utilization	67.63	78.54	71.24	83.15	93.72	0.88	$<0.001$
Format Correctness	70.22	76.36	72.18	80.60	99.37	0.97	$<0.001$

Appendix

Table 14: Multi-Domain Human Expert Evaluation. Three experts per domain rate agent-generated and gold analyses across six domains under the five-dimensional protocol (§ E.2 ). Gold shows expert ratings of AnaBench ’s author-written reference analyses under the identical protocol.

Figure 32: Anagent Tool Utilization. Investigation on tool utilization of training-free Anagent and finetuned Anagent .

Figure 33: Anagent Tool Utilization Strategy Optimization Through Finetuning. Optimization of tool utilization strategy through finetuning.

Method	MLLM Calls	Tokens/Call	Latency	$\bm{S}_{\textbf{{Avg}}}$	$\bm{\Delta}_{\textbf{{Rel}}}$
Single-Model Test-Time Scaling
Direct Generation	1.00	1.00 $\times$	1.00 $\times$	26.76	–
Self-Consistency (SC@10)	10.00	1.18 $\times$	7.09 $\times$	26.82	↑0.22%
Best-of- $N$ (BoN@10) †	10.00	1.07 $\times$	7.11 $\times$	27.03	↑1.01%
Self-Refine (SR@10)	10.00	4.06 $\times$	7.47 $\times$	26.79	↑0.11%
Multi-Agent (Training-Free)

Appendix

Table 15: Controlled Test-Time Scaling & Compute Accounting. All experiments use the same Qwen3-VL-4B backbone and the same evaluation set (§ G.4 ). MLLM calls, tokens per call, and latency are averaged per instance, with tokens per call and latency calculated relative to single-pass direct generation. Relative differences (Eq. 29 ) are shown as positive

\uparrow\Delta_{rel}\%

or negative

\downarrow\Delta_{rel}\%

. † BoN@10 selects the candidate with the highest

S_{\textsc{Avg}}

against the reference analysis, and is therefore an oracle upper bound rather than a deployable selection rule.

Figure 34: Example of Anagent Scientific Analysis Writing. This example ( $S_{\textsc{AVG}}=72.41\%$ ) shows a complete end-to-end scientific analysis process of Anagent , illustrating the full scientific analysis writing workflow from task planning, searching and executing, context-aware problem-solving, to iterative reflection and refinement. Input query and agent prompts are provided in § D.2 and omitted here for clarity.

Figure 35: Examination On Error Distribution. Comparison of the seven key error patterns (Fig. 6 ) between baseline and Anagent on the same set. Similar to Fig. 6 , the Y axis represents “Error Rate (%)" from low to high.

Figure 36: Example of Multi- Modality & Layout Perception Error. This example ( $S_{\textsc{AVG}}=25.64\%$ ) illustrates a scientific analysis failure case with multi- modality & layout perception errors (§ I.1 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 37: Example of Hallucination Error. This example ( $S_{\textsc{AVG}}=21.85\%$ ) illustrates a scientific analysis failure case with hallucination errors (§ I.2 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clearer presentation.

Figure 38: Example of Domain-Specific Error. This example ( $S_{\textsc{AVG}}=11.28\%$ ) illustrates a scientific analysis failure case with multi- modality & layout perception errors (§ I.3 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 39: Example of Long-Context Understanding Error. This example ( $S_{\textsc{AVG}}=24.43\%$ ) illustrates a scientific analysis failure case due to long-context understanding errors (§ I.4 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 40: Example of Analysis Width Error. This example ( $S_{\textsc{AVG}}=25.96\%$ ) illustrates a scientific analysis failure case due to analysis width errors (§ I.5 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 41: Example of Analysis Depth Error. This example ( $S_{\textsc{AVG}}=31.24\%$ ) illustrates a scientific analysis failure case due to analysis depth errors (§ I.6 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 42: Example of Analysis Objective Error. This example ( $S_{\textsc{AVG}}=31.45\%$ ) illustrates a scientific analysis failure case due to analysis objective errors (§ I.7 ). Input query and agent prompts are provided in § D.2 and omitted here for clarity. Other intermediate outputs and contexts are also omitted for clear presentation.

Figure 43: Example of Other Errors Due To MLLM Backbone Ability Constraints. This figure shows two scientific analysis failure cases due to MLLM backbone ability constraints (§ I.8 ): (a) Repetitive Content, (b) Intermediate Solution. Input query and agent prompts are provided in § D.2 and omitted here for clarity. Some contents denoted as [...] are omitted for presentation brevity.

Scientific Figure Understanding

See all 17 papers

Jun 10, 2026

Benchmarking AI Agents for Addressing Scientific Challenges Across Scales

AI agents are increasingly being developed to accelerate scientific discovery, yet their practical capabilities in real research settings remain poorly understood. Existing benchmarks for AI agents rarely capture the complexity, heterogeneity, and extended reasoning required by scientific work, whereas benchmarks for scientific tasks often reduce research to static, direct problems and provide limited support for interactive evaluation. Here, we introduce SciAgentArena, a systematic benchmark for evaluating AI agents in real-world scientific research scenarios drawn from emerging needs across multiple domains. SciAgentArena comprises approximately 200 tasks with stepwise verification and an interactive, agent-agnostic environment for assessing diverse AI agents. Using this benchmark, we find that current agents can contribute effectively to well-specified data-analysis workflows, particularly when the task structure and evaluation criteria are clear. However, their performance remains uneven across scientific contexts: agents struggle to generate genuinely novel insights, sustain self-directed exploration, and formulate robust solutions for open-ended research questions. We further characterize common failure modes across agents and identify opportunities for improving their reliability, autonomy, and scientific reasoning. Together, SciAgentArena provides a practical framework for measuring progress in AI agents for science and for guiding the design of future agents capable of addressing complex scientific challenges. Full codes, tasks, and datasets can be accessed via this link: https://sciagentarena.github.io/.

Tianyu Liu, Allen Xin Wang, Antonia Panescu +30

Yale University, CT, USA · The Pennsylvania State University, PA, USA · Northeastern University, MA, USA +7

AI for Science Scientific Agent Benchmarks AI Agent Evaluation AI Agent Benchmarks Autonomous Scientific Discovery

Ask PDF

Jul 18, 2026

From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data

Scientists at European XFEL conduct experiments that generate very large and complex datasets. The subsequent data analysis is challenging as scientists must combine their domain expertise with facility- and software-specific knowledge scattered across documentation, tools, and support channels. To address this problem, we designed and evaluated an agentic AI system tailored to the scientists' needs and integrated with the high-performance computing environment of European XFEL. Using a design science research approach, we conducted a rapid literature review, a systematic evaluation of 16 AI tools, multiple interviews, a focus group, and a user study with experts at European XFEL to develop and evaluate two prototypes. Our study identifies key knowledge challenges in scientific data analysis, derives requirements for an AI agent that supports knowledge retrieval and source code generation, and proposes design recommendations for a specialized system adaptable to the evolving AI tool landscape. These findings provide guidance for developing maintainable AI support in highly specialized scientific environments.

Tim Fuchs, Luca Gelisio, Steffen Hauf +1

University of Hamburg Hamburg, Germany · European XFEL Schenefeld, Germany · Hasso Plattner Institute University of Potsdam Potsdam, Germany

AI for Science AI-Assisted Scientific Research Scientific Code Generation Information Retrieval Scientific Document Retrieval

Ask PDF

Mar 31, 2026

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visualization (SciVis) tasks. Despite rapid progress, the community lacks a principled and reproducible benchmark for evaluating these emerging SciVis agents in realistic, multi-step analysis settings. We present SciVisAgentBench, a comprehensive and extensible benchmark for evaluating scientific data analysis and visualization agents. Our benchmark is grounded in a structured taxonomy spanning four dimensions: application domain, data type, complexity level, and visualization operation. It currently comprises 108 expert-crafted cases covering diverse SciVis scenarios. To enable reliable assessment, we introduce a multimodal outcome-centric evaluation pipeline that combines LLM-based judging with deterministic evaluators, including image-based metrics, code checkers, rule-based verifiers, and case-specific evaluators. We also conduct a validity study with 12 SciVis experts to examine the agreement between human and LLM judges. Using this framework, we evaluate representative SciVis agents and general-purpose coding agents to establish initial baselines and reveal capability gaps. SciVisAgentBench is designed as a living benchmark to support systematic comparison, diagnose failure modes, and drive progress in agentic SciVis. The benchmark is available at https://scivisagentbench.github.io/.

Kuangshi Ai, Haichao Miao, Kaiyuan Tang +13

University of Notre Dame · Lawrence Livermore National Laboratory · University of Utah +4

Scientific Visualization Data Visualization LLM Agent Evaluation AI Agent Benchmarks

Ask PDF

Abstract

Scientific Figure Understanding

Explore similar work

Benchmarking AI Agents for Addressing Scientific Challenges Across Scales

From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents