Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets

Aashish Panta; Hugo Lee; Giorgio Scorzelli; Kyongsik Yun; Valerio Pascucci

cs.HCAug 22, 2026

Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets

Authors: Aashish Panta, Hugo Lee, Giorgio Scorzelli, Kyongsik Yun, Valerio Pascucci

Organizations: University of Utah Salt Lake City, Utah, USA · Jet Propulsion Laboratory, Caltech Pasadena, CA, USA

Abstract

Modern scientific facilities and instruments generate datasets at scales that are difficult for individual researchers to discover, access, and explore. Although many datasets are publicly available, using them often requires familiarity with repository organization, data formats, multiresolution structures, and visualization parameters. We present WebVisus, a constrained and resource-aware multi-agent system for discovering and autonomously exploring remote, multiresolution scientific datasets. Given a natural-language research question, WebVisus identifies the user's intent and launches an autonomous exploration agent that examines slices, volumes, and timesteps while adapting data resolution and retrieval quality to available client memory and computational resources. This design supports progressive exploration without complete dataset downloads or manual configuration of low-level visualization parameters using natural languages. We report the system architecture, constrained agent protocol, resource-aware access mechanism, and case studies evaluating autonomous visual exploration and resource-aware agentic access across scientific datasets.

Figures & tables

Fig. 1: WebVisus interface for interactive and autonomous exploration of large volumetric data. The left panel provides manual control over slices, transfer functions, isovalues, rendering quality, and camera motion; the center displays the resulting 3D volume and orthogonal slice views; and the right panel shows the agent’s observations and findings. During autonomous exploration, the agent iteratively modifies slices along all three axes, adjusts isovalue and rendering parameters, rotates the view, and evaluates the resulting visual evidence. These operations are supported by resource-aware progressive retrieval, which fetches only the data resolution and region required for each analysis step.

Fig. 2: WebVisus system architecture. Users explore scientific volumes through interactive and autonomous operating modes that share a three-agent runtime consisting of an Orchestrator, Catalog Agent, and Executor. The interactive assistant handles turn-by-turn requests and can initiate an autonomous science loop that repeatedly observes the current analysis state, plans and executes visualization operations, and evaluates the resulting evidence. Analysis requests are mediated by a resource-aware progressive-access layer that plans data requests, incrementally retrieves multiresolution data, and adapts execution to available computational resources before returning progressive results to the visualization workspace. Application, model, and scientific data services provide system functionality, language-model inference, and access to multiresolution data archives, respectively.

Fig. 3: Autonomous exploration of a materials-science tomography volume. (a) The default rendering shows little internal contrast. (b) At step 2 of 12, the agent adds orthogonal slices and begins adjusting the visual representation. (c) At step 8, combined slices and an isovolume reveal spatial variation that is not apparent in the initial view. (d) At step 12, adjusted opacity, isovalue, and slice positions expose a candidate internal boundary and support a final exploration summary. No intermediate visualization parameters were selected by the user.

Fig. 4: Progressive access time across six scientific datasets. Megabyte scale representations support repeated observations, while larger transfers introduce substantial and dataset dependent delays. WebVisus uses measured latency and estimated memory cost to guide subsequent access decisions.

ID	Objective	Dataset	Agent Output	Time	Tokens	Outcome
T1	Find interesting patterns in the current data	amlan-tomography-1	Isovolume and orthogonal slices revealed an elliptical ring, prism shaped geometry, and granular alignments	172 s	90 k	Success
T2	Find a three dimensional tomography or micro CT dataset and inspect its internal structure within 2 MB	4D_micro_CT_sandstones	A central $Z$ slice and isovolume revealed high attenuation grains within a lower density matrix and visible pore regions	25 s	12 k	Success
T3	Identify the current dataset and produce an internal overview	Tomography	Multiple slices and an isovolume revealed concentric radial bands and granular texture	99 s	52 k	Success
T4	Characterize pore and solid regions using slices and an isovolume	2kbit1	Multiple isovalue bands and orthogonal slices revealed pore regions within a connected solid matrix	153 s	88 k	Success
T5	Find a time varying atmospheric volume and inspect its spatial and temporal structure within 2 MB	NASA DYAMOND Data	Four temporal samples with slices and an isovolume revealed evolving convective and frontal structures	89 s	51 k	Success

TABLE I: Outcomes of five autonomous exploration workflows.

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 288 papers

Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Tasks
May 20, 2026Haichao Miao, Zhimin Li, Kuangshi Ai +4Scientific Visualization Data Visualization
Lawrence Livermore National Laboratory · Vanderbilt University · University of Notre Dame
SciDataSailor: Deep Scientific Data Exploring
Jul 29, 2026Jiyong Rao, Yicheng Qiu, Chi Zhang +2Scientific QA AI Agent Benchmarks
Shanghai Artificial Intelligence Laboratory · Tongji University
Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale
Aug 20, 2026Xiaohan Huang, Qingqing Long, Xiaolei Du +8Knowledge Representation LLM Agent Skill Retrieval
Computer Network Information Center, Chinese Academy of Sciences University of the Chinese Academy of Sciences Beijing, China

May 20, 2026J/K move · Enter open · S save

Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Tasks

Haichao Miao, Zhimin Li, Kuangshi Ai, Kaiyuan Tang+3

Lawrence Livermore National Laboratory · Vanderbilt University · University of Notre Dame

Scientific Visualization Data Visualization AI Agents for Scientific Discovery Agentic Workflows Visual Analytics Long-Horizon Agent Tasks+1

Open paper AskPDF arXiv

The ability to inspect, interpret, and communicate complex data is crucial for virtually any scientific endeavor, but often requires significant expertise outside the core domain ranging from data management and analysis to visualization design and implementation. We present an end-to-end agentic harness that, based on only the data and a high level description of the tasks, independently designs custom visual analysis applications (VIS apps). This represents an important step towards a general AI co-scientist envisioned by many as an autonomous system that can autonomously execute long horizon tasks based on high-level directions. Our proposed VIS co-scientist is an essential component of this broader AI co-scientist vision: a harness that can autonomously analyze data and design visualization solutions using a collection of agents and specialized skills that coordinate exploratory analysis, plan, configure the environment, implement, validate the interface, and most importantly evaluate the overall task completion. Each stage produces document and instruction artifacts that guide downstream work and enable iterative refinement. We validate this approach on IEEE SciVis Contests spanning multiple science and engineering fields. These contests serve as ideal proving grounds because they encode real-world complexity: ambiguous requirements, diverse data modalities, design trade-offs, and task-driven validation. Given only the data and target tasks, our system autonomously produces functional single-page VIS Apps with verified linked-view behavior, highly customized to domain experts' specified tasks and needs.

Abstract

AI Agents for Scientific Discovery

Explore similar work

Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Tasks

SciDataSailor: Deep Scientific Data Exploring

Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale