Exploring Robust Multi-Agent Workflows for Environmental Data Management

Boyuan Guan; Jason Liu; Yanzhao Wu; Kiavash Bahreini

cs.AIApr 2, 2026

Exploring Robust Multi-Agent Workflows for Environmental Data Management

Authors: Boyuan Guan, Jason Liu, Yanzhao Wu, Kiavash Bahreini

Organizations: Florida International University, Miami, Florida, United States

Abstract

Embedding LLM-driven agents into environmental FAIR data management is compelling - they can externalize operational knowledge and scale curation across heterogeneous data and evolving conventions. However, replacing deterministic components with probabilistic workflows changes the failure mode: LLM pipelines may generate plausible but incorrect outputs that pass superficial checks and propagate into irreversible actions such as DOI minting and public release. We introduce EnviSmart, a production data management system deployed on campus-wide storage infrastructure for environmental research. EnviSmart treats reliability as an architectural property through two mechanisms: a three-track knowledge architecture that externalizes behaviors (governance constraints), domain knowledge (retrievable context), and skills (tool-using procedures) as persistent, interlocking artifacts; and a role-separated multi-agent design where deterministic validators and audited handoffs restore fail-stop semantics at trust boundaries before irreversible steps. We compare two production deployments. The University's GIS Center Ecological Archive (849 curated datasets) serves as a single-agent baseline. SF2Bench, a compound flooding benchmark comprising 2,452 monitoring stations and 8,557 published files spanning 39 years, validates the multi-agent workflow. The multi-agent approach improved both efficiency - completed by a single operator in two days with repeated artifact reuse across deployments - and reliability: audited handoffs detected and blocked a coordinate transformation error affecting all 2,452 stations before publication. A representative incident (ISS-004) demonstrated boundary-based containment with 10-minute detection latency, zero user exposure, and 80-minute resolution. This paper has been accepted at PEARC 2026.

Figures & tables

Figure 1. The “fail-open” problem under composition. LLM agents can succeed in bounded, reversible tasks (left), but reliability degrades across end-to-end publication workflows with irreversible steps (center). Core LLM limitations compound under composition (right), yielding multiplicative reliability decay $p^{n}$ (bottom). At production scale, silent failures create irreversible contamination and trust collapse.

Figure 2. System design overview. Top: Three-Track artifact store externalizing governance (Track 1), semantic context (Track 2), and executable procedures (Track 3) that interlock at execution time . Bottom: multi-agent operating model with role-separated agents (data preparation, publishing, platform operations) connected by audited handoffs (H1/H2) with deterministic validators before state-changing publication.

Figure 3. Three-Track artifact graph and an enforceable “do-not-guess” gate. Left: the full artifact graph with typed nodes for Behaviors (Track 1), Knowledge (Track 2), and Skills (Track 3) used during production. Right: zoomed interlock between a metadata-creation workflow (Track 3) and an authoritative mapping source behavior (Track 1). The behavior enforces that when filenames do not align, the workflow must obtain an authoritative mapping rather than guessing.

Figure 4. Audited handoff protocol. Each agent-to-agent transition follows four phases: prepare (package outputs with provenance), validate (deterministic gates), approve (record and escalate on failure), and commit (apply state changes). Failures block downstream execution. Inset: production incident ISS-004 , from boundary detection to verified resolution.

Table	Purpose	Backs
messages	Inter-agent MCP messages	Incident trail ( ISS-* )
supervisor_actions	Supervisory/commit decisions	Commit receipts
processing_logs	Per-dataset lifecycle	Dataset states
task_phases	Phase-level progress	Phased publication
audit_runs	Audit execution records	Auditor independence
llm_traces	Model-call provenance	Action reproducibility

Table 1. Control database: representative audit-trail tables.

Requirement / Metric	Case 1 (GIS)	Case 2 (SF2Bench)
DG1: Scale without coordination overhead (E1)
Developer workload	1 + 2 collab.	1 researcher
Primary curation unit	Dataset	Monitoring station dataset
Datasets processed	849	2,452
Files published	849	8,557
Time to completion	Weeks	2 days

Table 2. Evaluation evidence mapped to operational requirements.

Metric	Case 1 (GIS)	Case 2 (SF2Bench)
Primary scale	849 datasets (ecological archive datasets)	2,452 station datasets; 8,557 files
Completion time	Weeks	$\approx$ 2 days
Supervision placement	Per-step review	Boundary-only (audited handoffs)
Same-model audit differential	N/A	0/4 self-detections vs. 4/4 audit detections
Representative prevented incident	Manual QA/QC	ISS-004 blocked before publication (caught at first spatial validation; 0 user exposure)

Table 3. Hero summary: single-agent baseline vs. MAS operating model. Cases reflect sequential deployments under different workloads, not a controlled A/B comparison; differences are attributed to architectural properties (role separation, boundary validation).

ID	Outcome / type	Boundary point	Key metric(s)
ISS-004	Coordinate field mismatch $\rightarrow$ spatially implausible GeoJSON	Pre-publication audit	Caught at first spatial validation; 2,452 stations impacted; 0 user exposure; commit blocked
ISS-005	Root cause diagnosed; corrected GeoJSON regenerated	Remediation response	5 files regenerated with correct coordinate fields ( X COORD / Y COORD )
ISS-007	Deliverable placement corrected (shared network drive)	Handoff delivery/commit	Ensured publishing role consumed the audited artifact version (no local-only paths)
ISS-009	Independent verification of coordinate fix	Pre-republish verification	Quantitative range checks logged for 5 layers; republish approved
ISS-003	Completion audit (Phase 5A)	Boundary report	Verified 15GB uploaded; 8,557 files processed

Table 4. Representative audited boundary outcomes from the SF2Bench production audit trail.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,040 papers

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses
Sep 28, 2026Huachi Zhou, Yujing Zhang, Jiahe Du +7LLM Agent Harnesses LLM Agent Reliability
The Hong Kong Polytechnic University, Hong Kong, China · Jilin University, Changchun, China
A Hierarchical Multi-Agent System for Autonomous Discovery in Geoscientific Data Archives
Feb 24, 2026Dmitrii Pantiukhin, Ivan Kuznetsov, Boris Shapkin +3Earth Science Data Analysis Agents
Alfred Wegener Institute for Polar and Marine Research, Bremerhaven, Germany
Agentic Data Environments
Jul 8, 2026Elaine Ang, Chenxi Huang, Georgios Liargkovas +13AI Agent Reliability AI Agent Safety
Columbia University

Sep 28, 2026J/K move · Enter open · S save

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses

Huachi Zhou, Yujing Zhang, Jiahe Du, Jiacheng Cai+6

The Hong Kong Polytechnic University, Hong Kong, China · Jilin University, Changchun, China

LLM Agent Harnesses LLM Agent Reliability Data Analysis Agents

Open paper AskPDF arXiv

Large language model agents are increasingly deployed for data-intensive work, yet reliable data analysis requires more than general-purpose reasoning and ad hoc tool augmentation. Data Agents, equipped with workflow harnesses, offer a promising paradigm for automating the end-to-end data science lifecycle. This paper examines Data Agents from a harness-centric perspective. First, we introduce a taxonomy of Data Agents and associated data environments, organizing the literature around five functional stages: perception, planning, execution, verification, and repair. Second, we analyze the key technical routes within each stage, identifying 15 distinct approaches ranging from data structure probing to data state reconstruction. Third, we identify four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository. These problems explain why silent failures can persist even when individual components function correctly, highlighting the need for rigorous workflow harnesses and shared reliability resources. Finally, we summarize the horizontal task families of Data Agents, examine their vertical application settings, and benchmarks for evaluation, while maintaining a companion repository at https://github.com/DEEP-PolyU/Awesome-Data-Agents.

Abstract

Multi-Agent LLM Systems

Explore similar work

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses

A Hierarchical Multi-Agent System for Autonomous Discovery in Geoscientific Data Archives

Agentic Data Environments