cs.AIApr 2, 2026

Exploring Robust Multi-Agent Workflows for Environmental Data Management

Authors: Boyuan Guan, Jason Liu, Yanzhao Wu, Kiavash Bahreini

Organizations: Florida International University, Miami, Florida, United States

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Embedding LLM-driven agents into environmental FAIR data management is compelling - they can externalize operational knowledge and scale curation across heterogeneous data and evolving conventions. However, replacing deterministic components with probabilistic workflows changes the failure mode: LLM pipelines may generate plausible but incorrect outputs that pass superficial checks and propagate into irreversible actions such as DOI minting and public release. We introduce EnviSmart, a production data management system deployed on campus-wide storage infrastructure for environmental research. EnviSmart treats reliability as an architectural property through two mechanisms: a three-track knowledge architecture that externalizes behaviors (governance constraints), domain knowledge (retrievable context), and skills (tool-using procedures) as persistent, interlocking artifacts; and a role-separated multi-agent design where deterministic validators and audited handoffs restore fail-stop semantics at trust boundaries before irreversible steps. We compare two production deployments. The University's GIS Center Ecological Archive (849 curated datasets) serves as a single-agent baseline. SF2Bench, a compound flooding benchmark comprising 2,452 monitoring stations and 8,557 published files spanning 39 years, validates the multi-agent workflow. The multi-agent approach improved both efficiency - completed by a single operator in two days with repeated artifact reuse across deployments - and reliability: audited handoffs detected and blocked a coordinate transformation error affecting all 2,452 stations before publication. A representative incident (ISS-004) demonstrated boundary-based containment with 10-minute detection latency, zero user exposure, and 80-minute resolution. This paper has been accepted at PEARC 2026.

Figures & tables

Figure 1. The “fail-open” problem under composition. LLM agents can succeed in bounded, reversible tasks (left), but reliability degrades across end-to-end publication workflows with irreversible steps (center). Core LLM limitations compound under composition (right), yielding multiplicative reliability decay pnp^{n} (bottom). At production scale, silent failures create irreversible contamination and trust collapse.
Figure 2. System design overview. Top: Three-Track artifact store externalizing governance (Track 1), semantic context (Track 2), and executable procedures (Track 3) that interlock at execution time . Bottom: multi-agent operating model with role-separated agents (data preparation, publishing, platform operations) connected by audited handoffs (H1/H2) with deterministic validators before state-changing publication.
Figure 3. Three-Track artifact graph and an enforceable “do-not-guess” gate. Left: the full artifact graph with typed nodes for Behaviors (Track 1), Knowledge (Track 2), and Skills (Track 3) used during production. Right: zoomed interlock between a metadata-creation workflow (Track 3) and an authoritative mapping source behavior (Track 1). The behavior enforces that when filenames do not align, the workflow must obtain an authoritative mapping rather than guessing.
Figure 4. Audited handoff protocol. Each agent-to-agent transition follows four phases: prepare (package outputs with provenance), validate (deterministic gates), approve (record and escalate on failure), and commit (apply state changes). Failures block downstream execution. Inset: production incident ISS-004 , from boundary detection to verified resolution.
TablePurposeBacks
messagesInter-agent MCP messagesIncident trail ( ISS-* )
supervisor_actionsSupervisory/commit decisionsCommit receipts
processing_logsPer-dataset lifecycleDataset states
task_phasesPhase-level progressPhased publication
audit_runsAudit execution recordsAuditor independence
llm_tracesModel-call provenanceAction reproducibility
Table 1. Control database: representative audit-trail tables.
Requirement / MetricCase 1 (GIS)Case 2 (SF2Bench)
DG1: Scale without coordination overhead (E1)
Developer workload1 + 2 collab.1 researcher
Primary curation unitDatasetMonitoring station dataset
Datasets processed8492,452
Files published8498,557
Time to completionWeeks2 days
Table 2. Evaluation evidence mapped to operational requirements.
MetricCase 1 (GIS)Case 2 (SF2Bench)
Primary scale849 datasets (ecological archive datasets)2,452 station datasets; 8,557 files
Completion timeWeeks≈\approx 2 days
Supervision placementPer-step reviewBoundary-only (audited handoffs)
Same-model audit differentialN/A0/4 self-detections vs. 4/4 audit detections
Representative prevented incidentManual QA/QCISS-004 blocked before publication (caught at first spatial validation; 0 user exposure)
Table 3. Hero summary: single-agent baseline vs. MAS operating model. Cases reflect sequential deployments under different workloads, not a controlled A/B comparison; differences are attributed to architectural properties (role separation, boundary validation).
IDOutcome / typeBoundary pointKey metric(s)
ISS-004Coordinate field mismatch →\rightarrow spatially implausible GeoJSONPre-publication auditCaught at first spatial validation; 2,452 stations impacted; 0 user exposure; commit blocked
ISS-005Root cause diagnosed; corrected GeoJSON regeneratedRemediation response5 files regenerated with correct coordinate fields ( X COORD / Y COORD )
ISS-007Deliverable placement corrected (shared network drive)Handoff delivery/commitEnsured publishing role consumed the audited artifact version (no local-only paths)
ISS-009Independent verification of coordinate fixPre-republish verificationQuantitative range checks logged for 5 layers; republish approved
ISS-003Completion audit (Phase 5A)Boundary reportVerified 15GB uploaded; 8,557 files processed
Table 4. Representative audited boundary outcomes from the SF2Bench production audit trail.
More in this topic

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,039 papers

Explore similar work

CardsList
Sep 28, 2026

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses

Large language model agents are increasingly deployed for data-intensive work, yet reliable data analysis requires more than general-purpose reasoning and ad hoc tool augmentation. Data Agents, equipped with workflow harnesses, offer a promising paradigm for automating the end-to-end data science lifecycle. This paper examines Data Agents from a harness-centric perspective. First, we introduce a taxonomy of Data Agents and associated data environments, organizing the literature around five functional stages: perception, planning, execution, verification, and repair. Second, we analyze the key technical routes within each stage, identifying 15 distinct approaches ranging from data structure probing to data state reconstruction. Third, we identify four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository. These problems explain why silent failures can persist even when individual components function correctly, highlighting the need for rigorous workflow harnesses and shared reliability resources. Finally, we summarize the horizontal task families of Data Agents, examine their vertical application settings, and benchmarks for evaluation, while maintaining a companion repository at https://github.com/DEEP-PolyU/Awesome-Data-Agents.

Huachi Zhou, Yujing Zhang, Jiahe Du +7

The Hong Kong Polytechnic University, Hong Kong, China · Jilin University, Changchun, China

LLM Agent HarnessesLLM Agent ReliabilityData Analysis Agents
AskPDF
Feb 24, 2026

A Hierarchical Multi-Agent System for Autonomous Discovery in Geoscientific Data Archives

The rapid accumulation of Earth science data has created a significant scalability challenge; while repositories like PANGAEA host vast collections of datasets, citation metrics indicate that a substantial portion remains underutilized, limiting data reusability. Here we present PANGAEA-GPT, a hierarchical multi-agent framework designed for autonomous data discovery and analysis. Unlike standard Large Language Model (LLM) wrappers, our architecture implements a centralized Supervisor-Worker topology with strict data-type-aware routing, sandboxed deterministic code execution, and self-correction via execution feedback, enabling agents to diagnose and resolve runtime errors. Through use-case scenarios spanning physical oceanography and ecology, we demonstrate the system's capacity to execute complex, multi-step workflows with minimal human intervention. This framework provides a methodology for querying and analyzing heterogeneous repository data through coordinated agent workflows.

Dmitrii Pantiukhin, Ivan Kuznetsov, Boris Shapkin +3

Alfred Wegener Institute for Polar and Marine Research, Bremerhaven, Germany

Earth ScienceData Analysis AgentsAutonomous Scientific DiscoveryMulti-Agent SystemsMulti-Agent Orchestration
AskPDF
Jul 8, 2026

Agentic Data Environments

Autonomous agents promise substantial gains in speed, scale, and labor efficiency, but their failures can impose abrupt and often irreversible costs. The central challenge for agentic automation is therefore to increase the benefits of automation while bounding the consequences of failure. While databases remain central to modern computing, agents operate over a broader data environment spanning files, APIs, applications, and system state. In this talk, I will outline early work on Agentic Data Environments -- the execution substrate in which agents operate -- that both amplify agent capabilities and enforce safety guarantees. This perspective reframes data systems from passive stores of state into active substrates for safe, reliable execution.

Elaine Ang, Chenxi Huang, Georgios Liargkovas +13

Columbia University

AI Agent ReliabilityAI Agent Safety
AskPDF