DataJoint 2.0: A Computational Substrate for Agentic Scientific Workflows
Organizations: DataJoint Inc., Houston, USA · DataJoint Inc.
Abstract
Operational rigor determines whether human-agent collaboration succeeds or fails. Scientific data pipelines need the equivalent of DevOps -- SciOps -- yet common approaches fragment provenance across disconnected systems without transactional guarantees. DataJoint 2.0 addresses this gap through the relational workflow model: tables represent workflow steps, rows represent artifacts, foreign keys prescribe execution order. The schema specifies not only what data exists but how it is derived -- a single formal system where data structure, computational dependencies, and integrity constraints are all queryable, enforceable, and machine-readable. Four technical innovations extend this foundation: object-augmented schemas integrating relational metadata with scalable object storage, semantic matching using attribute lineage to prevent erroneous joins, an extensible type system for domain-specific formats, and distributed job coordination designed for composability with external orchestration. By unifying data structure, data, and computational transformations, DataJoint creates a substrate for SciOps where agents can participate in scientific workflows without risking data corruption.
Figures & tables
| Aspect | Mathematical (Codd) | ER (Chen) | Relational Workflow |
|---|---|---|---|
| Core Question | Functional dependencies | Entity types | When/how created |
| Workflow Dimension | Not addressed | Not central | Fundamental |
| Implementation Gap | High | High | None |
| Workflow Support | None | None | Native |
Scientific Workflow Automation
Scientific workflow automation uses software, AI agents, and laboratory systems to plan, execute, validate, and reproduce research tasks.
See all 161 papers