Talk Freely, Execute Strictly: Schema-Gated Agentic AI for Flexible and Reproducible Scientific Workflows
Authors: Joel Strickland, Arjun Vijeta, Chris Moores, Oliwia Bodek, Bogdan Nenchev, Thomas Whitehead, Charles Phillips, Karl Tassenberg, Gareth Conduit, Ben Pellegrini
Organizations: Intellegens, The Studio, Chesterton Mill, Cambridge, CB4 3NP, UK · Independent Researcher · Cavendish Laboratory, J.J. Thomson Avenue, Cambridge, UK
Large language models (LLMs) can now translate a researcher's plain-language goal into executable computation, yet scientific workflows demand determinism, provenance, and governance that are difficult to guarantee when an LLM decides what runs. Semi-structured interviews with 18 experts across 10 industrial R&D stakeholders surface 2 competing requirements--deterministic, constrained execution and conversational flexibility without workflow rigidity--together with boundary properties (human-in-the-loop control and transparency) that any resolution must satisfy. We propose schema-gated orchestration as the resolving principle: the schema becomes a mandatory execution boundary at the composed-workflow level, so that nothing runs unless the complete action--including cross-step dependencies--validates against a machine-checkable specification. We operationalize the 2 requirements as execution determinism (ED) and conversational flexibility (CF), and use these axes to review 20 systems spanning 5 architectural groups along a validation-scope spectrum. Scores are assigned via a multi-model protocol--15 independent sessions across 3 LLM families--yielding substantial-to-near-perfect inter-model agreement (Krippendorff a=0.80 for ED and a=0.98 for CF), demonstrating that multi-model LLM scoring can serve as a reusable alternative to human expert panels for architectural assessment. The resulting landscape reveals an empirical Pareto front--no reviewed system achieves both high flexibility and high determinism--but a convergence zone emerges between the generative and workflow-centric extremes. We argue that a schema-gated architecture, separating conversational from execution authority, is positioned to decouple this trade-off, and distill 3 operational principles--clarification-before-execution, constrained plan-act orchestration, and tool-to-workflow-level gating--to guide adoption.
Figures & tables
Group
Theme
Mentions
Breadth
System Integration
191
10/10
Workflow & Task Automation
112
10/10
Data Cleaning & Preparation
70
10/10
Security & IP Protection
67
9/10
Domain / Scientific Expertise
62
9/10
Determinism pole (527 total; xˉ=88 )
Safety & Ethical Boundaries
25
9/10
Table 1: Themes from practitioner interviews grouped by architectural implication. Mentions is the total count across all experts; Breadth is the number of stakeholders (out of 10) in which the theme appeared. Group headers show total mentions and per-theme average ( xˉ ) to allow comparison across groups of different size. Row colour encodes thematic group: blue = determinism pole, orange = flexibility pole, green = cross-cutting, grey = out of scope (matching Figure 1 ).
Figure 1: Thematic landscape from practitioner interviews. Each numbered circle represents one of the 17 themes; the key maps numbers to theme names. Position shows stakeholder breadth ( x -axis: number of stakeholders mentioning the theme, out of 10) and total mention count ( y -axis). Colour encodes theme group: blue = determinism pole, orange = flexibility pole, green = cross-cutting / boundary properties, grey = out of architectural scope. Dashed lines mark the median stakeholder breadth and median total mentions; the shaded quadrant highlights themes above both medians. Dotted ellipses enclose themes that share approximately the same grid position.
Score
ED (Execution Determinism)
CF (Conversational Flexibility)
1
Unconstrained generated code; no formal pre-execution gate
Limited NL assistance (form help, search, hints); NL does not control execution
3
Developer-defined tool interfaces; partial validation but not uniformly schema-governed
NL assists authoring/modifying workflows; execution requires an externalized artifact
4
Schema/type-validated tool calls with strong pre-execution validation
NL directly selects/parameterizes tool/workflow invocations via structured calls
5
Explicit workflow spec/registry with static validation; reproducibility by construction
Free-form NL or agentic loops directly driving actions with minimal structural constraint
Table 2: Ordinal rubric for execution determinism (ED) and conversational flexibility (CF).
Figure 2: Architectural design space for conversational scientific workflows. Systems are positioned by mean ED and CF scores across 15 independent scoring runs (Appendix C reports consensus medians; means are used here for visual separation of co-located systems); point labels correspond to system IDs. Colours encode five groups: generative (red), tool-augmented (orange), schema-gated (green), workflow + NL (purple), and workflow-centric (blue). The dashed line is the empirical Pareto front connecting non-dominated positions with piecewise-linear segments (i.e. straight lines between observed Pareto-optimal points); the shaded region below the front contains dominated positions. The gold star marks the ideal region of high flexibility and high determinism. Caveat: scores are ordinal (1–5) and non-quantitative; the piecewise-linear front is displayed for visual communication of the trade-off shape and should not be interpreted as implying interval-scale distances between positions or a continuous functional relationship.
Table 4: Architectural paradigms for conversational scientific workflows (summary). Column colours correspond to the paradigm groups defined in Table 3 .
Figure 3: Reference architecture separating conversational authority from execution authority. Context and state (user query, chat history, system information, internal data, and prior tool/workflow outputs) are assembled for an LLM orchestrator (planner/reasoner), which produces an assistant message and, optionally, an action proposal. Proposals enter the execution-authority gate (red dashed region): an output parser validates them against a JSON schema or workflow specification. Invalid proposals trigger a clarification loop; valid proposals are forwarded to the workflow executor. Resulting data, logs, and artifacts update the shared context for subsequent turns. The LLM may converse freely, but cannot execute—execution authority resides entirely in schema validation.
Figure 4: Domain tool schema validation and inclusion in the registry. New domain tools undergo schema and compliance checks—including type validation, documentation completeness, and service availability—before being added to the validated tool registry. Only schema-verified domain tools become available for workflow execution.
Figure 5: Workflow schema composition in the validated registry. Each workflow combines schema-validated domain tools into a DAG, defining explicit data and parameter flow between steps. Only workflows that pass acyclicity, type-compatibility, and parameter-resolution checks are included in the validated workflow registry.
Figure 6: Schema-gated workflow execution process. The system identifies candidate workflows, loads the corresponding schema, collects and validates parameters through dialogue, and executes only after all constraints are satisfied. Validation failures trigger clarification loops; conversational context is updated throughout.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Industry sector
Revenue (USD)
Employees
Experts
Org-1
Food & beverage
> $50B
> 100,000
1
Org-2
Professional tools
$5–10B
10,000+
2
Org-3
Packaging & materials
$5–10B
10,000–50,000
2
Org-4
Food ingredients
$5–10B
10,000–15,000
6
Org-5
Consumer products
$5–10B
5,000–10,000
1
Org-6
Specialty chemicals
$1–5B
5,000–10,000
2
Appendix
Table A.1: Anonymised participant demographics. Revenue and employee figures are approximate bands based on publicly available data (fiscal year 2025 or latest available). The independent consultant’s career spans multiple specialty-chemicals firms. Pseudonym assignment is randomised.
Theme
Description / Coding Scope
System Integration
Integration with specific named systems: LIMS, ELN, SQL, APIs, etc.
Workflow & Task Automation
Automating repetitive R&D tasks: design of experiments (DOE), literature review, report generation, experiment design
Data Cleaning & Preparation
Automated data cleaning, formatting, standardisation, quality checks, outlier removal
Security & IP Protection
Data security, IP protection, confidentiality, compliance, access controls
Domain / Scientific Expertise
AI understanding domain-specific context, scientific knowledge, industry standards
Safety & Ethical Boundaries
Tasks that should never be automated: safety, ethics, strategic decisions, compliance
Appendix
Table A.2: Codebook: theme definitions used for coding practitioner interview responses. Row colour encodes thematic group: determinism, flexibility, cross-cutting, or out of scope (see Table 1 ).
Theme
Count
Max drop
Mean drop
Most impactful pattern
System Integration
191
23.0%
3.9%
API[s]?
Workflow Automation
112
29.5%
4.1%
DOE
Data Search & Retrieval
108
17.6%
4.2%
quer(y|ying|ies)
Natural Language Interface
86
23.3%
4.7%
prompt(s|ing)?
Explainability
83
30.1%
4.5%
explain(ability|able)?
Data Cleaning
70
32.9%
4.4%
data\s+prep
Appendix
Table B.1: Leave-one-out pattern stability. Max drop is the largest percentage decrease in coded paragraphs caused by removing any single pattern from the theme’s 20-pattern set.
Scientific workflow management (WMSs) systems automate execution, yet orchestrate using fixed, hand-tuned rules. LLM agents promise more autonomous orchestration, but it remains unclear where to introduce agentic reasoning, how to bound its risk, and when it actually helps. We present Avatar, an actor-based architecture comprising an orchestrator, an executor, and a provenance monitor. Each actor's decision policy is pluggable (rule-based or LLM-backed) via a single adapter-validated action catalog, so conventional and agentic control run on the same core across different WMSs. We present an implementation using the Academy framework and evaluate Avatar across three workloads. We observe that Avatar's rule mode reproduces native execution, with a single unchanged core running all three. Moreover, LLM-backed Avatar reports a reduction of compute wastage by 55% and cuts GPU-busy time by 40%. Overall, we envision Avatar as a step toward workflow systems that reason about their own orchestration rather than follow pre-fixed rules.
Suman Raj, Hai Duc Nguyen, Haochen Pan +3
Department of Computer Science, University of Chicago, IL, USA · Data Science and Learning Division, Argonne National Laboratory, Lemont, IL, USA
Scientific workflow systems automate execution -- scheduling, fault tolerance, resource management -- but not the semantic translation that precedes it. Scientists still manually convert research questions into workflow specifications, a task requiring both domain knowledge and infrastructure expertise. We propose an agentic architecture that closes this gap through three layers: an LLM interprets natural language into structured intents (semantic layer); validated generators produce reproducible workflow DAGs (deterministic layer); and domain experts author ``Skills'': markdown documents encoding vocabulary mappings, parameter constraints, and optimization strategies (knowledge layer). This decomposition confines LLM non-determinism to intent extraction: identical intents always yield identical workflows. We implement and evaluate the architecture on the 1000 Genomes population genetics workflow and Hyperflow WMS running on Kubernetes. In an ablation study on 150 queries, Skills raise full-match intent accuracy from 44% to 83%; skill-driven deferred workflow generation reduces data transfer by 92%; and the end-to-end pipeline completes queries on Kubernetes with LLM overhead below 15 seconds and cost under $0.001 per query.
Bartosz Balis, Michal Orzechowski, Piotr Kica +2
AGH University of Krakow, Krakow, Poland · Sano Centre for Computational Medicine, Krakow, Poland
In schema-guided reasoning (SGR) pipelines, LLMs produce explicit intermediate structures -- rubrics, checklists, or verification queries -- before committing to a final decision. SGR is increasingly adopted because it promises controllability: practitioners expect to inspect, edit, and override these structures to steer the outcome. But does the promise hold? We introduce a causal evaluation protocol to measure it: by selecting tasks where a deterministic function maps intermediate structures to decisions, every controlled edit implies a unique correct output. Across 12 models and 4 benchmarks, models appear self-consistent with their own intermediate structures but fail to update predictions after intervention -- revealing that apparent faithfulness is fragile once the intermediate structure changes. When derivation of the final decision from the structure is delegated to an external tool, this fragility largely disappears; stronger prompting yields only limited improvements, while preference optimization substantially improves intervention faithfulness. Overall, intermediate structures in schema-guided pipelines function as influential context rather than stable causal mediators.