Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps

Sagar Srinivas Sakhinana; Venkataramana Runkana

cs.MAAug 30, 2026

Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps

Authors: Sagar Srinivas Sakhinana, Venkataramana Runkana

Organizations: Tata Research Development and Design Centre

Abstract

Across industries, machine-learning systems support applications ranging from prediction and anomaly detection to forecasting, optimization, and scheduling, yet operationalizing these systems requires coordinating application development, model pipelines, cloud infrastructure, security, deployment, monitoring, retraining, recovery, and rollback. We present an evidence-gated multi-agent framework for transforming a natural-language MLOps cloud engineering task into a verified repository and operational cloud deployment. The framework combines graph engineering, loop engineering, and agent harness engineering. A stateful Graph Orchestrator coordinates specialized agents for repository generation, review, execution, verification, release, and monitoring while governing workflow dependencies, evidence gates, retry bounds, recovery paths, and termination. Consequential lifecycle transitions proceed only when their required predicates are supported by verifiable execution or runtime evidence. Verification failures activate bounded reflection, repair, and re-verification, while runtime evidence of failure, drift, degradation, or policy violation can trigger bounded adaptation, recovery, or rollback. Agent harness engineering constrains repository generation, review, and repair, artifact execution, and cloud operations through controlled capabilities and isolated execution environments. We realize the framework on Google Cloud Platform and evaluate repository completeness, controlled execution, evidence-gated transitions, cloud promotion, and bounded recovery. Our experimental results show that the framework prevents unsupported lifecycle transitions and drives each run toward either a verified operational deployment or an auditable terminal failure.

Figures & tables

Panel of Figure 1 — Figure 1: The figure shows how the evidence-gated autonomous cloud MLOps transforms a natural-language engineering task into a verified operational cloud deployment. Graph engineering governs long-horizon lifecycle, loop engineering provides bounded artifact correction and runtime adaptation, agent harness engineering constrains execution, and verified evidence gates consequential lifecycle transitions.

Panel of Figure 2 — Figure 2: The figure shows the multi-agent architecture for evidence-gated autonomous cloud MLOps. The Graph Orchestrator manages workflow state, dependencies, evidence gates, retry bounds, recovery paths, and termination across specialized agents. Verification failures and adverse runtime evidence activate bounded reflection and repair. Agent Harness Engineering constrains agent access through controlled sandboxes and cloud infrastructure.

Panel of Figure 3 — Figure 3: Google Cloud realization of the evidence-gated autonomous MLOps framework. The Graph Orchestrator, implemented with Google ADK on GKE, coordinates specialized agents through Agent2Agent (A2A), while the Model Context Protocol (MCP) provides policy-scoped access to repository and browser tools, isolated execution, and cloud deployment, release, recovery, and rollback operations.

Overall / RQ	Metric	Interpretation
Overall	Verified Operational Deployment Rate (VODR)	Did the entire execution end in a verified operational cloud deployment, with $\Phi_{\mathrm{repo}}=1$ , $\Phi_{\mathrm{release}}=1$ , and $\Phi_{\mathrm{runtime}}=1$ ?
RQ1	Repository Completeness Score (RCS)	How complete is the generated repository in terms of the required artifact categories?
RQ1	Repository Acceptance Rate (RAR)	Did the repository ultimately satisfy $\Phi_{\mathrm{repo}}$ after bounded correction and re-verification?
RQ2	Controlled Execution Rate (CER)	Were executable repository artifacts processed through the GKE Sandbox with gVisor and accompanied by retained machine-checkable execution evidence?
RQ3	Verified Progression Rate (VPR)	When a forward transition’s verification predicate was satisfied, did the Graph Orchestrator permit progression?
RQ3	Blocked Forward Progression Rate (BFR)	When a forward transition’s verification predicate was unsatisfied, did the Graph Orchestrator block progression?

Table 1: Summary of evaluation metrics.

Model	Nominal	Repository Verification	Cloud Verification	Runtime Verification
GPT-5.6 Sol	0.99	0.95	0.96	0.94
Gemini 2.5 Pro	0.78	0.62	0.65	0.59
Gemini 2.5 Flash	0.52	0.38	0.41	0.35
Gemini 2.5 Flash-Lite	0.38	0.24	0.27	0.21

Table 2: Verified Operational Deployment Rate (VODR) across experimental scenarios.

Model	Metric	Nominal	Repository Verification	Cloud Verification	Runtime Verification	Retry-Budget Exhaustion
GPT-5.6 Sol	RCS	0.99	0.99	0.99	0.99	0.98
	RAR	0.99	0.97	0.99	0.99	0.56
Gemini 2.5 Pro	RCS	0.97	0.95	0.96	0.96	0.94
	RAR	0.82	0.67	0.79	0.77	0.42
Gemini 2.5 Flash	RCS	0.93	0.91	0.92	0.92	0.90
	RAR	0.62	0.47	0.60	0.57	0.36

Table 3: Repository completeness and acceptance across experimental scenarios.

Model	Nominal	Repository Verification	Cloud Verification	Runtime Verification	Retry-Budget Exhaustion
GPT-5.6 Sol	1.00	1.00	1.00	1.00	1.00
Gemini 2.5 Pro	1.00	1.00	1.00	1.00	1.00
Gemini 2.5 Flash	1.00	1.00	1.00	1.00	1.00
Gemini 2.5 Flash-Lite	1.00	1.00	1.00	1.00	1.00

Table 4: Controlled Execution Rate (CER) across experimental scenarios.

Model	Metric	Nominal	Repository Verification	Cloud Verification	Runtime Verification	Retry-Budget Exhaustion
GPT-5.6 Sol	VPR	1.00	1.00	1.00	1.00	1.00
	BFR	1.00	1.00	1.00	1.00	1.00
Gemini 2.5 Pro	VPR	1.00	1.00	1.00	1.00	1.00
	BFR	1.00	1.00	1.00	1.00	1.00
Gemini 2.5 Flash	VPR	1.00	1.00	1.00	1.00	1.00
	BFR	1.00	1.00	1.00	1.00	1.00

Table 5: Verified Progression Rate (VPR) and Blocked Forward Progression Rate (BFR) across experimental scenarios.

Model	Nominal	Repository Verification	Cloud Verification	Runtime Verification
GPT-5.6 Sol	1.00	0.99	0.98	0.98
Gemini 2.5 Pro	0.98	0.96	0.84	0.91
Gemini 2.5 Flash	0.90	0.87	0.70	0.79
Gemini 2.5 Flash-Lite	0.89	0.87	0.65	0.77

Table 6: Release and Promotion Success Rate (RPSR) across experimental scenarios.

Verification Target	Measure	GPT-5.6 Sol	Gemini 2.5 Pro	Gemini 2.5 Flash	Gemini 2.5 Flash-Lite
Repository Verification	Reached	99	96	94	88
	RSR	0.98	0.70	0.50	0.34
Cloud Verification	Reached	99	77	57	40
	RSR	0.98	0.86	0.74	0.70
Runtime Verification	Reached	97	70	45	30
	RSR	0.97	0.84	0.78	0.70

Table 7: Recovery and retry-budget-exhaustion outcomes by verification target.

Level	Dataset	Application	Two-Tier Application	ML Workflow
Easy	MNIST	Digit classification	Image-upload frontend with predicted-digit visualization; backend for image preprocessing and authenticated inference.	Image classification, reproducible training, evaluation, monitoring, and retraining.
Easy	California Housing	House-value regression	Feature-input frontend with predicted-value visualization; backend for feature preprocessing and authenticated inference.	Tabular preprocessing, regression training, evaluation, monitoring, and retraining.
Easy	IMDB Large Movie Review	Sentiment classification	Text-input frontend with sentiment visualization; backend for text preprocessing and authenticated inference.	Text preprocessing, sentiment classification, evaluation, monitoring, and retraining.
Medium	CIFAR-10	Image classification	Image-upload frontend with class visualization; backend for image preprocessing and authenticated inference.	Vision-model training, augmentation, evaluation, serving, monitoring, and retraining.
Medium	UCI Bike Sharing	Demand forecasting	Temporal-input frontend with demand-forecast visualization; backend for temporal feature processing and authenticated forecasting.	Temporal feature construction, forecasting, evaluation, monitoring, and scheduled retraining.
Medium	UCI Human Activity Recognition	Activity recognition	Sensor-input frontend with activity visualization; backend for accelerometer and gyroscope preprocessing and authenticated inference.	Multivariate sensor preprocessing, activity classification, evaluation, monitoring, and retraining.

Table 8: Representative tasks from the 100-task cloud MLOps benchmark.

Variant	Modification
Full Framework	Evidence-gated transitions, graph-controlled correction and recovery, re-verification, and retry-budget semantics remain unchanged.
Evidence-Gate Bypass	Verification predicates are evaluated, but their outcomes do not gate enabled forward transitions. An unsatisfied predicate therefore does not block progression or enter the associated corrective or terminal-failure path.
Zero Retry Budget	All transitions that are retryable in the Full Framework are assigned zero retry budget. Evidence gating remains active, but an unsatisfied predicate causes terminal failure without entering the graph-controlled correction or recovery path.
Retry-Budget Sensitivity	The retry budget for each retryable transition is varied over $\{1,3,5,10,20\}$ while evidence gating and all other framework mechanisms remain unchanged.

Table 9: Ablation and retry-budget configurations.

Variant	VODR Repository	VODR Cloud	VODR Runtime	BFR Repository	BFR Cloud	BFR Runtime	RSR Repository	RSR Cloud	RSR Runtime
Full Framework	0.95	0.96	0.94	1.00	1.00	1.00	0.98	0.98	0.97
Evidence-Gate Bypass	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Zero Retry Budget	0.00	0.00	0.00	1.00	1.00	1.00	0.00	0.00	0.00

Table 10: Primary ablation results for GPT-5.6 Sol.

Retry Budget	VODR Repository	VODR Cloud	VODR Runtime	RSR Repository	RSR Cloud	RSR Runtime
1	0.74	0.77	0.71	0.75	0.78	0.74
3	0.87	0.89	0.83	0.88	0.90	0.86
5	0.93	0.94	0.89	0.94	0.95	0.92
10	0.95	0.96	0.93	0.96	0.97	0.96
20	0.95	0.96	0.94	0.98	0.98	0.97

Table 11: Retry-budget sensitivity for GPT-5.6 Sol.

Panel of Figure 4 — Figure 4: High-level lifecycle of the evidence-gated autonomous cloud MLOps framework. A natural-language MLOps cloud engineering task progresses through multi-agent engineering, artifact generation and controlled execution, artifact and release verification, cloud deployment, and runtime verification. Failed artifact or release verification returns to multi-agent engineering through the correction path, while runtime verification can return through the adaptation path. The lifecycle is therefore cyclic rather than a one-shot CI/CD pipeline.

Panel of Figure 5 — Figure 5: Persistent workflow and audit state using AlloyDB for PostgreSQL with row-level security for tenant isolation.

Panel of Figure 6 — Figure 6: Controlled software-supply-chain and release path through build, artifact publication, GitOps reconciliation, policy enforcement, and deployment to GKE.

Panel of Figure 7 — Figure 7: Complementary security controls for workload identity, authenticated communication, network isolation, and managed secret access.

Panel of Figure 8 — Figure 8: Runtime evidence and bounded adaptation. Metrics, logs, traces, and Falco alerts support runtime verification, while adverse evidence invokes reflection and repair before controlled lifecycle re-entry.

Panel of Figure 9 — Figure 9: The figure shows the execution-agent and sandbox separation on GKE Autopilot. The Graph Orchestrator and specialized agents run as regular GKE workloads. The Execution Agent requests controlled execution through the Execution MCP, whereas the GKE Sandbox with gVisor provides the isolated environment for generated-artifact execution. Execution evidence is evaluated by the Verification Agent; failed verification activates bounded reflection and repair, while verified artifacts proceed to release and monitoring.

Panel of Figure 10 — Figure 10: The figure depicts build and Artifact Pipeline.

Panel of Figure 11 — Figure 11: GitOps reconciliation, policy enforcement, and evidence-gated promotion from staging to production.

Panel of Figure 12 — Figure 12: Multi-layer zero-trust security controls spanning cloud identity, workload identity, authorization, network isolation, and secret management.

Panel of Figure 13 — Figure 13: Runtime observability, evidence derivation, and evidence-gated verification for continued operation or bounded recovery.

Explore similar work

Apr 29, 2026cs.AI

Think it, Run it: Autonomous ML pipeline generation via self-healing multi-agent AI

The purpose of our paper is to develop a unified multi-agent architecture that automates end-to-end machine learning (ML) pipeline generation from datasets and natural-language (NL) goals, improving efficiency, robustness and explainability. A five-agent system is proposed to handle profiling, intent parsing, microservice recommendation, Directed Acyclic Graph (DAG) construction and execution. It integrates code-grounded Retrieval-Augmented Generation (RAG) for microservice understanding, an explainable hybrid recommender combining multiple criteria, a self-healing mechanism using Large Language Model (LLM)-based error interpretation and adaptive learning from execution history. The approach is evaluated on 150 ML tasks across diverse scenarios. The system achieves an 84.7% end-to-end pipeline success rate, outperforming baseline methods. It demonstrates improved robustness through self-healing and reduces workflow development time compared to manual construction. The study introduces a novel integration of code-grounded RAG, explainable recommendation, self-healing execution and adaptive learning within a single architecture, showing that tightly coupled intelligent components can outperform isolated solutions.

Adela Bara, Gabriela Dobrita, Simona-Vasilica Oprea

Bucharest University of Economic Studies, Department of Economic Informatics and Cybernetics, no.6 Piața Romană, Bucharest, 010374, Romania

Data Generation Pipelines Microservices Self-Repair Real-World Content Recommendation Problem

Ask PDF

May 11, 2026cs.AI

Beyond Autonomy: A Dynamic Tiered AgentRunner Framework for Governable and Resilient Enterprise AI Execution

Current large language model agent frameworks prioritize autonomy but lack the governability mechanisms required for enterprise deployment. High-risk write operations proceed without independent review, complex tasks lack acceptance verification, and computational resources are allocated uniformly regardless of risk level. We propose the Dynamic Tiered AgentRunner, a controlled execution protocol distilled from a production-grade multi-tenant SaaS platform. The framework introduces three core mechanisms: (1) Risk-Adaptive Tiering that dynamically allocates computational resources and review intensity based on task risk profiles, achieving Pareto-optimal trade-offs between safety and efficiency; (2) Separation of Powers architecture where proposal, review, execution, and verification are performed by independent agents with physically isolated boundaries; and (3) Resilience-by-Design through a Verifier-Recovery closed loop that treats failure as a first-class system state. We formalize the tier selectio

Kai Pan, Rong Hou

A2A Lab

Artificial Intelligence Deployment Agentic Deployments Governance Governed

Ask PDF

Jun 16, 2026cs.MA

Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization

Big-Data-as-a-Service (BDaaS) platforms require re liable automation across data ingestion, cleaning, feature engi neering, model development, deployment, and post-deployment monitoring. However, existing LLM-based data science agents and AutoML systems mainly focus on isolated workflow stages, leaving limited support for lifecycle-level orchestration, artifact governance, human oversight, and drift-aware adaptation. This paper proposes a trustworthy self-composable BDaaS frame work based on LLM-orchestrated multi-agent collaboration. The proposed architecture decomposes the BDaaS lifecycle into specialized agents for data ingestion, data cleaning, feature engineering, AutoML training, model evaluation, MLOps de ployment, monitoring, and drift detection. A central LLM or chestration layer coordinates agent execution, validates interme diate outputs, manages workflow context, and enables dynamic workflow composition. The framework also incorporates shared artifact governance, reproducibility support, human-in-the-loop checkpoints, and drift-aware feedback loops. A prototype-based evaluation is conducted using controlled tabular benchmark datasets with missing values, categorical variables, outliers, class imbalance, and simulated covariate drift. Compared with manual ML, AutoML-only, and single-agent LLM baselines, the pro posed multi-agent BDaaS pipeline achieves competitive predictive performance while improving lifecycle-level reliability, including workflow completion, artifact traceability, deployment readiness, reproducibility, and drift recovery. The results suggest that LLM-orchestrated multi-agent systems can extend conventional AutoML toward trustworthy, adaptive, and production-oriented BDaaS lifecycle automation.

Aueaphum Aueawatthanaphisut, Badri Raj Lamichhane

School of Information, Computer, and Communication Technology Sirindhorn International Institute of Technology, Thammasat University Pathum Thani, Thailand

Data Science Agents Large Language Model Workflows Multi-Agent Orchestration Large Language Model Baselines

Ask PDF