AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis

Pei Yang; Wanyi Chen; Asuka Yuxi Zheng; Xueqian Li; Xiang Li; Haoqin Tu; Jie Xiao; Yifan Pang

cs.LGMar 3, 2026

AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis

Authors: Pei Yang, Wanyi Chen, Asuka Yuxi Zheng, Xueqian Li, Xiang Li, Haoqin Tu, Jie Xiao, Yifan Pang,

Organizations: Gradient · Soochow University · UC Santa Cruz · Georgia Institute of Technology ·

Abstract

Large language model (LLM) agents offer a promising data-driven approach to automating Site Reliability Engineering (SRE), yet their enterprise deployment is constrained by three challenges: restricted access to proprietary data, unsafe action execution under permission-governed environments, and the inability of closed systems to improve from failures. We present AOI (Autonomous Operations Intelligence), a trainable multi-agent framework formulating automated operations as a structured trajectory learning problem under security constraints. Our approach integrates three key components. First, a trainable diagnostic system applies Group Relative Policy Optimization (GRPO) to distill expert-level knowledge into locally deployed open-source models, enabling preference-based learning without exposing sensitive data. Second, a read-write separated execution architecture decomposes operational trajectories into observation, reasoning, and action phases, allowing safe learning while preventing unauthorized state mutation. Third, a Failure Trajectory Closed-Loop Evolver mines unsuccessful trajectories and converts them into corrective supervision signals, enabling continual data augmentation. Evaluated on the AIOpsLab benchmark, our contributions yield cumulative gains. (1) The AOI runtime alone achieves 66.3% best@5 success on all 86 tasks, outperforming the prior state-of-the-art (41.9%) by 24.4 points. (2) Adding Observer GRPO training, a locally deployed 14B model reaches 42.9% avg@1 on 63 held-out tasks with unseen fault types, surpassing Claude Sonnet 4.5. (3) The Evolver converts 37 failed trajectories into diagnostic guidance, improving end-to-end avg@5 by 4.8 points while reducing variance by 35%.

Figures & tables

Figure 1: AOI System Overview. Left: Closed-Loop Evolution Pipeline —a Judge classifies SRE troubleshooting workflows by outcome. Failed workflows are repaired by the Evolver into corrected command sequences that serve as diagnostic guidance at inference time. Successful workflows are distilled by the Purifier into optimal diagnostic paths that serve as training data. Right: Multi-Agent Runtime —the Observer coordinates read-only diagnosis (Probe) and write-gated remediation (Executor). The Observer is trained via GRPO, and at inference time receives the Evolver’s corrected plans as structured prompts.

Figure 2: AOI Runtime Agent Architecture. The Observer coordinates diagnosis through the Probe (read-only) and Executor (write-gated) agents. The Compressor maintains context efficiency via dual-timescale memory.

Agent	Responsibility	Permission	$\mathcal{M}_{raw}$	$\mathcal{M}_{task}$	$\mathcal{M}_{comp}$
Observer ( $\mathcal{O}$ )	Planning, hypothesis tracking	Read compressed	-	R/W	R
Probe ( $\mathcal{P}$ )	Diagnostic exploration	Read-only exec	W	R	-
Executor ( $\mathcal{X}$ )	Remediation actions	Write (gated)	W	R	-
Compressor ( $\mathcal{C}$ )	Context distillation	Read raw, write comp.	R	-	W

Table 1: Agent roles and permissions in AOI. Right columns show the access control matrix over three memory stores (R = read, W = write, - = no access):

\mathcal{M}_{raw}

stores raw environment outputs (e.g., full kubectl responses);

\mathcal{M}_{task}

maintains the diagnostic task queue and hypothesis state;

\mathcal{M}_{comp}

holds compressed context produced by the Compressor, serving as the Observer’s sole information source.

Model	Method	Detection	Localization	RCA	Mitigation	Overall
GPT-4o-mini	AOL-agent	25.0	9.5	7.7	7.7	14.7
GPT-4o-mini	STRATUS	78.1	25.0	15.4	23.1	43.0
GPT-4o-mini	AOI	90.6	32.1	38.5	53.8	58.1
Claude Sonnet 4.5	AOL-agent	68.8	53.6	15.4	76.9	57.0
Qwen3-14B (best@5/avg@5)	STRATUS	75.0/41.3	32.1/11.4	7.7/4.6	15.4/15.4	41.9/22.1
Qwen3-14B (best@5/avg@5)	AOI	100/66.9	53.6/27.9	30.8/7.7	46.2/23.1	66.3/38.6

Table 2: AOI runtime comparison on full benchmark (86 tasks, %). GPT-4o-mini OpenAI (2023) , Claude Sonnet 4.5 uses single-run; Qwen3-14B uses 5-round sampling (best@5/avg@5).

Method	Det. $\uparrow$	Loc. $\uparrow$	RCA $\uparrow$	Mit. $\uparrow$	Overall $\uparrow$
	(22)	(22)	(12)	(7)	(63)
Sonnet 4.5 (AOL-agent)	54.5	40.9	8.3	57.1	41.3
AOI (Untrained)	65.5	22.7	6.7	14.3	33.7
AOI (Observer-GRPO)	90.9	18.2	16.7	14.3	42.9

Table 3: Observer GRPO on held-out fault types (63 tasks, %).

Method	Det.	Loc.	RCA	Mit.	Overall
	(10)	(13)	(11)	(3)	(37)
Base	100/50.0	54 /26.2	27/7.3	0/0	54 /24.9
Evolver-prompts	90/64.0	54 /24.6	18/12.7	0/0	49/29.7
Observer-GRPO	90/64.0	38/21.5	36 / 29.1	0/0	49/33.5
Observer-GRPO [-3pt]+ Evolver-prompts	100 /72.5	31/19.2	36 /25.0	0/0	49/ 33.8

Table 4: Component ablation on

\mathcal{D}^{test}_{evolver}

(37 tasks, best / avg %). First three rows use 5 runs; last row uses 4 runs.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Trajectory-Corrective Evolver Architecture. The Evolver observes failed command sequences from the Observer’s execution history, generates

G

candidate corrections via GRPO sampling (sampling from a policy trained on successful trajectories), and provides the highest-scoring correction as a structured prompt to guide the Observer’s next attempt. Key insight: This closed-loop mechanism converts failed diagnostic attempts into learning opportunities without requiring manual expert intervention.

AOI Runtime
Parameter	Value
Max iterations	15
Max Probe rounds/iteration	5
Context budget/iteration	4096 tokens
Long-term memory capacity	10 summaries
Executor whitelist	47 command patterns

Appendix

Table 5: Complete hyperparameter settings.

Task Type	Count
Detection	10
Localization	13
Mitigation	3
RCA	11
Total	37

Appendix

Table 10

Figure 6: Nested data partition structure.

\mathcal{D}^{all}

(86 tasks) contains

\mathcal{D}^{train}_{evolver}

(49 successful trajectories from Claude Sonnet 4.5), which in turn contains

\mathcal{D}^{train}_{obs}

(23 tasks covering 11 fault types). The strict subset relationship ensures that no test data appears in any training set. Visual interpretation: The outer circle represents all benchmark tasks; the middle ring shows Evolver training data (all successful Sonnet trajectories); the innermost core shows Observer training data (carefully selected to cover diverse fault types while maintaining strict hold-out). This design enables fair evaluation of the combined Observer+Evolver system.

Fault Type	GT Tasks	Total Tasks
k8s_target_port-misconfig	9	12
scale_pod_zero_social_net	3	4
misconfig_app_hotel_res	3	4
astronomy_shop_cart_service_failure	1	2
astronomy_shop_payment_service_failure	1	2
astronomy_shop_product_catalog_failure	1	2

Appendix

Table 6: Training fault types (11 types

\to

23 successful trajectories). These fault types were selected to provide diverse coverage of common failure modes while ensuring sufficient test data remains for evaluation.

Fault Type	Tasks
revoke_auth_mongodb	8
user_unregistered_mongodb	8
assign_to_non_existent_node_social_net	4
wrong_bin_usage	4
astronomy_shop_ad_service_high_cpu	2
astronomy_shop_ad_service_manual_gc	2

Appendix

Table 7: Test fault types (15 types

\to

48 tasks) plus 15 tasks from training fault types where Sonnet failed, totaling 63 held-out tasks. Critical: All test fault types are completely unseen during Observer training, ensuring we evaluate true generalization capability.

Figure 7: best@

k

success rate vs number of runs. The curve shows clear diminishing returns: the first retry (

k

\to

2) provides +19.8pp improvement, while subsequent retries (

k

\to

5) each provide only +2-3pp. Interpretation: The rapid initial improvement suggests many tasks have “near-miss” failure modes that succeed on retry, while the flattening curve indicates some tasks are fundamentally beyond current capability. The inflection point at

k

=2-3 provides clear guidance for resource-constrained deployments.

Round	Det.	Loc.	RCA	Mit.	Overall
R1	53.1	21.4	7.7	23.1	31.4
R2	75.0	32.1	0.0	23.1	41.9
R3	59.4	32.1	15.4	23.1	38.4
R4	75.0	21.4	15.4	23.1	40.7
R5	71.9	32.1	0.0	23.1	40.7
Avg	66.9	27.9	7.7	23.1	38.6

Appendix

Table 8: Per-round success rates (%) by task type.

Figure 8: Per-round success rate variance by task type. Detection exhibits the widest range (22pp), indicating high stochastic sensitivity—tasks that fail in one run often succeed in another due to different exploration paths. Mitigation’s flat line confirms that current failures are deterministic capability gaps rather than sampling artifacts. Actionable insight: Detection tasks benefit most from multi-round sampling, while Mitigation requires architectural improvements rather than more attempts.

Success	Category	Count	%
5/5	Consistently solved	14	16.3
3–4/5	Mostly solved	16	18.6
1–2/5	Occasionally solved	27	31.4
0/5	Never solved	29	33.7

Appendix

Table 9: Task stability distribution (86 tasks total).

Figure 9: Task stability distribution across 86 benchmark tasks. The large middle segment (50% of tasks with 1-4/5 success) validates our multi-round sampling strategy—these tasks are neither trivially easy nor impossibly hard, but rather exhibit outcome variance across runs. Design implication: Future work should focus on converting the 0/5 category (capability gaps) into the 1-2/5 category (stochastic successes), which can then be reliably solved via multi-round sampling.

Figure 10: Observer GRPO training effect by task type (best@5). Detection improves dramatically (+25.5pp) as the Observer learns systematic anomaly-detection patterns. Localization decreases (

-

4.5pp) due to over-exploration causing multi-anomaly confusion (detailed in Section E.4 ). RCA improves (+8.4pp) from better evidence-gathering strategies. Critical insight: GRPO training is not uniformly beneficial—it helps exploration-heavy tasks but can harm precision-critical tasks where thoroughness conflicts with efficiency.

Figure 11: Task-level changes after GRPO training. Net +9 tasks improved (11 gains, 2 losses). Both degraded tasks are Localization, confirming that GRPO’s deep-exploration strategy harms precision-critical fault localization. Interpretation: The asymmetric gains (strong Detection improvements, Localization degradations) suggest task-type-aware training as future work.

Task Type	Training Tasks	Test Tasks
Detection	10 (43.5%)	22
Localization	6 (26.1%)	22
Analysis (RCA)	1 (4.3%)	12
Mitigation	6 (26.1%)	7
Total	23	63

Appendix

Table 10: Training set composition by task type. The heavy Detection bias (43.5%) explains why Detection benefits most from GRPO.

Figure 12: Observer-GRPO vs Base performance by task type (best@5). Detection improves by +36.4pp (from 59.1% to 95.5%), while Localization degrades by

-

18.2pp (50.0% to 31.8%). Key takeaway: GRPO benefits evidence-gathering tasks (Detection, Analysis) but harms precision tasks (Localization) where over-exploration introduces confusion. This heterogeneity motivates task-type-aware exploration policies.

Task Type	Obs-GRPO	Base	Diff.
Detection	11.0	1.9	+9.1
Localization	11.2	1.8	+9.4
Analysis	9.6	1.1	+8.5
Mitigation	12.0	2.8	+9.2
Average	10.9	1.9	+9.0

Appendix

Table 11: Average exploration steps by task type (lower = more efficient).

Figure 13: Exploration depth vs success rate trade-off. Each point represents a task type; arrows show the transition from Base to Observer-GRPO. GRPO uniformly increases exploration depth (arrows point right), but success direction varies: upward for Detection/Analysis (more exploration helps), downward for Localization (more exploration harms). Critical implication: A single exploration strategy cannot optimize all task types simultaneously—future work should adapt exploration depth based on detected task category.

Outcome	Count	%
Obs-GRPO wins (Base fails)	2	9.1%
Base wins (Obs-GRPO fails)	6	27.3%
Both succeed	5	22.7%
Both fail	9	40.9%
best@5	Obs-GRPO: 31.8% vs Base: 50.0%

Appendix

Table 12: Localization task outcomes (22 tasks total).

Outcome	Count	%
Obs-GRPO wins (Base fails)	2	16.7%
Base wins (Obs-GRPO fails)	0	0.0%
Both succeed	3	25.0%
Both fail	7	58.3%
best@5	Obs-GRPO: 41.7% vs Base: 25.0%

Appendix

Table 13: Analysis task outcomes (12 tasks total).

Figure 14: Task outcome distribution for Localization vs Analysis. Localization shows Base advantage (6 exclusive wins vs 2), while Analysis shows Obs-GRPO advantage (2 vs 0). Interpretation: Confirms that deep exploration helps Analysis (which benefits from comprehensive evidence) but harms Localization (where first instinct is often correct). The asymmetry validates task-type-specific exploration strategies.

Task Type	Optimal Strategy	Why
Detection	Shallow OK	Any anomaly suffices
Localization	Shallow better	Avoid multi-anomaly confusion
Analysis	Deep better	Need comprehensive evidence
Mitigation	Deep required	Must understand before fixing

Appendix

Table 14: Exploration depth trade-off by task type.

Figure 15: Exploration strategy effectiveness heatmap. Green = strategy helps; Red = strategy harms. Localization uniquely benefits from shallow exploration, while Analysis requires deep exploration. This heterogeneity strongly motivates task-type-aware policies as future work.

Problem ID	R1	R2	R3	R4	R5	Best
assign_to_non_existent_node-det-1	T	T	T	T	T	5/5
astronomy_shop_ad_service_failure-det-1	F	T	F	T	F	2/5
astronomy_shop_ad_service_high_cpu-det-1	T	T	F	T	T	4/5
astronomy_shop_ad_service_manual_gc-det-1	F	T	T	T	F	3/5
astronomy_shop_cart_service_failure-det-1	F	T	F	T	F	2/5
astronomy_shop_image_slow_load-det-1	F	T	T	T	T	4/5

Appendix

Table 15: Detailed results: Detection tasks (32 total). High variance across runs indicates stochastic sensitivity to exploration paths.

Problem ID	R1	R2	R3	R4	R5	Best
assign_to_non_existent_node-loc-1	T	T	T	T	T	5/5
astronomy_shop_ad_service_failure-loc-1	F	T	F	F	F	1/5
astronomy_shop_ad_service_high_cpu-loc-1	F	T	F	F	T	2/5
astronomy_shop_ad_service_manual_gc-loc-1	F	F	F	F	F	0/5
astronomy_shop_cart_service_failure-loc-1	F	F	F	F	F	0/5
astronomy_shop_image_slow_load-loc-1	F	F	F	F	F	0/5

Appendix

Table 16: Detailed results: Localization tasks (28 total).

Problem ID	R1	R2	R3	R4	R5	Best
assign_to_non_existent_node-rca-1	F	F	F	F	F	0/5
auth_miss_mongodb-rca-1	F	F	F	F	F	0/5
k8s_target_port-misconfig-rca-1	F	F	F	F	F	0/5
k8s_target_port-misconfig-rca-2	T	F	F	F	F	1/5
k8s_target_port-misconfig-rca-3	F	F	T	F	F	1/5
misconfig_app_hotel_res-rca-1	F	F	F	T	F	1/5

Appendix

Table 17: Detailed results: RCA tasks (13 total).

Problem ID	R1	R2	R3	R4	R5	Best
assign_to_non_existent_node-mit-1	T	T	T	T	T	5/5
auth_miss_mongodb-mit-1	F	F	F	F	F	0/5
k8s_target_port-misconfig-mit-1	T	F	F	F	F	1/5
k8s_target_port-misconfig-mit-2	F	F	F	T	F	1/5
k8s_target_port-misconfig-mit-3	F	F	T	F	F	1/5
misconfig_app_hotel_res-mit-1	F	F	F	F	F	0/5

Appendix

Table 18: Detailed results: Mitigation tasks (13 total).

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 585 papers

AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compression
Dec 15, 2025Zishan Bai, Hanxuan Chen, Jiayi Gu +9Multi-Agent Orchestration LLM Context Management
Columbia University · Hunan University · Central University of Finance and Economics +8
A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis
Jun 28, 2026Yuanhong Cai, Xiaohui Nie, Kanglin Yin +8Software Engineering LLM Agent Evaluation
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Key Laboratory for Satellite Digitalization Technology, Chinese Academy of Sciences, Shanghai, China · Nankai University, Tianjin, China +2
AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
Feb 2, 2026Shraddha Barke, Arnav Goyal, Alind Khare +3Fault Diagnosis AI Agent Reliability
1Microsoft Research · 2Microsoft · 3UIUC

Dec 15, 2025J/K move · Enter open · S save

AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compression

Zishan Bai, Hanxuan Chen, Jiayi Gu, Wenqian Weng+8

Columbia University · Hunan University · Central University of Finance and Economics+8

Multi-Agent Orchestration LLM Context Management Memory-Augmented Agents LLM Agent Reliability

Open paper AskPDF arXiv

Cloud-native systems have made operational work both more powerful and harder to automate: incidents unfold across microservices, logs and metrics arrive faster than operators can inspect them, and recovery actions must be coordinated without losing the causal context that makes them safe. We present AOI (AI-Oriented Operations), a context-aware multi-agent framework for autonomous IT operations. AOI separates operational responsibility across an Observer, a read-only Probe, and a guarded Executor, and connects them through dynamic scheduling and a hierarchical memory system with LLM-based context compression. This design turns long-running incident response into an iterative loop of observation, evidence gathering, safe intervention, and memory update. Across AIOpsLab simulations and real-world Loghub-derived scenarios, AOI improves task success to 94.2%, reduces mean time to resolution by 34.4% relative to the strongest baseline, and compresses operational context by 72.4% while preserving 92.8% of diagnostic information. Ablations show that these gains come from the combination of agent specialization, adaptive scheduling, and memory-aware compression rather than from any single module alone. The results suggest that autonomous operations systems can move beyond alert classification toward reliable, context-preserving recovery in complex infrastructure.

Abstract

LLM Agent Training

Explore similar work

AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compression

A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories