MetaKube: An Experience-Aware LLM Framework for Kubernetes Failure Diagnosis

Wei Sun; Ting Wang; Xinran Tian; Wanshun Lan; Xuhan Feng; Haoyue Li; Fangxin Wang

cs.LGMar 24, 2026

MetaKube: An Experience-Aware LLM Framework for Kubernetes Failure Diagnosis

Authors: Wei Sun, Ting Wang, Xinran Tian, Wanshun Lan, Xuhan Feng, Haoyue Li, Fangxin Wang

Organizations: School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen; Shenzhen Future Network of Intelligence Institute, Shenzhen, China · Shenzhen Future Network of Intelligence Institute, Shenzhen, China · China Mobile Communications Group Guangdong Co., Ltd., Guangzhou, China

Abstract

Existing LLM-based Kubernetes diagnostic systems cannot learn from operational experience, operating on static knowledge bases without improving from past resolutions. We present MetaKube, an experience-aware LLM framework through three synergistic innovations: (1) an Episodic Pattern Memory Network (EPMN) that abstracts diagnostic patterns from historical resolutions and provides confidence-calibrated retrieval for both rapid pattern matching and guided causal exploration, (2) a meta-cognitive controller that dynamically routes between intuitive and analytical pathways based on problem familiarity, optimizing the trade-off between speed and depth, and (3) KubeLLM, a locally-deployable 8B model enhanced through domain-specific post-training on our 7,000-sample Kubernetes Fault Resolution Dataset. Evaluation on 1,873 real-world scenarios demonstrates MetaKube transforms Qwen3-8B from 50.9 to 90.5 points, approaching GPT-4.1 performance while ensuring complete data privacy. EPMN contributes 15.3% improvement through experiential learning, with continuous learning experiments showing progressive gains as the system accumulates operational knowledge. The source code and related resources are available at https://github.com/MetaKube-LLM-for-Kubernetes-Diagnosis/MetaKube.

Figures & tables

Figure 1. The MetaKube dual-pathway diagnostic architecture. The system integrates memory-augmented pattern recognition (EPMN) with causal reasoning (KubeGraph) through two processing pathways: an intuitive pathway for rapid pattern-based diagnosis and an analytical pathway for complex causal analysis. A meta-cognitive controller dynamically routes queries based on confidence assessment, optimizing the trade-off between diagnostic speed and depth.

Figure 2. Construction of the Kubernetes Fault Resolution Dataset. Generate additional synthetic examples after collecting authentic Kubernetes troubleshooting cases, then performed deduplication, and enhanced the question-answer pairs with chain-of-thought reasoning using LLM.

Method	GPT-5 Automated Assessment					Human Expert Evaluation
Method	Eff.	Equ.	Com.	S/A	Avg.	Eff.	Equ.	Com.	S/A	Avg.
GPT-4.1 (Zero-shot)	72.1±4.2	74.3±3.8	69.8±4.5	78.9±3.1	73.8±2.9	56.4±9.1	59.2±9.8	52.7±9.4	64.8±8.2	58.3±7.3
GPT-4.1-mini (Zero-shot)	61.5±5.1	63.8±4.6	59.2±5.4	69.7±4.2	63.6±3.7	45.1±10.3	47.6±10.9	41.8±10.1	55.4±9.5	47.5±8.4
Qwen3-8B (Zero-shot)	48.7±6.8	51.2±6.3	46.1±7.1	57.4±5.7	50.9±5.2	31.5±12.1	35.8±11.6	28.9±12.4	42.3±10.8	34.6±9.7
GPT-4.1 (GraphRAG)	89.3±2.9	92.6±2.4	91.4±2.7	94.1±2.3	91.9±1.8	73.8±7.3	77.8±7.9	71.2±8.2	79.4±7.1	75.6±6.1
GPT-4.1-mini (GraphRAG)	79.8±4.1	81.3±3.7	78.4±4.5	85.2±3.2	81.2±2.9	59.3±9.1	61.7±8.8	57.4±9.4	68.9±8.3	61.8±7.2

Table 1. Overall performance comparison on KubeFault dataset (100-point scale).

Figure 3. EPMN ablation study results on KubeFault.

Figure 4. Performance scores of different models on the KFRD test set (5-point scale).

Dataset	Eff.	Equ.	Com.	S/A	Avg.
KubeFault
w/ KubeGraph	75.6±8.1	74.2±8.5	69.8±7.8	81.2±7.2	75.2±6.4
w/o KubeGraph	31.5±12.1	35.8±11.6	28.9±12.4	42.3±10.8	34.6±9.7
Telecom Dataset
w/ KubeGraph	58.4±9.2	55.7±9.6	52.3±8.9	64.1±8.4	57.6±7.8
w/o KubeGraph	19.8±13.5	24.2±12.8	17.1±13.7	28.5±11.9	22.4±11.2

Table 2. KubeGraph Ablation Results.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Category	Focus Areas
Resource Errors	OOMKilled events, CPU throttling, resource quotas, PVC mounting failures, memory leaks, autoscaler misconfigurations
Network Errors	Service discovery failures, DNS resolution, network policies, ingress configuration, load balancer issues, CNI problems
Scheduling Errors	Node affinity violations, taint/toleration mismatches, insufficient resources, pod priority conflicts, DaemonSet placement
Image Errors	ImagePullBackOff, registry authentication, rate limiting, private registry access, multi-arch compatibility, layer corruption
Configuration Errors	ConfigMap/Secret mounting, RBAC permissions, admission webhooks, environment variables, Helm values, operator configs
System Errors	Container runtime failures, kubelet crashes, etcd issues, certificate expiration, kernel panics, filesystem corruption

Appendix

Table 3. Knowledge categories and focus areas in KubeGraph

Fault Diagnosis

Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.

See all 147 papers

TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes
Sep 20, 2026Rohit Patel, Susil Kumar Mohanty, Jeenal ChaudharyLLM Inference Parallel LLM Inference
Department of Computer Science and Engineering Indian Institute of Technology Jodhpur, Jodhpur, India
A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis
Jun 28, 2026Yuanhong Cai, Xiaohui Nie, Kanglin Yin +8Software Engineering LLM Agent Evaluation
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Key Laboratory for Satellite Digitalization Technology, Chinese Academy of Sciences, Shanghai, China · Nankai University, Tianjin, China +2
OpsMem: Dual-Memory Reasoning with Cross-Memory Resonance for Failure Diagnosis
Jul 13, 2026Yongqian Sun, Rongchen Gao, Yu Luo +6LLM Agent Memory Agent Memory Management
Nankai University · Huawei Technologies Co., Ltd. · Tsinghua University

Sep 20, 2026J/K move · Enter open · S save

TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes

Rohit Patel, Susil Kumar Mohanty, Jeenal Chaudhary

Department of Computer Science and Engineering Indian Institute of Technology Jodhpur, Jodhpur, India

LLM Inference Parallel LLM Inference Cloud Computing Indirect Prompt Injection Root Cause Analysis

Open paper AskPDF arXiv

Root cause analysis at a remote site is slow: evidence is scattered across pod logs, Kubernetes events and cluster-level objects, and many operators cannot send production logs to a hosted model at all. On-premise inference removes the second constraint but raises a question live-cluster benchmarks have not addressed: when one workstation GPU fixes both the model and the context budget, how should evidence be retrieved, and what happens when the runbooks the model consults have been tampered with? We present TriFleetRCA, a pipeline running entirely on one on-premise GPU that collects evidence at one of three scopes (pod, namespace, cluster), ranks it by template de-duplication then BM25, filters runbooks through an ingest guard, and returns a root cause with the evidence lines supporting it. We evaluate on a live Kubernetes cluster into which we inject four faults, so ground truth is known by construction, across 100 analyses with Qwen2.5-14B-Instruct at temperature 0. The hit rate was 0.85, 0.90 and 0.95 at pod, namespace and cluster scope; intervals overlap, but the whole scope effect comes from the one fault whose cause is a cluster-level object, and cluster scope costs 55% more tokens. De-duplication before ranking raised the hit rate from 0.75 to 0.90 at equal token cost. A poisoned runbook telling the model to delete the namespace was rejected by the guard every run; with the guard disabled the model declined to follow it in all 20 analyses, making the guard defence in depth rather than the sole barrier. Separating citation quality from accuracy proved informative: one fault was diagnosed correctly and cited incorrectly every trial, a failure mode accuracy conceals. Median latency was 1.6 s at 2,200 prompt tokens. We release the pipeline, the fault injector and all records.

Abstract

Fault Diagnosis

Explore similar work

TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes

A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis

OpsMem: Dual-Memory Reasoning with Cross-Memory Resonance for Failure Diagnosis