The Abstention Protocol: RCA for Clos Fabrics

Madhava Gaikwad; Deepak Pandey

cs.AIAug 11, 2026

The Abstention Protocol: RCA for Clos Fabrics

Authors: Madhava Gaikwad, Deepak Pandey

Organizations: Independent · Microsoft

Abstract

Root cause analysis (RCA) in large datacenter networks is challenging because telemetry is noisy, partial, and asynchronous. Score-based approaches degrade under these conditions, often yielding unstable or incorrect attributions. We present CoreSec, a production RCA system that replaces weighted fusion with a PAM-style abstention algebra. Telemetry agents are composed using control flags that yield deterministic decisions and explicit abstention when evidence is ambiguous. CoreSec combines this algebra with topology-aware configurations that capture failure surfaces across Clos fabrics and converge monotonically as evidence accumulates. Deployed at hyperscale, CoreSec provides stable and explainable RCA behavior across diverse environments without retuning. Our experience shows that structured composition with abstention forms a practical foundation for automated RCA in real-world cloud networks.

Figures & tables

Figure 1: Three-tier Clos topology. Servers connect to TORs, TORs to T1 aggregation switches, and T1s to T2 spines.

Panel of Figure 2 — Figure 2: CoreSec dataflow. An incident triggers five parallel configurations; hierarchy heuristics combine their votes into one attribution or an abstention.

#	Failure surface	Authoritative agents
1	Server–TOR cable	port counters, NIC failsafe, RDMA timeout
2	TOR switch	aggregate CRC, host failures, control plane events
3	Switch–switch cable	active path probes, T1 counters
4	T1 switch	multi-uplink probes (NetBouncer-style)
5	T2 switch	end-to-end multi-path probes

Table 1: Five configurations and their authoritative agents.

	Weight-Based RCA		CoreSec (current)
Metric	Count	%	Count	%
Total incidents processed	712,345	100%	712,345	100%
Abstentions (false negatives)	0	0%	$\approx$ 10,685	1.5%
Automatically attributed	712,345	100%	$\approx$ 701,660	98.5%
False positives	128,222–156,716	18–22%	$<$ 7,017	$<$ 1%
Misattributions	—	—	$\approx$ 4/year	—

Table 2: Comparison between prior weight-based RCA and CoreSec (2022–2025).

Figure 4: FP–FN tradeoff over three years. Abstention dropped from 10% to 1.5% as agents were added.

Root Cause Category	Fraction of Incidents
Cable faults (server–TOR, switch–switch)	70%
TOR switch failures	28%
T1/T2/cluster-level failures	2%

Table 3: Distribution of root causes identified by CoreSec.

Figure 5: Threshold sensitivity. Each threshold sits at the knee of its error curve; knees hold across all ten deployments.

System	Focus	Method	Incident-scoped	Handles gaps	Abstains
Pingmesh [ 14 ]	Path loss and latency	Active probing	No	No	No
NetBouncer [ 37 ]	Device/link localization	Score-based voting	No	Renormalize	No
007 [ 5 ]	Faulty path inference	End-host voting	No	Majority vote	No
Everflow [ 44 ]	Packet-level debugging	Packet mirroring	No	No	No
Hostmesh [ 25 ]	RoCE link diagnosis	Topology-aware probes	No	No	No
AutoARTS [ 9 ]	Incident tagging	ML classification	Yes	Retrain	No

Table 4: Comparison of fault localization and RCA systems.

Root Cause Analysis

Root cause analysis identifies the initiating faults behind observed failures by tracing symptoms through causal and system dependencies.

See all 80 papers

Graphical Causal Reasoning for Root Cause Analysis in Cloud Networks
Jun 11, 2026Fabien Chraim, Dominik Janzing, John EvansTime-Series Causal Discovery Cloud Computing
Amazon Web Services
Root Cause Analysis with Latent Confounders using Partial Ancestral Graphs
Jun 18, 2026Henrique O. Caetano, Rafael Arone, Carlos Dias MacielCausal Inference with Unobserved Confounding Partial Identification
Signal Processing Laboratory, São Carlos School of Engineering, University of São Paulo, São Paulo, Brazil · Guaratinguetá School of Engineering, São Paulo State University, Guaratinguetá, Brazil
StableRCA: Robust Graph-Agnostic Mechanism-Level Root Cause Analysis
Jun 4, 2026Xiaoyu Lin, Nicholas Tagliapietra, Kehan Li +2Root Cause Analysis Causal Intervention
Computer Science Department TU Darmstadt, Germany

Jun 11, 2026J/K move · Enter open · S save

Graphical Causal Reasoning for Root Cause Analysis in Cloud Networks

Fabien Chraim, Dominik Janzing, John Evans

Amazon Web Services

Time-Series Causal Discovery Cloud Computing Causal Reasoning Root Cause Analysis

Open paper AskPDF arXiv

Cloud-computing relies on large-scale networks which are inherently complex systems. In this paper, we present a novel approach to root cause analysis (RCA) of cloud network incidents, leveraging graph-based causal discovery techniques. Our method addresses the limitations of rule-based automation by introducing a spatiotemporal grouping strategy and an automation ontology to reduce the dimensionality of the problem. We construct a causal graph from binary time series data using bivariate Granger causality and conditional independence tests. For inference, we introduce a probabilistic method that assigns edge-specific conditional probabilities as a function of time lag, allowing for interpretable, time-aware root cause scoring via causal graph traversal. We evaluated the system using a labeled dataset of 35 production incidents from a major cloud provider. The model successfully recalled the correct root cause in 85.7% of incidents and produced an exact match in 74.3%. In production, the deployed system has been used in over 800 real-world incidents, with positive qualitative feedback from network engineers. These results highlight the practicality of a data-driven, causal approach to RCA in dynamic and large-scale operational environments.

Abstract

Root Cause Analysis

Explore similar work

Graphical Causal Reasoning for Root Cause Analysis in Cloud Networks

Root Cause Analysis with Latent Confounders using Partial Ancestral Graphs

StableRCA: Robust Graph-Agnostic Mechanism-Level Root Cause Analysis