Collaborative AI Agents and Critics for Fault Detection and Cause Analysis in Network Telemetry

Syed Eqbal Alam; Zhan Shu

cs.AIMar 31, 2026

Collaborative AI Agents and Critics for Fault Detection and Cause Analysis in Network Telemetry

Authors: Syed Eqbal Alam, Zhan Shu

Organizations: Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada · SheQAI Research, Edmonton, Alberta, Canada

Abstract

We develop algorithms for collaborative control of AI agents and critics in a multi-actor, multi-critic federated multi-agent system. Each AI agent and critic has access to classical machine learning or generative AI foundation models. The AI agents and critics collaborate with a central server to complete multimodal tasks such as fault detection, severity, and cause analysis in a network telemetry system, text-to-image generation, video generation, healthcare diagnostics from medical images and patient records, etcetera. The AI agents complete their tasks and send them to AI critics for evaluation. The critics then send feedback to agents to improve their responses. Collaboratively, they minimize the overall cost to the system with no inter-agent or inter-critic communication. AI agents and critics keep their cost functions or derivatives of cost functions private. Using multi-time scale stochastic approximation techniques, we provide convergence guarantees on the time-average active states of AI agents and critics. The communication overhead is a little on the system, of the order of $\mathcal{O}(m)$ , for $m$ modalities and is independent of the number of AI agents and critics. Finally, we present an example of fault detection, severity, and cause analysis in network telemetry and thorough evaluation to check the algorithm's efficacy.

Figures & tables

Figure 1: Block diagram for essential components of AI agent-critic interaction. Models represent the foundation models; for example, large language models, large vision models, etc.

Panel of Figure 2 — Figure 2: Density plots for input data rate, output data rate, bandwidth, and bytes sent for the CSV file (all rows) of the network telemetry dataset $4$ of [ 15 ] .

Panel of Figure 3 — Figure 3: The plots for input data rate, load interval, bandwidth, output drops versus time for a few data points from the CSV dataset.

Figure 4: Federated multi-agent system’s block diagram: AI agents and critics coordinate with a central server to complete multimodal tasks.

\displaystyle B^{c}_{j\nu}(k)\leftarrow\begin{cases}1\quad\text{w. p. }\sigma^{c}_{j\nu}(\Theta^{c}_{j}(k),y_{j\nu}(k));\\ 0\quad\text{w. p. }1-\sigma^{c}_{j\nu}(\Theta^{c}_{j}(k),y_{j\nu}(k)).\end{cases}

Algorithm 3 Algorithm of AI critic

\nu

Panel of Figure 5 — Figure 5: The convergence of time-average active states $x_{1i}$ and $y_{2i}$ of AI agents performing tasks of (a) modality $1$ and (b) modality $2$ , respectively, to close to their respective optimal values $x_{1i}^{*}$ and $x_{2i}^{*}$ .

Panel of Figure 6 — Figure 6: The convergence of time-average active states $y_{1\nu}$ and $y_{2\ell}$ of AI critics evaluating tasks of (a) modality $1$ and (b) modality $2$ , respectively, to close to their respective optimal values $y^{*}_{1\nu}$ and $y^{*}_{2\ell}$ , $\nu=1,2,\ldots,C_{1}$ , and $\ell=1,2,\ldots,C_{2}$ .

Figure 7: The evolution of the ratio of total cost obtained by the developed algorithm and the total optimal cost obtained by the CVX solver.

Figure 8: The evolution of feedback signals $\Theta_{1}(k)$ and $\Theta_{2}(k)$ broadcast by the central server for agents completing tasks of modalities $1$ and $2$ , respectively.

Figure 9: The evolution of feedback signals $\Theta^{c}_{1}(k)$ and $\Theta^{c}_{2}(k)$ broadcast by the central server for critics evaluating tasks of modalities $1$ and $2$ , respectively.

Models	Accuracy	Precision	Recall	F1-Score
Agents with XG Boosting	0.9644	0.9397	0.9929	0.9656
Agents and Critics enabled with Llama3.2 (with RAG)	0.5882	0.8000	0.6154	0.6957
Agents and Critics enabled with Llama3.2 (without RAG)	0.4612	0.4027	0.6648	0.5016

Table 1: Evaluation of the algorithms for fault detection on network telemetry dataset

4

of [ 15 ] : Agents are enabled with either classical machine learning techniques (e.g., XG Boosting) or large language models with (or without) retrieval augmented generation (RAG). The agents enabled with XG Boosting perform better than the large language model-enabled agents on the dataset.

Models	Actor	Query	Severity	Cause Summary
Llama3.2		24. Predict whether Device with ‘name’: ‘Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate’, ‘time’: ‘2017-09-01 10:23:18.222000’, ‘EncodingPath’: ‘Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate’, ‘Producer’: ‘leaf3’, ‘bandwidth’: ‘100000000.0’, ‘input-data-rate’: ‘10261058.0’, ‘input-load’: ‘26.0’, ‘input-packet-rate’: ‘166870.0’, ‘interface-name’: ‘HundredGigE0/0/0/5’, ‘load-interval’: ‘0.0’, ‘output-data-rate’: ‘10482276.0’, ‘output-load’: ‘26.0’, ‘output-packet-rate’: ‘449830.0’, ‘peak-input-data-rate’: ‘0.0’, ‘peak-input-packet-rate’: ‘0.0’, ‘peak-output-data-rate’: ‘0.0’, ‘peak-output-packet-rate’: ‘0.0’, ‘reliability’: ‘255.0’ has a Critical fault or Non-critical fault.	Non-critical	The device experienced a sudden decrease in input data rate due to a likely temporary network performance issue with the ’HundredGigE0/0/0/5’ interface. However, it is stated that the interface is operational, indicating it’s not a critical failure.
DeepSeek-R1			Non-critical	Slight inefficiency due to output data rate increase relative to input with observed packet loss.
Mistral			Non-critical	The high levels of data rates and load on interface ’HundredGigE0/0/0/5’ are within acceptable bounds for a production network, indicating normal operation. However, it is advised to monitor these values over time to ensure they do not exceed thresholds, indicating potential network congestion or performance degradation.
Llava:7b	Agent		Critical	An issue with the network interface is causing output drops on the device, which could impact network performance and reliability.
IBM’s Granite3.2:8b			Non-critical	The device’s data rates, packet rates, and reliability metric all indicate normal operation with no unexpected spikes or drops in traffic. Although the load is high at 26.0, it does not signify a fault but heavy utilization, supported by the absence of critical-indicating peak values (peak-input/output-data-rate, peak-input/output-packet-rate).
Microsoft’s Phi4:14b			Non-critical	The interface operates within expected parameters, with no indicators of bandwidth saturation, significant packet loss, or unreliability. Metrics such as data rate and load are normal, and reliability is at maximum without errors affecting performance. Despite some high values mentioned elsewhere, none relate to a critical fault for this specific instance.

Table 2: Fault severity and cause analysis for agents enabled with large language models. The agents classify the query as Non-critical or Critical and briefly summarize the fault cause. The agents have access to network telemetry logs that were vector-embedded and stored in ChromaDB. Note that the large language models generated the text in the Cause Summary .

Models	Actor	Query	Severity	Cause Summary
Llama3.2		“Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’time’: ’2017-09-01 10:35:19.556000’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’Producer’: ’leaf3’, ’bytes-received’: ’934494113183718.0’, ’bytes-sent’: ’2736075649406230.0’, ’carrier-transitions’: ’0.0’, ’crc-errors’: ’0.0’, ’input-drops’: ’0.0’, ’input-errors’: ’0.0’, ’input-ignored-packets’: ’0.0’, ’input-queue-drops’: ’0.0’, ’interface-name’: ’HundredGigE0/0/0/5’, ’output-buffer-failures’: ’0.0’, ’output-drops’: ’19401038.0’, ’output-errors’: ’0.0’, ’output-queue-drops’: ’0.0’, ’packets-received’: ’224932326520.0’, ’packets-sent’: ’1573333104341.0’ has a Critical fault or Non-critical fault.”	Critical	High data rate on the interface combined with high packet loss indicates a potential interface issue.
DeepSeek-R1			Critical	The high number of output drops (over 19 million) suggests that there may be an issue with data transmission capacity or buffer overflow on the output link interface, potentially indicating hardware exhaustion or configuration errors leading to packet loss.
Llava:7b	Agent		Critical	The interface has experienced output queue drops, which can result in lost data and reduced network performance. This can have serious implications for the functionality of the network device.
Mistral		“Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’time’: ’2017-09-01 10:17:09.336000’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’Producer’: ’leaf3’, ’bytes-received’: ’5344118565224976.0’, ’bytes-sent’: ’5096276026294021.0’, ’carrier-transitions’: ’0.0’, ’crc-errors’: ’390.0’, ’input-drops’: ’0.0’, ’input-errors’: ’390.0’, ’input-ignored-packets’: ’0.0’, ’input-queue-drops’: ’0.0’, ’interface-name’: ’HundredGigE0/0/0/30’, ’output-buffer-failures’: ’0.0’, ’output-drops’: ’2573752423.0’, ’output-errors’: ’0.0’, ’output-queue-drops’: ’0.0’, ’packets-received’: ’1974518315803.0’, ’packets-sent’: ’2244342803311.0’ has a Critical fault or Non-critical fault.”	Critical	Potential network congestion, cable problems or misconfiguration due to high input errors and output drops on the ’HundredGigE0/0/0/30’ interface. Additionally, investigate possible causes for CRC errors across all interfaces.

Table 3: Fault severity and cause analysis: The table presents the text queries classified as ‘Critical’ by different LLMs. The LLMs generated the text in the Cause Summary .

Models	Actor	Query	Severity	Cause Summary
IBM’s Granite3.2:8b		“Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate’, ’time’: ’2017-09-01 10:31:39.580000’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate’, ’Producer’: ’spine2’, ’bandwidth’: ’100000000.0’, ’input-data-rate’: ’10273100.0’, ’input-load’: ’26.0’, ’input-packet-rate’: ’227678.0’, ’interface-name’: ’HundredGigE0/0/0/17’, ’load-interval’: ’0.0’, ’output-data-rate’: ’12093337.0’, ’output-load’: ’30.0’, ’output-packet-rate’: ’408305.0’, ’peak-input-data-rate’: ’0.0’, ’peak-input-packet-rate’: ’0.0’, ’peak-output-data-rate’: ’0.0’, ’peak-output-packet-rate’: ’0.0’, ’reliability’: ’255.0’ has a Critical fault or Non-critical fault.”	Critical	The high reliability metric (255) indicates a potential interface failure, which can lead to significant network disruptions. Despite no individual metrics indicating critical faults, the overall non-responsive or malfunctioning state of ’HundredGigE0/0/17’ is classified as critical due to its impact on network operations.
Microsoft’s Phi4:14b		Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’time’: ’2017-09-01 10:18:59.451000’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’Producer’: ’leaf3’, ’bytes-received’: ’5344118565224976.0’, ’bytes-sent’: ’5096276026294021.0’, ’carrier-transitions’: ’0.0’, ’crc-errors’: ’390.0’, ’input-drops’: ’0.0’, ’input-errors’: ’390.0’, ’input-ignored-packets’: ’0.0’, ’input-queue-drops’: ’0.0’, ’interface-name’: ’HundredGigE0/0/0/30’, ’output-buffer-failures’: ’0.0’, ’output-drops’: ’2573752423.0’, ’output-errors’: ’0.0’, ’output-queue-drops’: ’0.0’, ’packets-received’: ’1974518315803.0’, ’packets-sent’: ’2244342803311.0’ has a Critical fault or Non-critical fault	Critical	There is a possibility that the metrics are being incorrectly scaled or recorded due to misconfiguration or reporting inaccuracies by the telemetry system.

Table 4: Fault severity and cause analysis: The table presents the text queries classified as ‘Critical’ by different LLMs. The LLMs generated the text in the Cause Summary .

Models	Actor	Query	Severity	Cause Summary
Llama3.2		“Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’time’: ’2017-09-01 10:35:46.985999’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’Producer’: ’leaf3’, ’bytes-received’: ’659311227161861.0’, ’bytes-sent’: ’388892590647867.0’, ’carrier-transitions’: ’0.0’, ’crc-errors’: ’0.0’, ’input-drops’: ’0.0’, ’input-errors’: ’0.0’, ’input-ignored-packets’: ’0.0’, ’input-queue-drops’: ’0.0’, ’interface-name’: ’HundredGigE0/0/0/8’, ’output-buffer-failures’: ’0.0’, ’output-drops’: ’0.0’, ’output-errors’: ’0.0’, ’output-queue-drops’: ’0.0’, ’packets-received’: ’165436307368.0’, ’packets-sent’: ’112420913236.0’ has a Critical fault or Non-critical fault.”	Critical	No Feedback
DeepSeek-R1		Predict whether Device with ’name’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’time’: ’2017-09-01 10:17:40.959000’, ’EncodingPath’: ’Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters’, ’Producer’: ’spine2’, ’bytes-received’: ’765732261758138.0’, ’bytes-sent’: ’692455587027158.0’, ’carrier-transitions’: ’0.0’, ’crc-errors’: ’0.0’, ’input-drops’: ’0.0’, ’input-errors’: ’0.0’, ’input-ignored-packets’: ’0.0’, ’input-queue-drops’: ’0.0’, ’interface-name’: ’HundredGigE0/0/0/14’, ’output-buffer-failures’: ’0.0’, ’output-drops’: ’0.0’, ’output-errors’: ’0.0’, ’output-queue-drops’: ’0.0’, ’packets-received’: ’257183573922.0’, ’packets-sent’: ’295378443912.0’ has a Critical fault or Non-critical fault.	Non-critical	No significant issues observed with the interface operational state or data rates within acceptable limits.
Mistral	Critic		Non-critical	The given query data does not indicate a critical fault in the device’s interface. The high traffic and data rates shown can be seen as heavy load, which is generally considered non-critical, but monitoring for potential issues in the near future is recommended. To mitigate this, network administrators might consider optimizing traffic or upgrading bandwidth accordingly.
Llava:7b			Non-critical	No Feedback
IBM’s Granite3.2:8b			Non-critical	The provided data does not indicate any critical faults on interface HundredGigE0/0/0/14. All counters such as bytes-received, bytes-sent, carrier-transitions, crc-errors, input-drops, input-errors, input-ignored-packets, input-queue-drops, output-buffer-failures, output-drops, output-errors, and output-queue-drops are either zero or within normal operation ranges. The interface is up and functioning normally according to the given metrics.
Microsoft’s Phi4:14b			Non-critical	No Feedback

Table 5: Fault severity and cause analysis for critics enabled with large language models. The critics evaluate the responses of agents and send them feedback for improvement. They also classify the query as Non-critical or Critical and briefly summarize the fault cause. The critics also have access to network telemetry logs that were vector-embedded and stored in ChromaDB. Note that the large language models generated the text in the Cause Summary .

Figure 10: The proportion of critical and non-critical faults classified by various open-source large language models. The green slice represents the proportion of non-critical faults, and the exploded red slice represents the proportion of critical faults.

Figure 11: The UpSet plot illustrates the intersection of sets of faults classified as critical by various LLMs. The bars on the bottom left represent the number of critical faults classified by the LLMs. The bars above the single solid dot represent the number of faults classified as critical only by that large language model (a disjoint set). The connecting solid dots represent the common queries classified as critical by the connecting LLMs.

Explore similar work

Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems
Apr 6, 2026Anshul Pathak, Nishant JainAI Agent Governance Runtime Enforcement for AI Agents
Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution
May 26, 2026Mingyu Lu, Yushan Huang, Chris Lin +1Multi-Agent System Optimization Cooperative Game Theory
Paul G. Allen School of Computer Science & Engineering, University of Washington
Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors
Jul 8, 2026Oliver Makins, Orazio Angelini, Zohreh Shams +1Multi-Agent Control AI Agent Security
UK AI Security Institute · MATS Research

Apr 6, 2026J/K move · Enter open · S save

Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems

Anshul Pathak, Nishant Jain

AI Agent Governance Runtime Enforcement for AI Agents AI Agent Monitoring Multi-Agent Systems

Open paper AskPDF arXiv

Enterprise multi-agent AI systems produce thousands of inter-agent interactions per hour, yet existing observability tools capture these dependencies without enforcing anything. OpenTelemetry and Langfuse collect telemetry but treat governance as a downstream analytics concern, not a real-time enforcement target. The result is an "observe-but-do-not-act" gap where policy violations are detected only after damage is done. We present Governance-Aware Agent Telemetry (GAAT), a reference architecture that closes the loop between telemetry collection and automated policy enforcement for multi-agent systems. GAAT introduces (1) a Governance Telemetry Schema (GTS) extending OpenTelemetry with governance attributes; (2) a real-time policy violation detection engine using OPA-compatible declarative rules under sub-200 ms latency; (3) a Governance Enforcement Bus (GEB) with graduated interventions; and (4) a Trusted Telemetry Plane with cryptographic provenance.

Abstract

Explore similar work

Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors