Enterprise IT support is constrained by heterogeneous devices, evolving policies, and long-tail failure modes that are difficult to resolve centrally. We present VIGIL, an edge-extended agentic AI system that deploys desktop-resident agents to perform situated diagnosis, retrieval over enterprise knowledge, and policy-governed remediation directly on user devices with explicit consent and end-to-end observability. In a 10-week pilot of VIGIL's operational loop on 100 resource-constrained endpoints, VIGIL reduces interaction rounds by 39%, achieves at least 4 times faster diagnosis, and supports self-service resolution in 82% of matched cases. Users report excellent usability, high trust, and low cognitive workload across four validated instruments, with qualitative feedback highlighting transparency as critical for trust. Notably, users rated the system higher when no historical matches were available, suggesting on-device diagnosis provides value independent of knowledge base coverage. This pilot establishes safety and observability foundations for fleet-wide continuous improvement.
Figures & tables
Figure 1: Operational and self-improvement loops within VIGIL. Device-level cognition, the focus of this paper, executes locally on endpoints (shown in blue ), while cloud-level reflection, designed for future activation, refines policies, memory, and prompts over time (shown in red ). Monitoring at the fleet level operates in parallel across both loops (shown in black ).
Figure 2: VIGIL’s agentic architecture integrating local edge agents and remote cloud agents. Edge agents execute diagnosis and remediation on-device, while cloud agents provide coordination, governance, and observability across the fleet.
Tool
Purpose
system_uptime
Retrieves device uptime to identify reboots
security_updates
Detects pending enterprise security updates
cpu_process
Retrieves top running processes by CPU usage
disk_usage
Reports disk usage across all mount points.
network_status
Retrieves network connectivity status
Table 1: Illustrative diagnostic tools used in the diagnose–retrieve–remediate loop.
Metric
SHIELD
Human Support
Interaction rounds (median)
11 cycles
18 turns ∗
Diagnosis time (median)
36.5 s
≥ 2.7 min †
Response quality (median)
8.0 / 10
—
Self-service potential
82% (high + medium confidence)
Table 2: Operational metrics on matched cases (n=60). ∗ Human support rounds reflect full conversational turns between user and support engineer. Interaction rounds were reduced by 39% (83% of cases). † Conservative lower bound assuming 10% of median contact duration (26.5 min) reflects active work; realistic estimates yield 4–17 × speedup.
Figure 3: User experience survey: (a) SUS score distribution, (b) NASA-TLX workload, (c) user trust (bottom-left) and (d) technology acceptance
Instrument
Score
SUS
86.2 ± 15.7
Trust in Automation
4.29 / 5.0
TAM
4.41 / 5.0
NASA-TLX
2.53 / 7.0
Table 3: User experience scores across four validated instruments (n=23). SUS industry average is 68.0.
Figure 4: Normalized summary of the user experience survey across the four validated instruments (n=23)
Layer
Technology
Agent Framework
AWS Strands (Python)
LLM Inference
Claude Opus 4.1
Inter-Agent Protocol
Agent2Agent (A2A)
Packaging
PyInstaller
Observability
OpenTelemetry tracing
Table 4: Core implementation components of VIGIL
Figure 5: Item-level SUS score breakdown (1–5 scale). Positive-framed items (higher is better) and negative-framed items (lower is better) are shown separately. The dashed line indicates the neutral midpoint (3.0).
In large-scale cloud service platforms, thousands of customer tickets are generated daily and are typically handled through on-call dialogues. This high volume of on-call interactions imposes a substantial workload on human support analysts. Recent studies have explored reactive agents that leverage large language models as a first line of support to interact with customers directly and resolve issues. However, when issues remain unresolved and are escalated to human support, these agents are typically disengaged. As a result, they cannot assist with follow-up inquiries, track resolution progress, or learn from the cases they fail to address. In this paper, we introduce Vigil, a novel proactive agent system designed to operate throughout the entire on-call life-cycle. Unlike reactive agents, Vigil focuses on providing assistance during the phase in which human support is already involved. It integrates into the dialogue between the customer and the analyst, proactively offering assistance without explicit user invocation. Moreover, Vigil incorporates a continuous self-improvement mechanism that extracts knowledge from human-resolved cases to autonomously update its capabilities. Vigil has been deployed on Volcano Engine, ByteDance's cloud platform, for over ten months, and comprehensive evaluations based on this deployment demonstrate its effectiveness and practicality. The open source version of this work is publicly available at https://github.com/volcengine/veaiops.
Fengrui Liu, Xiao He, Tieying Zhang
ByteDance Beijing, China · ByteDance Hangzhou, China · ByteDance San Jose, United States
Enterprise support agents operate in rapidly changing environments where policies, product capabilities, and knowledge bases evolve continuously, making static assistants brittle and costly to maintain. We present LinkedIn's self-evolving agentic support system, which integrates retrieval-augmented generation with evolutionary auto-prompting and a modular, production-aligned evaluation framework to enable safe, continuous improvement without retraining foundation models. The system treats prompts, retrieval, and evaluation as a closed-loop, versioned workflow with operational guardrails. Offline simulations and ablations show clear quality gains over vanilla RAG and baseline agents, including reduced hallucinations and improved response completeness. In a two-week user-randomized A/B test on LinkedIn's production support traffic, the integrated self-evolved workflow increased QA self-serve by 9.0 percentage points, cancellation self-serve by 4.8 points, and routing accuracy by 30.6 points. These results demonstrate a practical path to scalable, self-evolving AI agents in real-world enterprise settings.
Enterprise IT support interactions are fundamentally diagnostic: effective resolution requires iterative evidence gathering from ambiguous user reports to identify an underlying root cause. While retrieval-augmented generation (RAG) provides grounding through historical cases, standard multi-turn RAG systems lack explicit diagnostic state and therefore struggle to accumulate evidence and resolve competing hypotheses across turns. We introduce DQA, a diagnostic question-answering framework that maintains persistent diagnostic state and aggregates retrieved cases at the level of root causes rather than individual documents. DQA combines conversational query rewriting, retrieval aggregation, and state-conditioned response generation to support systematic troubleshooting under enterprise latency and context constraints. We evaluate DQA on 150 anonymized enterprise IT support scenarios using a replay-based protocol. Averaged over three independent runs, DQA achieves a 78.7% success rate under a trajectory-level success criterion, compared to 41.3% for a multi-turn RAG baseline, while reducing average turns from 8.4 to 3.9.