cs.SENov 11, 2026

AskALCF: A Self-Improving Agentic System for Intelligent HPC User Support

Authors: Jingyan Jiang, Akhilesh Bondapalli, Krishna Rauniyar, Murat Keceli, Yasaman Ghadar, Haritha Siddabathuni Som, Venkatram Vishwanath, Michael E. Papka, 

Organizations: Argonne National Laboratory Lemont, IL, USA · Clemson University Clemson, SC, USA · University of Louisiana at Lafayette Lafayette, LA, USA

Abstract

AI-based question-answering systems are increasingly used for user support, including at high-performance computing (HPC) facilities. However, existing systems lack a systematic, deployment-oriented approach for evaluating final-answer quality and attributing failures to specific components. They also remain largely static after deployment, apart from periodic knowledge-base updates. We present AskALCF, an agentic log-analysis and improvement pipeline for HPC user support. AskALCF evaluates production interactions at a fine-grained level, identifies whether failures arise from missing knowledge, retrieval, prompting, model behavior, or tool execution, and routes each failure to an actionable repair. Its diagnostic core combines answer-quality assessment with a two-scope sufficiency test that distinguishes information missing from the documentation corpus from information present but not retrieved. We further introduce a controlled harness-training process that uses expert references, null edits, and explicit ship, revert, or decline decisions to improve the diagnostic pipeline reliably. Across 2,174 support turns, AskALCF reduced unattributable failures from 41.4% to 14.2%, while harness training improved expert agreement from 108/150 to 123/150 turns. These results demonstrate a governed form of self-evolution in which production evidence drives targeted system improvements. By turning production interactions into fine-grained diagnoses and actionable repairs, AskALCF provides a systematic foundation for continuously improving deployed AI support systems beyond knowledge-base updates alone.

Figures & tables

Explore similar work

CardsList
  1. Before the Ticket: Grounding HPC User Support in Documentation and Resolved Tickets with an Agentic LLM Assistant

    Oct 21, 2026Elham J. Barezi, Primus C Kabuo, Niveditha NerellaQuestion AnsweringDocument QA

  2. DQA: Diagnostic Question Answering for IT Support

    Apr 7, 2026Vishaal Kapoor, Mariam Dundua, Sarthak Ahuja +6Question AnsweringCustomer Support Automation