FRAGATA: Semantic Retrieval of HPC Support Tickets via Hybrid RAG over 20 Years of Request Tracker History

Santiago Paramés-Estévez; Nicolás Filloy-Montesino; Jorge Fernández-Fabeiro; José Carlos Mouriño-Gallego

cs.IRApr 15, 2026

FRAGATA: Semantic Retrieval of HPC Support Tickets via Hybrid RAG over 20 Years of Request Tracker History

Authors: Santiago Paramés-Estévez, Nicolás Filloy-Montesino, Jorge Fernández-Fabeiro, José Carlos Mouriño-Gallego

Organizations: Galicia Supercomputing Center (CESGA), Santiago de Compostela · Universidade de Vigo

Abstract

The technical support team of a supercomputing centre accumulates, over the course of decades, a large volume of resolved incidents that constitute critical operational knowledge. At the Galician Supercomputing Center (CESGA) this history has been managed for over twenty years with Request Tracker (RT), whose built-in search engine has significant limitations that hinder knowledge reuse by the support staff. This paper presents Fragata, a semantic ticket search system that combines modern information retrieval techniques with the full RT history. The system can find relevant past incidents regardless of language, the presence of typos, or the specific wording of the query. The architecture is deployed on CESGA's infrastructure, supports incremental updates without service interruption, and offloads the most expensive stages to the FinisTerrae III supercomputer. Preliminary results show a substantial qualitative improvement over RT's native search.

Semantic Search

Semantic search retrieves information by meaning, matching queries and documents even when they use different words.

See all 42 papers

Before the Ticket: Grounding HPC User Support in Documentation and Resolved Tickets with an Agentic LLM Assistant
Oct 21, 2026Elham J. Barezi, Primus C Kabuo, Niveditha NerellaQuestion Answering Document QA
Rosen Center for Advanced Computing, Purdue University, West Lafayette, IN, USA
RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents
Sep 17, 2026Mingxuan Zhang, Xiaowen Wang, Anupma Sharan +4Case-Based Reasoning Retrieval-Augmented Generation
Microsoft, Redmond, WA, USA
Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking
May 31, 2026Fachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang +6Multi-Hop QA Document Chunking
School of Computer Science and Engineering, University of Electronic Science and Technology of China · Massachusetts General Hospital, Harvard University · Nota AI

Oct 21, 2026J/K move · Enter open · S save

Before the Ticket: Grounding HPC User Support in Documentation and Resolved Tickets with an Agentic LLM Assistant

Elham J. Barezi, Primus C Kabuo, Niveditha Nerella

Rosen Center for Advanced Computing, Purdue University, West Lafayette, IN, USA

Question Answering Document QA LLM Tool Use High-Performance Computing Retrieval-Augmented Generation

Open paper AskPDF arXiv

Most questions that reach an HPC help desk have already been answered in the center's user guides or in a ticket someone else filed last semester; users still file new tickets because the guides are hard to navigate and the ticket history is not theirs to search. We present an assistant that answers from both sources: an orchestrator calls two tools over the Model Context Protocol, retrieval-augmented generation (RAG) over the live documentation and a Llama-3-8B model fine-tuned with QLoRA on 11,244 query-resolution pairs mined from historical tickets, and merges their answers. Every model runs on-premises or on the university's hosted LLM service because ticket data may not leave the institution. On 400 held-out tickets from Purdue's Rosen Center for Advanced Computing, disjoint from training and free of near-duplicates, and evaluated by two judges from model families that generated none of the answers, fine-tuning on resolutions rewritten into self-service guidance is the clear winner in agreement with RCAC's resolutions (BERTScore F1 improves from 0.06 to 0.27 over the base model) and is judged most correct on average. The end-to-end system does not improve on its tools, although they are complementary: choosing the better answer per question would exceed either tool (judged correctness 3.41 versus 3.20, an oracle bound), while the merge follows the documentation tool in 78% of answers. Returning one tool's answer instead recovers the fine-tuned tool's quality (3.19 versus 2.66), but neither a learned selector nor an LLM verifier beats that tool alone.

Abstract

Semantic Search

Explore similar work

Before the Ticket: Grounding HPC User Support in Documentation and Resolved Tickets with an Agentic LLM Assistant

RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking