astro-ph.IM · 2510.01299 Copy arXiv ID · Oct 1, 2025 Save Domain-Specific Agents for Cherenkov Telescope Array Control Software and Gamma-Ray Data Analysis Authors: Dmitriy Kostunin , Elisa Jones , Vladimir Sotnikov , Valery Sotnikov , Sergo Golovachev , Alexandre Strube
Organizations: Deutsches Elektronen-Synchrotron DESY, 15738 Zeuthen, Germany · JetBrains Limited, 8046 Paphos, Cyprus · JetBrains GmbH, 80639 München, Germany · Jülich Supercomputing Centre, 52425 Jülich, Germany
Abstract We present domain-adapted large language model agents designed to support Cherenkov Telescope Array operation and data analysis. The agents combine contextual knowledge with automated validation and iterative correction to produce more reliable outputs. This approach reduces manual effort, improves consistency, and helps accelerate operational and scientific workflows. The results demonstrate the potential of agentic systems as practical assistants in specialized research environments.
Explore similar work May 13, 2026 · Darius A. Faroughy, Sofia Palacios Schweitzer, Ian Pang +2 Agentic Benchmarks Large Language Model Agents
Jul 18, 2026 · Tim Fuchs, Luca Gelisio, Steffen Hauf +1 Scientific Agents Scientific Discovery
Apr 30, 2026 · Rahul Ramachandran, Nidhi Jha, Muthukumaran Ramasubramanian Artificial Intelligence Agents Scientific Discovery
May 13, 2026 · cs.LG J/K move · Enter open · S save
Darius A. Faroughy, Sofia Palacios Schweitzer, Ian Pang, Siddharth Mishra-Sharma +1
Autonomous language-model agents are increasingly evaluated on long-horizon tool-use tasks, but existing benchmarks rarely capture the complexity and nuance of real scientific work. To address this gap, we introduce Collider-Bench, a benchmark for evaluating whether LLM agents can reproduce experimental analyses from the Large Hadron Collider (LHC) using only public papers and open scientific software. Such analyses are often difficult to reproduce because the public toolchain only approximates the software used internally by the experimental collaborations, while the published papers inevitably omit implementation details needed for a faithful reconstruction. Agents must therefore rely on physical reasoning, domain knowledge, and trial-and-error to fill these gaps. Each task requires the agent to turn a published analysis into an executable simulation-and-selection pipeline and submit predicted collision event yields in specified signal regions. These predictions are evaluated with standard histogram metrics that provide continuous fidelity scores without a hand-written rubric. We also report the computational cost incurred by each agent per task. Finally, we evaluate the codebase and full session trace using an LLM judge to catch qualitative failure modes such as fabrications, hallucinations and duplications. We release an initial set of tasks drawn from LHC searches, together with a containerized sandbox and event simulation tools. We evaluate across a capability ladder of general purpose coding agents. Our results show that on average no agent reliably beats the physicist-in-the-loop solution.