cs.SENov 2, 2026

DistDebug-Bench: Can LLM Agents Diagnose Bugs in Distributed Systems?

Authors: Yuxuan Jiang, Chenglin Liu, John Rose, Mason Miller, Ruthesh Thavamani, Juntao Wu, Yinwei Huang, Jiahao Xu, 

Organizations: University of Michigan

Abstract

LLM agents show strong potential in software engineering tasks such as competitive programming and code synthesis. Existing benchmarks, however, largely focus on small-scale settings such as self-contained Python modules, leaving distributed systems, an essential class of production software, under-examined. We introduce DistDebug-Bench, a benchmark for evaluating agents' ability to diagnose complex-system failures at the code level. It curates 120 high-quality real-world bugs from 11 widely used distributed systems, spanning 5 programming languages and 10 failure patterns. Through an extensive evaluation of state-of-the-art LLMs and agents, we identify substantial gaps between current agents and real-world debugging tasks. A recurring gap is tunnel vision due to a lack of explicit reasoning structure and nondeterminism from inconsistent tool-use scope. The benchmark and evaluator are available at https://github.com/OrderLab/distdebug-bench, with agent traces and a leaderboard at https://distdebug.com.

Figures & tables

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DDBench: A Benchmark for Agentic Debugging on Distributed Systems

    Aug 14, 2026Yibo Yan, Huijuan Wang, Junzhou He +4AI Coding AgentsLLM Agent Evaluation

  2. HWE-Bench: Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks

    Apr 16, 2026Fan Cui, Hongyuan Hou, Zizhang Luo +2Automated Program RepairElectronic Design Automation

  3. ProgramBench: Can Language Models Rebuild Programs From Scratch?

    May 5, 2026John Yang, Kilian Lieret, Jeffrey Ma +9Software Engineering AgentsSoftware Engineering Benchmarks