Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits

John Hu; Andrew Ash

cs.HCAug 3, 2026

Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits

Authors: John Hu, Andrew Ash

Organizations: School of Electrical and Computer Engineering Oklahoma State University Stillwater, OK 74078, USA

Abstract

This research paper describes an exploratory study on the effectiveness of Chat Debugging: troubleshooting malfunctioning analog circuits on breadboards and printed circuit boards (PCB) by undergraduates through conversations with public-domain large language models (LLMs). Through thematic analysis of students' voluntarily shared chat logs when debugging pre-determined buggy circuits under exam and time pressure, we discovered multimodal usage patterns by students and considerable domain knowledge and sensible debugging suggestions offered by off-the-shelf LLMs. Meanwhile, we also identified major gaps in LLM technologies and students' skills during human-AI collaborative debugging, such as LLMs' limitations in 2D/3D image-based reasoning, unjustified tone of confidence, and students' deficits in fundamental concepts and critical thinking.

Explore similar work

Dataset Construction for Training LLM to Learn Analog Circuit Knowledge
Aug 14, 2025Zihao Chen, Ji Zhuang, Jinyi Shen +11Analog Circuits Textual Dataset
Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving
Dec 10, 2025Liangliang Chen, Weiyu Sun, Huiru Xie +2Circuits Large Language
SPARC: A Multi-Agent System for Electrical Circuit Question Answering
Jun 5, 2026Mushtari Sadia, Zhenning Yang, Umme Habiba Lamia +3Circuits Multi-Agent Systems

Aug 14, 2025 · cs.ARJ/K move · Enter open · S save

Dataset Construction for Training LLM to Learn Analog Circuit Knowledge

Zihao Chen, Ji Zhuang, Jinyi Shen, Xiaoyue Ke+10

Fudan University, Shanghai, China

This paper constructs a textual dataset for training large language models (LLMs) to learn analog circuit knowledge and customizes LLM training techniques. For dataset construction, high-quality textbooks are collected and decomposed into fine-grained learning nodes, which are then used to construct structured question-thinking-solution-answer (QTSA) quadruples using a multi-agent framework to capture both final answers and thought processes. The resulting dataset consists of 7.26M tokens of unlabeled data for continual pre-training (CPT) and 112.65M tokens of labeled data for supervised fine-tuning (SFT). We customize the training techniques including initial model selection, training paradigms, regularization techniques, and practical implementation references. Instruct models are identified as suitable training initialization points, an SFT-centric training paradigm is established (finding that CPT provides marginal benefits compared with SFT due to imbalanced data distribution), and SFT with KL divergence regularization can achieve a 2.71 percentage-point improvement over SFT alone. A practical training implementation method is provided for resource-constrained scenarios. Experiments demonstrate that the dataset and training techniques enhance LLMs' analog circuit knowledge. The trained 32B instruct model achieves 84.59% accuracy on the AMSBench-TQA benchmark, showing a 15.67 percentage-point improvement over the initial model. The trained model also shows capability in the operational amplifier design task based on the Atelier framework.

Analog Circuits Textual Dataset Novel Modular Learning Framework Construction

Open paper Ask about it PDF

Abstract

Explore similar work

Dataset Construction for Training LLM to Learn Analog Circuit Knowledge

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

SPARC: A Multi-Agent System for Electrical Circuit Question Answering