Long-Horizon Analog Design Bench: Benchmarking Agents on Hours-Long Analog and Mixed-Signal Circuit Design Tasks

Analog Design Bench Team

cs.AISep 27, 2026

Long-Horizon Analog Design Bench: Benchmarking Agents on Hours-Long Analog and Mixed-Signal Circuit Design Tasks

Authors: Analog Design Bench Team

Abstract

Coding agents now sustain hours-long, tool-driven loops, yet their ability to carry long-horizon analog and mixed-signal circuits to electrical specification remains unmeasured. We introduce Analog Design Bench, a long-horizon agentic benchmark of 50 transistor-level design tasks contributed by 17 chip designers. Agents work with an open-source simulator, while an isolated verifier evaluates the submitted circuit using specification-based electrical tests. We evaluate 15 agent configurations across 2,250 two-hour attempts and observe full-specification pass rates from 8.0% to 78.0%. Coding-benchmark performance correlates with analog results but leaves much of the performance spread unexplained. Our failure analysis shows that most unsuccessful submissions have no recorded legality rejection but fail electrical acceptance, identifying electrical closure as the dominant endpoint challenge. We test time, reasoning effort, agent harness, and supplied design knowledge as interventions. Longer budgets and higher reasoning effort improve performance, while general skill documents provide little benefit and sometimes reduce performance. Supplying a task-matched reference topology, an idealized form of circuit-IP retrieval, raises DeepSeek V4 Pro by 18.7 percentage points and mainly accelerates GPT-5.6 Sol.

Figures & tables

Figure 1: Agentic analog design loop and a real Analog Design Bench trajectory. Agents iteratively select topologies, edit and size circuits, simulate, diagnose, and revise toward electrical closure. Right: GPT-5.6 Sol [max] reaches its first full pass at 197.4 min. Radar plots show eight representative specifications from the task’s 15 scoring gates.

General Coding
Work	Task	Scoring	# Tasks	Original tasks	Long horizon *	Tool-use autonomy $\dagger$	Isolated verification
Terminal-Bench ( Merrill et al., 2026 )	Terminal tasks	AT	89	$\bm{\checkmark}$	$\bm{\checkmark}$	$\bm{\checkmark}$	$\bm{\times}$
DeepSWE ( Huang et al., 2026 )	Repository changes	AT	113	$\bm{\checkmark}$	$\bm{\checkmark}$	$\bm{\checkmark}$	$\bm{\checkmark}$
Digital Circuit
VerilogEval ( Liu et al., 2023b )	RTL design	AT	156	$\bm{\times}$	$\bm{\times}$	$\bm{\times}$	$\bm{\times}$
RTLLM ( Lu et al., 2024 )	RTL design	AT	50	$\bm{\checkmark}$	$\bm{\times}$	$\bm{\times}$	$\bm{\times}$

Table 1: Related work and positioning. Among the listed analog benchmarks, Analog Design Bench uniquely combines original expert-contributed tasks, long-horizon tool use, autonomous simulation, and isolated verification.

Figure 2: Benchmark construction and evaluation. Left: task construction and screening, from 100+ candidates by 20+ designers to 50 retained tasks by 17 contributors. Center: task counts across six circuit families. Right: the agent edits and simulates in one sandbox; only its declared circuit crosses into an independent verifier sandbox for legality checks and specification-based electrical grading. Hidden grading results remain inaccessible to the agent during design attempts.

	pass@1	R1 / R2 / R3	pass@3	SpecScore	Cost	Out tok	Turns
Configuration	(%)	(%)	(%)	(%)	(USD)	(k)
MiMo 2.5 Pro [thinking]	8.00	6 / 12 / 6	16.00	22.45	2.66	332	78
GLM-5.2 [max]	8.00	6 / 8 / 10	18.00	22.87	1.57	101	39
Doubao Seed 2.1 Pro [high]	14.00	16 / 16 / 10	26.00	28.99	3.34	63	56
DeepSeek V4 Flash [max]	16.00	16 / 20 / 12	24.00	30.68	0.12	191	84
GLM-5.3 Flash [max]	18.67	20 / 24 / 12	38.00	33.09	0.49	382	70

Table 2: Main experiment: 150 two-hour attempts per configuration; metrics as defined in the text; cost, output tokens, and turns are per-attempt means. Bold marks the best score and the lowest resource use in each column.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Slot	Family	Solved by	Checks	Design objective
1	Power management and references	15	30	Implement a half-bridge class-D power amplifier targeting > 95% peak efficiency and > 30 mW output power.
2	Power management and references	15	360	Implement a programmable ICC/IPTAT NMOS current mirror targeting 1.00 mA nominal output current and $\leq$ 8% output-current error.
3	Signal-chain amplifiers and active filters	12	75	Implement a source-degenerated GM-R differential amplifier targeting 1.98–2.02 V/V gain and > 60 MHz bandwidth.
4	Signal-chain amplifiers and active filters	14	25	Implement a source-degenerated GM-R differential amplifier targeting 1.98–2.02 V/V gain and > 70 MHz bandwidth.
5	Interfaces and drivers	5	243	Implement an all-NMOS half-bridge bootstrap gate driver with a designable off-chip bootstrap capacitor, targeting 3–7 ns dead time and < 3.5 mW driver power.
6	Signal-chain amplifiers and active filters	9	13	Implement a three-stage switched-capacitor ring amplifier targeting 7.2–8.8 V/V closed-loop gain and $\leq$ 50 ns settling time.

Appendix

Table A1: The 50 scored design tasks. Solved by is the number of configurations, out of 15, that pass the task at least once across three attempts. Checks gives the condition-level electrical acceptance count under the convention in Appendix C .

Functional family	Tasks	Full pass (%)	SpecScore (%)
Data conversion and sampling	9	30.4	45.5
General-purpose op amps and OTAs	9	28.9	41.1
Interfaces and drivers	3	33.3	50.4
Power management and references	11	50.7	67.9
RF, timing, and high-speed	9	48.9	62.2
Signal-chain amplifiers and active filters	9	31.6	43.8

Appendix

Table A2: Descriptive performance by functional family. The task set is outcome-conditioned, so these values should not be interpreted as intrinsic family difficulty.

Harness	Version
Codex	0.144.1
Claude Code	2.1.220
Kimi Code	0.32.0
MCode	0.2.20
OpenCode	1.18.18
Pi	0.80.7

Appendix

Table A3: Agent harness versions.

Figure A4: Mean reward for every task/configuration pair in the main cohort. Cell shade is the three-rollout mean scoring-gate reward; a dot marks at least one full-spec pass. The strip above the matrix gives the functional family of each task slot.

Skill	Package	Size
Held-out guidance : distilled by GPT-5.6 Sol [max] from the 1,260 main-experiment trajectories of tasks 1 to 30; evaluated on the held-out tasks 31 to 50
1	General handbook	1,295 words
2	Compact workflow	668 words
3	Failure-diagnosis decision tree	1,258 words
Full-suite design documents : distilled by GPT-5.6 Sol [max] from every successful and failed main-experiment trajectory then available, versions 2 and 3 revising version 1; evaluated on all 50 tasks
4	Design document, version 1	5,716 words

Appendix

Table A4: The eight skill packages of Section 5.2 , grouped by how they were built. Every package is evaluated with DeepSeek V4 Pro [max] under the six-hour budget, three attempts per task; skill 8 is also evaluated with GPT-5.6 Sol [max].

Explore similar work

AgenticSizing: A Large Language Model-based Multi-Agent Framework for Analog Circuit Sizing
Sep 22, 2026Yijia Hao, Pratibha Verma, Dongxu Guo +4Analog Design Model-Based Multi-Agent Systems
Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, UK · Department of Electrical Engineering, Indian Institute of Technology Indore, India · School of Informatics, The University of Edinburgh, UK +1
AaLLM: An End-to-End Analog Circuit Design Framework from Topology Generation to Sizing Using Large Language Models
Aug 13, 2026Mohammed Ayman Habib, Rylan Hart, Morteza FayaziAnalog Design Circuits
University of Utah, Salt Lake City, Utah, USA
EXPLORE: Exploration with Guided Search for Analog Topology Generation using Language Models
Jul 15, 2026Guanglei Zhou, Chen-Chia Chang, Yikang Shen +5Analog Design Communication Topologies
Electrical and Computer Engineering, Duke University, Durham, USA · MIT-IBM Watson AI Lab, Cambridge, USA · IBM T. J. Watson Research Center, Yorktown Heights, USA

Sep 22, 2026, cs.AIJ/K move · Enter open · S save

AgenticSizing: A Large Language Model-based Multi-Agent Framework for Analog Circuit Sizing

Yijia Hao, Pratibha Verma, Dongxu Guo, Cristian Sestito+3

Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, UK · Department of Electrical Engineering, Indian Institute of Technology Indore, India · School of Informatics, The University of Edinburgh, UK+1

Analog Design Model-Based Multi-Agent Systems Communication Topologies Lightweight

Open paper AskPDF arXiv

Analog circuit sizing remains a challenging and time-consuming task due to the large design space, strong performance trade-offs, and increasing circuit complexity in scaled technologies. Although recent large language model (LLM)-based methods show promise in improving sample efficiency and interpretability, existing approaches often lack explicit circuit-topology understanding and are mainly evaluated on relatively simple analog building blocks. This paper presents a multi-agent LLM-based framework for complex analog circuit sizing. The proposed framework first analyzes the circuit topology and decomposes the netlist into functional blocks and substructures. It also extracts lightweight design knowledge for reuse. Based on the extracted topology and knowledge, a planner coordinates multiple role-specialized sizing agents to update design variables and achieve global performance specifications. This workflow mimics the collaborative process of an expert analog design team and provides a structured, interpretable, and simulation-driven optimization procedure. The framework was validated on eight circuits, with the largest design containing up to 55 transistors and 60 sizing variables. Notably, for the LDO benchmark, the proposed method achieved a 60% success rate with an average of 83 iterations, where classical optimizers failed to find feasible solutions. Further, ablation studies demonstrate that topology understanding, design-knowledge infusion, and agent specialization provide complementary benefits. The source code is available to support reproducibility.

Abstract

Explore similar work

AgenticSizing: A Large Language Model-based Multi-Agent Framework for Analog Circuit Sizing

AaLLM: An End-to-End Analog Circuit Design Framework from Topology Generation to Sizing Using Large Language Models

EXPLORE: Exploration with Guided Search for Analog Topology Generation using Language Models