Existing analog integrated circuit design benchmarks make two questions hard to answer: whether a model has learned transferable circuit skills rather than recalled familiar examples, and whether its output works under defined process and test conditions. We introduce a benchmark built from human-designed, open-source circuits associated with Tiny Tapeout manufacturing shuttles. The collection contains 273 topologically distinct top-level designs. Every source is retrieved at the revision recorded for its shuttle submission and processed in a fixed containerized environment. The pipeline exports each eligible schematic image and its SPICE netlist from the same source file, giving transcription an exact structural reference. Commit dates support model-specific training-cutoff analysis, while author testbenches provide the simulation context for sizing. The benchmark evaluates schematic-to-netlist transcription and device sizing. Across seven models on a fixed set of 66 transcription tasks, the strongest model reaches 56.1% exact graph isomorphism, and six of seven models drop sharply from the small to the medium tier. For one frontier model, removing author-chosen labels reduces exact matches while preserving aggregate structural F1, suggesting that labels can aid connectivity tracing. On the 17 sizing tasks, the leading model converges on all 17 proposals and reaches 91.2 out of 100 against the human reference, while the two newest Claude models refuse 4 and 11 of the same prompts they transcribe without objection; a proposal without sizes scores zero. The two tasks produce different model rankings, exposing distinct visual and design capabilities and, in one family, a policy rather than capability limit.
Figures & tables
Xschem
Figure 1: Benchmark overview. Versioned Tiny Tapeout sources are converted into matched schematic images and netlists, with author testbenches retained for simulation. Transcription is scored by device–net graph isomorphism, the comparison at the core of an LVS check; sizing is scored by simulation; and potential exposure is analyzed with commit-aware controls.
Benchmark
Scale
Provenance
Sized
TB
PDK
Audit
AnalogGenie [ 4 ]
3,350 topo.
textbooks + papers
no
no
no
no
CktGNN / OCB [ 5 ]
10k op-amps
synthetic
part
part
no
no
AICircuit [ 9 ]
9 topo. / 361k pts
authored
yes
yes
part
no
OSIRIS [ 10 ]
5 templ. / 87.1k layouts
authored
yes
yes
yes
no
MAGICAL-CIRCUITS [ 11 ]
26 blocks
UT Austin silicon
yes
part
no
no
AnalogCoder [ 6 ]
24 tasks
authored
part
yes
no
no
Table 1: Analog benchmark landscape. Sized = devices carry numeric sizes. TB = author testbenches are included. PDK = an open process design kit simulation flow is included. “part” marks partial coverage. Audit = the benchmark ships a contamination audit. Scale follows the unit reported by each source and is therefore not directly comparable.
From index to task sets
n
Index projects (27 shuttles)
4,572
Entries with analog pins
237
Repositories contributing circuits
125
Schematics fetched at pinned commits
1,448
Circuits
933
Top-level designs
425
Table 2: Dataset spine: from the shuttle index to the fixed task sets. Appendix B gives the full triage, licensing, composition, and redundancy inventory.
Figure 2: Both tasks, end to end. Top: transcription. The input schematic image (left) and claude-opus-5’s verbatim output netlist (right) for a five-transistor OTA. Its net names differ from the drawing, but its graph is isomorphic to the reference. Bottom: sizing for a fully differential amplifier. The model receives the topology with sizes removed, then its proposal and the human reference run in the same author testbench. Aggregate results appear in Table 3 .
Figure 3: Main results for both tasks. (a) Task 1: each model’s 66 structural F1 scores sorted in descending order. The width of the plateau at 1.0 equals the exact-match count, and the tail shape is the failure mode: Opus falls steeply once it fails, while Sol declines gradually. The encoding uses no banding thresholds. (b) Task 2: the complete score matrix, seven models by 17 sizing tasks, one cell per proposal. Each cell prints its score, with blue depth as a redundant encoding; a gray cross marks a nonconvergent sizing, and a red R marks a proposal that provided no sizes, which scores zero. The left group is scored by how close the proposal’s simulated output comes to the human design’s, hardest answered task first; in the right group the author analyses expose no usable scalar, so a convergent sizing scores one. The two .00 cells, on a DAC task, are convergent sizings whose output collapsed. (c) Transcription accuracy versus sizing score: model rankings differ across the two capabilities. The sizing axis is truncated to keep the seven points legible. The difficulty-tier breakdown of Task 1 is in Appendix A .
Model
Cutoff
Iso
Struct
W/L
Post n
Post / pre iso
Sizing
No ans.
claude-fable-5
2026-01
56.1%
0.853
0.921
13
30.8% / 62.3%
31.8
11
claude-opus-5
2026-05
51.5%
0.711
0.929
0
—
74.3
4
gpt-5.6-sol
2026-02
42.4%
0.831
0.942
13
30.8% / 45.3%
78.2
0
gemini-3.1-pro-preview
2025-01
39.4%
0.767
0.977
36
38.9% / 40.0%
91.2
0
claude-sonnet-5
2026-01
19.7%
0.604
1.000
13
7.7% / 22.6%
85.4
0
gpt-5
2024-09
13.6%
0.512
0.988
38
15.8% / 10.7%
79.0
0
Table 3: Model results on the fixed transcription set. Cutoffs are vendor-reported; post n counts tasks whose pinned revision postdates the cutoff, ruling out that exact revision from the model’s pretraining window. Post/pre columns give exact isomorphism on each side. Sizing is the Task 2 score on a 100-point scale, with a proposal that provides no sizes scoring zero; No ans. counts those proposals (refusals, one unparseable answer) out of 17.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Exact isomorphism by difficulty tier ( n=9,38,19 ): five models drop from small to medium, while Haiku is already at floor; Opus and Gemini improve from trivial to small. Markers with identical scores are offset slightly within each tier for visibility; only their vertical position encodes accuracy.
Harvest funnel
n
Index projects (27 shuttles)
4,572
Entries with analog pins
237
Unique repositories
195
ship xschem [ 15 ] schematics
133
ship SPICE netlists only
12
layout data only (unusable)
48
Appendix
Table 4: The full dataset inventory. Left: the harvest funnel with triage and licensing detail. Right: composition by role, transistor-count tier, era, and device–net graph hash.
Figure 5: The simulation browser on one verified pair, a differential amplifier from the corpus: published specifications, circuit and testbench schematics, and the transient waveforms produced by a run inside the pinned container (exit 0 in 26.9 seconds; the two complementary output sines around the input common mode).
Electrical and Computer Engineering, Duke University, Durham, USA · MIT-IBM Watson AI Lab, Cambridge, USA · IBM T. J. Watson Research Center, Yorktown Heights, USA