A Sim-to-Real Study of Surface-Code Decoder Benchmarking

Shay J. Manor; Leila S. Erhili; Yassine Jebbouri

quant-phSep 3, 2026

A Sim-to-Real Study of Surface-Code Decoder Benchmarking

Authors: Shay J. Manor, Leila S. Erhili, Yassine Jebbouri

Organizations: Department of Computer Science, Purdue University, West Lafayette, IN 47907 USA · Department of Computer Science, Columbia University, New York, NY 10027 USA · Data Science Institute, Columbia University, New York, NY 10027 USA

Abstract

Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under such noise transfers to hardware and improves as the noise model becomes more realistic. The Willow processor, the first to operate below the surface-code threshold, allows us to test this assumption. We rank a panel of six decoders using a four-rung ladder of noise models with increasing fidelity, evaluated against real data across three code distances, two bases, and fifteen round counts. Rank agreement with hardware appears once the noise model gives each operation type its own error rate. Calibrating the model to the device improves absolute error rates but not rank agreement. We additionally provide the first independent evaluation of NVIDIA's Ising pre-decoder on hardware, at code distances below its training receptive field and via a mapping onto the lattice on which it was trained. Under these conditions, it holds no accuracy-latency advantage: another panel decoder matches or improves on it in both per-cycle error rate and decode latency in 278 of the 280 evaluations. We release the full pipeline and the per-shot outcome of every evaluation, so future decoders and devices can be compared.

Figures & tables

Fig. 1: Ingest pipeline: a memory experiment on-device is turned into the detection-event lattice a decoder consumes and the observable flip to score against.

	Willow	Synthetic
MWPM	420 $\times$ 50k	420 $\times$ 20k
MWPM-RL	420 $\times$ 50k	—
BP+LSD	168 $\times$ 50k	112 $\times$ 20k
Ising-fast	140 $\times$ 50k	84 $\times$ 20k
Ising-accurate	140 $\times$ 50k	84 $\times$ 20k
BeliefMatching	168 $\times$ 10k	112 $\times$ 5k

TABLE I: Evaluations per decoder. Each cell is configurations

\times

shots per configuration, on Willow and on each of the four synthetic rungs.

Fig. 2: Kendall’s $\tau$ between each synthetic noise model’s decoder ranking and the real-Willow ranking, by code distance, over the rounds 2–30 window. $\tau=1$ denotes an identical ordering. Filled markers indicate agreement significant at $p<0.05$ against a random ordering; open markers indicate agreement that does not reach significance, both under uniform depolarizing noise.

Fig. 3: Decoder rank under each noise model and on real Willow, by code distance. Rank 1 is the lowest per-cycle logical error rate.

Fig. 4: Mean per-cycle logical error rate by decoder, noise model, and code distance, over the rounds 2–30 window.

Pair	$d=3$	$d=5$	$d=7$
MWPM $\leftrightarrow$ BP+LSD	0.00	0.20	0.38
BP+OSD $\leftrightarrow$ MWPM	0.00	0.00	0.25
Ising-fast $\leftrightarrow$ Ising-acc.	0.03	0.00	0.00
Any pair	0.03	0.20	0.62

TABLE II: Fraction of the 40 bootstrap refits in which a decoder pair swaps rank, listed by pair with their union in the last row.

Fig. 5: Per-cycle logical error rate against decode latency on Willow, by code distance. The line marks the Pareto frontier.

Decoder	d=3	d=5	d=7	growth /+2d	extrap. d9
Tesseract	1.253	49.842	305.501	15.6x	4.8 s
BeliefMatch	1.225	9.748	26.052	4.6x	120 ms
MWPM	0.001	0.007	0.017	3.6x	0.061 ms
MWPM-RL	0.001	0.007	0.017	3.7x	0.064 ms
BP+OSD	1.181	31.478	182.085	12.4x	2.3 s
BP+LSD	0.336	3.335	11.744	5.9x	69 ms

TABLE III: Median decode latency (ms/shot) on real Willow per decoder

\times

distance, with fitted growth per +2 code distance and the extrapolated d=9 cost.

Fig. 6: Per-cycle logical error rate against round count on Willow, by code distance.

Fig. 7: Independent code patches per distance in the Willow corpus (left) and the largest edge probability in the syndrome-estimated DEM at each distance (right). An edge probability of $0.5$ carries no information about whether that error occurred.

Distance	real X/Z	fit X/Z	syndrome X/Z
d=3	1.107	1.055	0.979
d=5	1.124	0.981	0.934
d=7	1.162	0.947	0.981

TABLE IV: X

>

Z basis asymmetry (mean per-cycle LER ratio, MWPM). Real Willow is X

>

Z in 24/24 cells (mean +13.6%, ratio 1.04-1.39); the fit rung reproduces it in only 14/21 cells (median ratio 1.02) and syndrome in 11/21 – neither Pauli rung reproduces the device asymmetry.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 8: Per-cycle MWPM logical error rate versus code distance on Willow, both bases.

Memory basis	Rotation	Ising-fast	Ising-acc	MWPM ref
X	XV	0.0148	0.0145	0.0049
X	XH	0.0175	0.0184
X	ZV	0.0390	0.0330
X	ZH	0.0097	0.0088
Z	XV	0.0119	0.0116	0.0038
Z	XH	0.0132	0.0146

Appendix

TABLE V: Ising per-cycle logical error rate at

d=7

and

r=70

under the four CSS lattice orientations, both model sizes, against the MWPM reference.

Noise model	$d=3$	$d=5$	$d=7$
uniform	0.619 (p=0.069)	0.714 (p=0.030)	0.619 (p=0.069)
SI1000	0.905 (p=0.003)	0.905 (p=0.003)	1.000 (p $<$ 0.001)
25-param fit	0.905 (p=0.003)	0.905 (p=0.003)	0.810 (p=0.011)
syndrome DEM	1.000 (p $<$ 0.001)	0.905 (p=0.003)	0.905 (p=0.003)

Appendix

TABLE VI: Kendall’s

\tau

between each synthetic rung’s decoder ranking and the Willow ranking, per code distance, over the rounds 2–30 window.

p

-values test against a random ordering.

Rung	Decoder	d=3	d=5	d=7
uniform	Tesseract	0.00590	0.00383	0.00236
	BeliefMatch	0.00609	0.00481	0.00365
	MWPM	0.00772	0.00831	0.00780
	BP+OSD	0.01418	0.03380	0.13394
	BP+LSD	0.03697	0.05456	0.14913
	Ising-fast	0.01962	0.03518	0.05277

Appendix

TABLE VII: Full fidelity-ladder mean per-cycle LER matrix (rounds 2-30 window), decoder

\times

code distance, one block per rung.

Quantum Error Correction

Quantum error correction detects and corrects qubit noise so quantum information and computations remain reliable.

See all 33 papers

Latency-Constrained Hardware-Aware Quantum Error Correction Co-Design with Adaptive Confidence-Gated Neural Decoding for the Rotated Surface Code
Jul 7, 2026Sumit ChongderNeural Decoding Quantum Error Correction
Department of Physics, Quantum Information and Computation, Indian Institute of Technology Jodhpur, Jodhpur, Rajasthan 342037, India
Rethink the Role of Neural Decoders in Quantum Error Correction
May 12, 2026Ge Yan, Shanchuan Li, Yuxuan DuNeural Decoding Quantum Error Correction
College of Computing and Data Science, Nanyang Technological University, Singapore 639798, Singapore · Department of Electrical Engineering and Computer Science, Tokyo University of Agriculture & Technology, Koganei, Tokyo, 184-8588, Japan · School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore 639798, Singapore
QAdapt: A Noise-Adaptive Neural Pre-Decoding Framework for Quantum Error Correction
Jul 30, 2026Ran Miao, Rui Luo, Xiaohan Shan +1Distribution Shift Neural Decoding
Beijing Zhongke Qhub Technology Co., Ltd. (Qhub), Beijing, China · Center for Quantum Information, Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Jul 7, 2026J/K move · Enter open · S save

Latency-Constrained Hardware-Aware Quantum Error Correction Co-Design with Adaptive Confidence-Gated Neural Decoding for the Rotated Surface Code

Sumit Chongder

Department of Physics, Quantum Information and Computation, Indian Institute of Technology Jodhpur, Jodhpur, Rajasthan 342037, India

Neural Decoding Quantum Error Correction Quantum Error Correction Decoding Error-Correcting Codes

Open paper AskPDF arXiv

Real-time decoding is a major bottleneck in scaling quantum error correction (QEC) from noisy intermediate-scale quantum (NISQ) devices to fault-tolerant quantum computing. We present an adaptive confidence-gated decoding framework for the rotated surface code that treats decoding as a two-stage inference problem. A lightweight feed-forward neural network performs fast-path decoding for the majority of syndrome measurements, while only low-confidence predictions are escalated to a minimum-weight perfect matching (MWPM) refinement stage. We benchmark the framework on rotated surface codes with distances

d \in \{3,5,7,9,11\}

under circuit-level depolarising noise using the Stim stabiliser simulator. The evaluation characterises logical accuracy, confidence-controlled accuracy-latency trade-offs, decoding throughput, per-shot latency, and decoding-graph resource scaling. Routing only 3.3%-6.2% of syndromes to the refinement stage improves logical accuracy from 99.21% for the neural-only baseline to 99.81% at a confidence threshold of 0.95 while incurring only a bounded increase in average decoding cost. Neural-decoder throughput saturates near

4.6 \times 10^{5}

samples s

^{-1}

at batch size 512 on commodity CPU hardware, indicating that the neural fast path is not the dominant throughput bottleneck beyond code distance

d=7

. We release the complete benchmarking pipeline, trained models, raw benchmark data, and source code, and explicitly distinguish the experimentally validated contributions from the broader hardware-aware QEC co-design roadmap, including hardware-constrained code discovery, GPU-accelerated inference, and multi-noise optimisation, which remain directions for future work.

Abstract

Quantum Error Correction

Explore similar work

Latency-Constrained Hardware-Aware Quantum Error Correction Co-Design with Adaptive Confidence-Gated Neural Decoding for the Rotated Surface Code

Rethink the Role of Neural Decoders in Quantum Error Correction

QAdapt: A Noise-Adaptive Neural Pre-Decoding Framework for Quantum Error Correction