LESS: Lightweight Evolutionary Supernet Search in Minutes

Aviral Gandhi; Jinglue Xu; Jialong Li; Hitoshi Iba

cs.NEOct 1, 2026

LESS: Lightweight Evolutionary Supernet Search in Minutes

Authors: Aviral Gandhi, Jinglue Xu, Jialong Li, Hitoshi Iba

Organizations: The University of Tokyo · Sakana AI · Waseda University

Abstract

Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves $93.189\pm0.467\%$ CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately $1/24$ of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by $0.577$ percentage points while changing best-visited accuracy by only $0.054$ points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with $69.615\pm1.139\%$ and $43.720\pm1.697\%$ accuracy. Applied without tuning to the larger DARTS space, LESS achieves $96.95\pm0.14\%$ on CIFAR-10 and $82.43\pm0.80\%$ on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.

Figures & tables

Figure 1: LESS search trajectory. A short fair hard-path warm-up initializes the supernet. At each search epoch, CMA-ES generates genotype-distinct proposals, which are evaluated alongside the incumbent from a shared parent state. Proposal losses update CMA-ES, while the epoch winner determines the persistent pre-BatchNorm supernet state. The final architecture is selected by aggregating rankings across the complete search trajectory using Borda scoring.

Figure 2: Candidate-conditioned fair evaluation. Each candidate anchors a group of complementary hard paths that jointly covers every eligible operation once per edge. Their gradients are averaged to produce one balanced supernet update, repeated across the calibration steps. Only the candidate architecture is scored after BatchNorm refresh. The refreshed scoring copy is discarded, while the saved pre-refresh state is inherited only if the candidate wins the epoch.

Method	Search class	CIFAR-10	CIFAR-100	ImageNet16-120	Time (s)	Runs
REA ( Real et al., 2019 ; Mellor et al., 2021 )	evolution	$93.92\pm 0.30$	$71.84\pm 0.99$	$45.54\pm 1.03$	–	500
Random search ( Mellor et al., 2021 )	training	$93.70\pm 0.36$	$71.04\pm 1.07$	$44.57\pm 1.25$	–	500
BOHB ( Falkner et al., 2018 ; Mellor et al., 2021 )	multi-fidelity	$93.61\pm 0.52$	$70.85\pm 1.28$	$44.42\pm 1.49$	–	500
RSPS ( Li and Talwalkar, 2019 ; Mellor et al., 2021 )	weight sharing	$87.66\pm 1.69$	$58.33\pm 4.34$	$31.14\pm 3.88$	7,587	3
ENAS ( Pham et al., 2018 ; Mellor et al., 2021 )	weight sharing	$54.30$	$15.61$	$16.32$	13,315	3
SETN ( Mellor et al., 2021 )	weight sharing	$86.19\pm 4.63$	$56.87\pm 7.77$	$31.90\pm 4.07$	31,010	3

Table 1: NAS-Bench-201 accuracy–cost comparison. All accuracy values are test percentages. Times are source-reported and use different hardware and implementations. For LESS, the reported times on CIFAR-10, CIFAR-100, and ImageNet16-120 are 409.1, 408.9, and 453.1 seconds, respectively.

Figure 3: Calibration and warm-up sensitivity. Mean selected validation accuracy over eight matched seeds; error bars show one standard deviation. The frozen

K=6

and

W=25

settings are highlighted.

Figure 4: CIFAR-100 calibration depth and accuracy–compute trade-off. Left: mean CIFAR-100 validation accuracy over eight seeds, with one-standard-deviation error bars. Right: LESS E30–E120 prefixes and source-reported efficient NAS baselines; times use different hardware and implementations.

Method	C10 Error (%)	C100 Error (%)	Params (M)	GPU-days
DARTS-2 ( Liu et al., 2019 )	$2.76\pm 0.09$	$17.54$	3.3	4.00
SNAS ( Xie et al., 2019 )	$2.85\pm 0.02$	–	2.8	1.50
GDAS (FRC) ( Dong and Yang, 2019 )	$2.82$	$18.13$	2.5	0.17
P-DARTS ( Chen et al., 2019 )	$2.50$	$17.20$	3.4	0.30
PC-DARTS ( Xu et al., 2020 )	$2.57\pm 0.07$	–	3.6	0.10
TE-NAS ( Chen et al., 2021 )	$2.63\pm 0.064$	–	3.8	0.05

Table 2: DARTS-space comparison. Test errors are percentages. Search costs follow the respective sources and are not hardware-normalized. All comparison methods use configurations developed for the DARTS space, whereas LESS transfers its frozen NAS-Bench-201 search hyperparameters without retuning. LESS reports the mean and standard deviation across three independently searched architectures per dataset. A dash denotes an unavailable canonical CIFAR-100 result.

Appendix figures & tables50 assets

Supplementary material from the paper’s appendix.

Appendix

Cohort	Seeds
Initial 16	1333636, 2621901, 3011368, 3046636, 3274845, 3458191, 3750948, 4096113, 4564271, 4611954, 5197346, 7848893, 7990734, 8224849, 8629488, 8682075
Prospective 8	3297086, 4007814, 4450079, 5392407, 5805960, 6963249, 8147065, 8167106
Component / transfer subset	4611954, 4564271, 2621901, 1333636, 3046636, 3274845, 4096113, 7848893
DARTS architecture seeds	184271, 392113, 618407
DARTS retraining seed	123

Appendix

Table 3: Architecture and retraining seed inventory.

Setting	NB201	DARTS
Initial channels / cells	16 / 8	16 / 8
Latent logits	30	224
Warm-up epochs / batch	25 / 256	25 / 256
Warm-up LR / momentum	0.1 cosine / 0.9	0.1 cosine / 0.9
Warm-up weight decay / clipping	0.0003 / 5	0.0003 / 5
Hard paths per warm-up update	5	7

Appendix

Table 4: Frozen configuration. DARTS-specific topology is detailed in Appendix N .

Dataset	Input	Classes	Search train	Search valid
CIFAR-10	32 by 32	10	25,000	25,000
CIFAR-100	32 by 32	100	25,000	25,000
ImageNet16-120	16 by 16	120	151,700	3,000

Appendix

Table 5: Search data split sizes; official test data do not score search candidates.

Dataset	Mean (R,G,B)	Standard deviation (R,G,B)
CIFAR-10	0.49139968, 0.48215827, 0.44653124	0.24703233, 0.24348505, 0.26158768
CIFAR-100	0.50707516, 0.48654887, 0.44091784	0.26733429, 0.25643846, 0.27615047

Appendix

Table 6: CIFAR channel normalization used by the search and DARTS retraining.

Quantity	NB201	DARTS
Candidate evaluations	780	780
Calibration optimizer steps	4680	4680
Calibration forward/backward passes	23400	32760
Focal BN-refresh forwards	6240	6240
Validation forwards	6240	6240
Validation example-evaluations	798,720	798,720

Appendix

Table 7: Default search workload, excluding warm-up and retraining.

Cohort	Arm	n	Validation	Validation 95% CI	Test
Initial 16	K6	16	$89.848\pm 0.565$	$[89.547,90.149]$	$93.167\pm 0.497$
Initial 16	K0	16	$89.319\pm 0.612$	$[88.993,89.645]$	$92.694\pm 0.471$
Prospective 8	K6	8	$90.235\pm 0.609$	$[89.726,90.745]$	$93.233\pm 0.429$
Prospective 8	K0	8	$89.564\pm 0.524$	$[89.126,90.003]$	$92.692\pm 0.366$
Pooled 24	K6	24	$89.977\pm 0.597$	$[89.725,90.229]$	$93.189\pm 0.467$
Pooled 24	K0	24	$89.401\pm 0.584$	$[89.154,89.647]$	$92.693\pm 0.431$

Appendix

Table 8: Principal CIFAR-10 outcomes. Test accuracy is report-only;

n

counts search seeds.

Comparison	Mean	Median	t 95% CI	Bootstrap 95% CI	W/T/L	Sign p
Initial 16: K6-K0	0.529	0.741	$[0.086,0.973]$	$[0.135,0.924]$	11/0/5	0.2101
Prospective 8: K6-K0	0.671	0.480	$[0.197,1.145]$	$[0.346,1.071]$	8/0/0	0.0078
Pooled 24: K6-K0	0.577	0.545	$[0.262,0.891]$	$[0.287,0.869]$	19/0/5	0.0066
Subset: K6-K0	0.716	0.801	$[0.293,1.138]$	$[0.379,1.038]$	7/0/1	0.0703
K6-K0-E180	0.643	0.840	$[-0.190,1.477]$	$[-0.064,1.218]$	6/0/2	0.2891
CMA-uniform	1.030	0.893	$[0.466,1.595]$	$[0.646,1.510]$	8/0/0	0.0078

Appendix

Table 9: Paired validation differences, in percentage points. All component contrasts use eight seeds.

Cohort	Seed	K0 valid	K6 valid	Difference	K0 test	K6 test
Initial 16	1333636	89.355	89.880	0.525	92.890	93.375
Initial 16	2621901	88.734	89.624	0.890	92.245	93.620
Initial 16	3011368	88.798	90.085	1.287	92.155	93.630
Initial 16	3046636	89.282	90.094	0.812	92.640	93.150
Initial 16	3274845	88.432	90.024	1.592	92.235	93.425
Initial 16	3458191	89.267	89.969	0.703	92.617	93.330

Appendix

Table 10: All 24 principal CIFAR-10 pairs. Test columns are descriptive.

Cohort	Arm	Best visited	Regret	Unique	Spearman	Kendall
Initial 16	K6	90.661	0.812	127.2	0.508	0.358
Initial 16	K0	90.629	1.311	144.5	0.474	0.333
Prospective 8	K6	90.870	0.635	120.1	0.469	0.336
Prospective 8	K0	90.769	1.204	152.1	0.440	0.306
Pooled 24	K6	90.730	0.753	124.9	0.495	0.350
Pooled 24	K0	90.676	1.275	147.0	0.463	0.324

Appendix

Table 11: Pool and ranking diagnostics for the principal cohorts; entries are seed means.

Cohort	Arm	None	Skip	Conv 1	Conv 3	Pool
Initial 16	K0	0.438	1.062	1.688	2.375	0.438
Initial 16	K6	0.125	0.562	2.375	2.875	0.062
Prospective 8	K0	0.250	1.375	1.750	2.125	0.500
Prospective 8	K6	0.125	1.000	2.250	2.625	0.000
Pooled 24	K0	0.375	1.167	1.708	2.292	0.458
Pooled 24	K6	0.125	0.708	2.333	2.792	0.042

Appendix

Table 12: Mean selected-cell operation counts (six edges per cell).

Arm	Validation	Test	Best visited	Regret	Unique	Spearman
K6 reference	$90.041\pm 0.482$	$93.388\pm 0.367$	90.663	0.623	132.9	0.525
K0 E180	$89.397\pm 0.931$	$92.762\pm 0.613$	90.759	1.362	216.9	0.381
Uniform K6	$89.010\pm 0.288$	$92.078\pm 0.149$	91.389	2.379	704.5	0.721
No prior K6	$89.493\pm 0.837$	$92.486\pm 0.660$	91.103	1.610	187.2	0.373

Appendix

Table 13: Matched component controls on

\mathcal{S}_{8}

. Test is report-only.

Seed	CMA K6	K0 E180	Uniform K6	Prior off K6
4611954	90.132	88.662	89.202	88.712
4564271	89.900	90.094	89.005	89.300
2621901	89.624	88.734	89.119	88.873
1333636	89.880	89.355	89.328	90.694
3046636	90.094	88.670	88.451	89.060
3274845	90.024	88.432	89.132	90.904

Appendix

Table 14: Per-seed selected validation for matched component controls.

Selector	n	Validation	Median	t 95% CI
Borda	16	$89.848\pm 0.565$	89.935	$[89.547,90.149]$
Final-epoch input incumbent	16	$89.881\pm 0.646$	89.902	$[89.537,90.225]$
Last-epoch winner	16	$89.955\pm 0.649$	89.878	$[89.609,90.301]$

Appendix

Table 15: Recorded offline selector replay on the initial 16 K6 trajectories.

Arm	Validation	Test	Best visited	Regret	Unique	Time (s)
Conditioned	$90.041\pm 0.482$	$93.388\pm 0.367$	$90.663\pm 0.589$	$0.623\pm 0.358$	$132.9\pm 45.4$	$410.2\pm 2.6$
Independent	$89.031\pm 2.131$	$92.451\pm 1.889$	$90.611\pm 0.462$	$1.580\pm 1.909$	$115.2\pm 41.6$	$407.5\pm 3.7$

Appendix

Table 16: Candidate conditioning control on eight paired seeds; means with sample SD.

Seed	C valid	I valid	C-I	C best	I best	C regret	I regret
4611954	90.132	89.588	0.544	91.533	90.164	1.401	0.576
4564271	89.900	89.900	0.000	90.712	90.712	0.812	0.812
2621901	89.624	89.520	0.104	90.106	90.474	0.482	0.954
1333636	89.880	90.024	-0.144	90.386	90.712	0.506	0.688
3046636	90.094	87.831	2.263	90.302	90.806	0.208	2.975
3274845	90.024	90.024	0.000	90.466	90.386	0.442	0.362

Appendix

Table 17: Candidate-conditioned (C) versus independent (I) fair calibration, per seed.

Metric	Mean	Median	t 95% CI	Bootstrap 95% CI
selected validation	1.009	0.057	$[-0.581,2.600]$	$[0.029,2.349]$
best visited validation	0.052	0.027	$[-0.435,0.538]$	$[-0.259,0.477]$
selection regret	-0.957	-0.091	$[-2.634,0.719]$	$[-2.379,0.144]$
complete method seconds	2.636	3.563	$[-0.046,5.319]$	$[0.533,4.641]$

Appendix

Table 18: Conditioned-minus-independent effects. All entries are paired; runtime is seconds.

K	Validation	Test	Best	Regret	Unique	Spearman	Time (s)
0	$89.325\pm 0.631$	$92.693\pm 0.438$	90.679	1.354	149.1	0.447	$205.4\pm 5.8$
1	$90.131\pm 0.769$	$93.344\pm 0.611$	90.961	0.830	138.4	0.424	$243.1\pm 3.6$
2	$89.726\pm 0.761$	$92.951\pm 0.535$	90.648	0.922	144.6	0.393	$278.4\pm 3.4$
3	$89.510\pm 0.668$	$92.972\pm 0.505$	90.488	0.978	128.8	0.481	$310.5\pm 1.7$
4	$89.378\pm 0.367$	$92.722\pm 0.387$	90.543	1.165	126.9	0.467	$344.0\pm 2.6$
5	$89.909\pm 0.598$	$93.216\pm 0.475$	90.520	0.611	110.1	0.506	$375.8\pm 2.4$

Appendix

Table 19: Dense K curve on eight CIFAR-10 seeds. Test is report-only; runtime retains all measurements.

Seed	K0	K1	K2	K3	K4	K5	K6	K7	K8
4611954	89.252	89.461	89.461	89.252	89.461	89.626	90.132	89.626	89.461
4564271	90.094	90.268	90.268	90.046	89.900	89.900	89.900	89.900	89.774
2621901	88.734	89.624	88.441	89.520	89.626	89.520	89.624	89.488	89.624
1333636	89.355	90.024	89.928	90.820	89.679	89.928	89.880	89.928	89.277
3046636	89.282	90.094	89.592	89.078	89.136	89.588	90.094	89.382	89.591
3274845	88.432	91.344	91.098	89.624	89.234	89.830	90.024	90.024	90.024

Appendix

Table 20: Per-seed selected validation for the dense K curve.

Comparison	Mean	Median	t 95% CI	Bootstrap 95% CI	W/T/L
K0 - K6	-0.716	-0.801	$[-1.138,-0.293]$	$[-1.039,-0.382]$	1/0/7
K1 - K6	0.090	0.000	$[-0.405,0.586]$	$[-0.257,0.504]$	3/2/3
K2 - K6	-0.314	-0.251	$[-1.043,0.415]$	$[-0.871,0.244]$	3/1/4
K3 - K6	-0.530	-0.474	$[-1.342,0.281]$	$[-1.189,0.071]$	2/0/6
K4 - K6	-0.663	-0.487	$[-1.317,-0.009]$	$[-1.221,-0.232]$	1/1/6
K5 - K6	-0.132	-0.056	$[-0.349,0.085]$	$[-0.304,0.029]$	2/1/5

Appendix

Table 21: Dense K comparisons against the frozen reference. No multiplicity correction.

W	Validation	Test	Best	Regret	Unique	Spearman	Time (s)
5	$88.865\pm 1.817$	$92.253\pm 1.853$	90.560	1.695	114.2	0.187	$327.8\pm 6.0$
10	$89.715\pm 0.423$	$93.066\pm 0.408$	90.805	1.090	134.6	0.356	$348.9\pm 2.8$
15	$89.762\pm 0.360$	$93.114\pm 0.400$	90.451	0.689	129.2	0.455	$368.9\pm 2.9$
25	$90.041\pm 0.482$	$93.388\pm 0.367$	90.663	0.623	132.9	0.525	$410.2\pm 2.6$
35	$89.837\pm 0.809$	$92.969\pm 0.553$	90.759	0.923	122.4	0.494	$447.5\pm 2.1$
50	$89.880\pm 0.289$	$93.168\pm 0.252$	90.634	0.754	136.9	0.507	$509.3\pm 1.9$

Appendix

Table 22: Dense W curve on eight CIFAR-10 seeds. Test is report-only; runtime retains all measurements.

Seed	W5	W10	W15	W25	W35	W50
4611954	86.264	89.461	89.461	90.132	89.586	89.461
4564271	89.508	89.774	89.900	89.900	89.900	89.900
2621901	89.744	89.624	89.006	89.624	89.626	89.944
1333636	90.024	88.835	90.024	89.880	90.477	89.880
3046636	85.672	90.108	89.948	90.094	88.659	90.094
3274845	89.728	89.800	90.024	90.024	90.108	89.488

Appendix

Table 23: Per-seed selected validation for the dense W curve.

Comparison	Mean	Median	t 95% CI	Bootstrap 95% CI	W/T/L
W5 - W25	-1.176	-0.344	$[-2.731,0.380]$	$[-2.507,-0.109]$	3/0/5
W10 - W25	-0.326	-0.175	$[-0.767,0.115]$	$[-0.671,0.011]$	2/1/5
W15 - W25	-0.279	-0.073	$[-0.671,0.113]$	$[-0.598,0.002]$	2/2/4
W35 - W25	-0.204	0.001	$[-0.717,0.309]$	$[-0.637,0.157]$	4/1/3
W50 - W25	-0.160	0.000	$[-0.671,0.350]$	$[-0.567,0.230]$	2/3/3

Appendix

Table 24: Dense W comparisons against the frozen reference. No multiplicity correction.

Prefix	Validation	Test	Best	Regret	Unique	Time (s)
E30	$89.976\pm 0.514$	$93.319\pm 0.386$	$90.655\pm 0.596$	$0.679\pm 0.538$	$95.5\pm 30.5$	$257.3\pm 0.9$
E60	$90.041\pm 0.482$	$93.388\pm 0.367$	$90.663\pm 0.589$	$0.623\pm 0.358$	$132.9\pm 45.4$	$410.3\pm 2.2$
E90	$90.203\pm 0.550$	$93.519\pm 0.303$	$90.729\pm 0.623$	$0.525\pm 0.352$	$145.6\pm 55.8$	$567.4\pm 8.1$
E120	$90.183\pm 0.587$	$93.432\pm 0.391$	$90.765\pm 0.596$	$0.582\pm 0.359$	$153.5\pm 57.8$	$731.4\pm 37.6$

Appendix

Table 25: Borda selection on prefixes of eight E120 trajectories; means with sample SD.

Seed	E30 val	E60 val	E90 val	E120 val	E30 uniq	E60 uniq	E90 uniq	E120 uniq
4611954	89.588	90.132	91.012	91.012	115	161	167	171
4564271	89.900	89.900	89.900	89.900	68	87	103	108
2621901	89.624	89.624	89.624	89.626	88	104	109	120
1333636	89.880	89.880	89.880	89.526	99	133	136	136
3046636	90.094	90.094	90.094	90.094	74	97	98	104
3274845	90.024	90.024	90.024	90.024	97	131	154	178

Appendix

Table 26: Per-seed selected validation and unique-genotype counts at each prefix.

Contrast	Mean	t 95% CI	Bootstrap 95% CI	W/T/L
E120-E60	0.142	[-0.191, 0.475]	[-0.088, 0.407]	3/4/1
E90-E60	0.163	[-0.109, 0.435]	[0.000, 0.383]	2/6/0
E120-E90	-0.021	[-0.146, 0.104]	[-0.133, 0.070]	2/5/1
E60-E30	0.064	[-0.098, 0.227]	[-0.011, 0.204]	1/6/1

Appendix

Table 27: Selected-validation differences between trajectory prefixes.

Block	Raw draws	Acceptance	Duplicate rej.	Incumbent rej.	New cells	Fallbacks
1–30	2,145.8	0.325	0.499	0.176	95.5	0.000
31–60	2,689.0	0.230	0.563	0.207	37.4	0.000
61–90	31,018.2	0.076	0.719	0.206	12.8	0.125
91–120	87,594.6	0.139	0.652	0.210	7.9	1.500

Appendix

Table 28: Mean 30-epoch block diagnostics over eight seeds; fallback is mean events per seed.

Quantity	CMA	Uniform	A: prefix	B: subsample
Unique genotypes	132.9	704.5	139.6	132.9
Retained epoch-evaluations	780	780	151.1	147.1
Epochs per genotype	6.38	1.11	1.08	1.11
Selected validation	90.04	89.01	89.35	88.92
Selected validation t CI	[89.64,90.44]	[88.77,89.25]	[88.79,89.90]	[88.50,89.33]
Test (report-only)	93.39	92.08	92.55	92.06

Appendix

Table 29: Offline archive-size controls. Protocol B entries average 200 subsamples within each seed.

Subset	Cells	Single rho	Mean rho	Difference	t 95% CI	Bootstrap 95% CI
All (primary)	245	0.622	0.652	0.029	[-0.001,0.059]	[-0.009,0.066]
Chance redraws	124	0.718	0.772	0.055	[0.018,0.091]	[0.004,0.102]
Incumbent-involved	121	0.053	0.063	0.011	[-0.175,0.197]	[-0.122,0.164]

Appendix

Table 30: Repeated-genotype analysis using raw stored loss. Correlations are oriented toward higher accuracy.

K	Validation	Test	Best	Regret	Unique	Spearman	Time (s)
0	$69.452\pm 0.773$	$69.640\pm 0.758$	71.485	2.033	153.5	0.323	204.2
2	$69.732\pm 1.375$	$69.657\pm 1.152$	71.220	1.489	132.2	0.358	281.3
4	$69.390\pm 1.539$	$69.562\pm 1.509$	71.467	2.077	138.6	0.371	339.8
6	$69.538\pm 1.064$	$69.615\pm 1.139$	71.546	2.007	131.1	0.399	408.9
8	$69.755\pm 1.437$	$69.755\pm 1.284$	71.978	2.223	143.2	0.370	472.0

Appendix

Table 31: CIFAR-100 depth curve on

\mathcal{S}_{8}

. Test is report-only; K6 is the frozen transfer arm.

Seed	K0 valid	K2 valid	K4 valid	K6 valid	K8 valid	K6 test
4611954	68.380	68.130	68.793	69.940	69.450	69.490
4564271	70.880	70.880	69.947	69.947	69.860	70.700
2621901	69.827	67.280	69.400	70.770	67.540	70.400
1333636	69.553	69.947	71.293	69.860	69.480	70.027
3046636	69.590	70.140	67.370	68.520	70.140	69.120
3274845	69.080	71.440	71.440	69.960	69.340	70.200

Appendix

Table 32: CIFAR-100 selected validation by depth and K6 report-only test result.

Contrast	Mean	t 95% CI	Bootstrap 95% CI	W/T/L	Sign p
K2-K0	0.279	[-0.891,1.450]	[-0.694,1.128]	5/1/2	0.4531
K4-K0	-0.063	[-1.496,1.371]	[-1.176,1.039]	4/0/4	1.0000
K6-K0	0.086	[-1.050,1.221]	[-0.835,0.913]	5/0/3	0.7266
K8-K0	0.303	[-1.242,1.847]	[-0.795,1.598]	4/0/4	1.0000
K6-K2	-0.193	[-1.879,1.493]	[-1.415,1.187]	3/0/5	0.7266
K6-K4	0.148	[-0.787,1.084]	[-0.590,0.828]	4/2/2	0.6875

Appendix

Table 33: Paired CIFAR-100 validation contrasts; all intervals are exploratory.

Seed	Validation	Test	Percentile	Best	Regret	Unique	Time (s)
4611954	44.200	45.133	96.483	45.567	1.367	143	449.7
4564271	44.683	44.533	98.122	46.183	1.500	172	453.2
2621901	44.433	44.983	97.430	45.567	1.133	173	452.4
1333636	41.778	42.267	84.835	45.633	3.856	149	453.3
3046636	39.667	40.333	71.939	45.633	5.967	107	452.4
3274845	44.678	45.022	98.090	46.033	1.356	183	453.2

Appendix

Table 34: ImageNet16-120 seed-level outcomes. Test is report-only; percentile is full-space midrank.

Metric	CIFAR-10	CIFAR-100	ImageNet16-120
Validation	$90.041\pm 0.482$	$69.538\pm 1.064$	$43.180\pm 1.759$
Test (report-only)	$93.388\pm 0.367$	$69.615\pm 1.139$	$43.720\pm 1.697$
Selected percentile	$93.52\pm 3.44$	$90.59\pm 9.49$	$91.10\pm 9.09$
Best-visited percentile	$97.24\pm 1.90$	$99.29\pm 0.57$	$99.73\pm 0.14$
Normalized oracle regret	$0.1977\pm 0.0608$	$0.3244\pm 0.0873$	$0.2749\pm 0.1361$
Spearman	$0.525\pm 0.094$	$0.399\pm 0.176$	$0.516\pm 0.103$

Appendix

Table 35: Matched eight-seed transfer diagnostics. Raw accuracy columns describe their own datasets only.

Quantity	CIFAR-10	CIFAR-100	ImageNet16-120
Exploration gap	0.1191	0.1597	0.0689
Selection gap	0.0786	0.1646	0.2060
Selection share of total gap	0.398	0.508	0.749
Seeds with selection gap larger	2/8	4/8	7/8
Best-visited Borda rank	52.8	39.9	47.1
Median selected truth rank in pool	19.5	31.0	23.0

Appendix

Table 36: Normalized exploration/selection decomposition; entries are seed means unless noted.

Dataset	Seed	Percentile	Regret	Unique	Spearman	Kendall	Best rank
CIFAR-10	4611954	95.17	1.401	161	0.424	0.293	5
CIFAR-10	4564271	93.20	0.812	87	0.559	0.392	15
CIFAR-10	2621901	89.58	0.482	104	0.636	0.464	57
CIFAR-10	1333636	92.94	0.506	133	0.587	0.420	24
CIFAR-10	3046636	94.86	0.208	97	0.504	0.366	16
CIFAR-10	3274845	94.28	0.442	131	0.623	0.442	57

Appendix

Table 37: Per-seed transfer diagnostics on the matched eight-seed cohort.

Cohort/arm	Seed	0-1	0-2	1-2	0-3	1-3	2-3
C10 I K0	1333636	C3	A3	C1	C3	C3	S
C10 I K0	2621901	C1	C3	C1	C3	N	N
C10 I K0	3011368	N	C3	C1	C3	C3	S
C10 I K0	3046636	C1	C3	S	C3	C3	S
C10 I K0	3274845	C3	S	N	C3	A3	C3
C10 I K0	3458191	S	A3	C1	C1	C3	C3

Appendix

Table 38: Returned NB201 genotypes. C10 I/P denotes initial/prospective cohort.

Setting	Value
Epochs / batch	600 / 96
Initial channels / cells	36 / 20
Training seed	123
Training / test examples	50,000 / 10,000
SGD LR / momentum / weight decay	0.025 / 0.9 / 0.0003
Scheduler / norm clipping	cosine to zero / 5

Appendix

Table 39: Fixed DARTS retraining protocol, applied separately to each selected genotype.

Dataset	Search seed	Final acc.	Final err.	Best acc.	Best epoch	Search min	Train h
CIFAR-10	184271	96.92	3.08	96.96	594	43.63	8.04
CIFAR-10	392113	96.93	3.07	97.08	569	43.51	7.36
CIFAR-10	618407	96.66	3.34	96.81	580	43.10	6.48
CIFAR-100	184271	82.93	17.07	83.22	574	43.53	8.02
CIFAR-100	392113	81.39	18.61	81.63	595	43.36	7.11
CIFAR-100	618407	82.21	17.79	82.45	593	43.49	7.26

Appendix

Table 40: All six DARTS outcomes. Best accuracy over 600 epochs is the primary endpoint; final-epoch accuracy and error are supplementary.

Dataset	Epoch 600 acc.	Epoch 600 err.	Best acc.	Active min	GPU-days
CIFAR-10	$96.84\pm 0.15$	$3.16\pm 0.15$	$96.95\pm 0.14$	$43.41\pm 0.28$	0.03015
CIFAR-100	$82.18\pm 0.77$	$17.82\pm 0.77$	$82.43\pm 0.80$	$43.46\pm 0.09$	0.03018

Appendix

Table 41: DARTS results over three architecture seeds, each retrained with seed 123. Best accuracy over 600 epochs is the primary endpoint; epoch-600 results are supplementary.

Dataset	Search seed	Excluding auxiliary	Including auxiliary
CIFAR-10	184271	3.5692	4.0457
CIFAR-10	392113	3.2267	3.7031
CIFAR-10	618407	2.9638	3.4402
CIFAR-100	184271	3.6212	4.1668
CIFAR-100	392113	3.2198	3.7655
CIFAR-100	618407	3.3520	3.8977

Appendix

Table 42: Reconstructed final-model parameter counts in millions, with counting conventions separated.

Dataset	Seed	Unique	Inc. wins	Transitions	Borda	Appearances	Peak GiB
CIFAR-10	184271	578	59	1	60.000	60	4.29
CIFAR-10	392113	667	37	23	29.917	30	4.28
CIFAR-10	618407	566	26	34	14.333	17	4.30
CIFAR-100	184271	510	59	1	60.000	60	4.16
CIFAR-100	392113	514	59	1	60.000	60	4.20
CIFAR-100	618407	574	37	23	20.583	22	4.22

Appendix

Table 43: Recorded DARTS search dynamics; peak memory is allocated GiB.

Dataset	Seed	Node	Normal incoming pairs	Reduction incoming pairs
CIFAR-10	184271	2	(S5,0), (M3,1)	(A3,0), (A3,1)
CIFAR-10	184271	3	(S5,0), (S3,1)	(S3,0), (S3,1)
CIFAR-10	184271	4	(D5,0), (S3,1)	(S3,0), (A3,2)
CIFAR-10	184271	5	(M3,3), (D5,4)	(S3,0), (D3,1)
CIFAR-10	392113	2	(D5,0), (M3,1)	(M3,0), (M3,1)
CIFAR-10	392113	3	(A3,0), (D5,2)	(S5,0), (S3,2)

Appendix

Table 44: Complete normal and reduction cells for every reported DARTS search.

Cohort	Arm	Timing n	Warm-up	Search	Selection	Complete
Initial 16	K6	16	99.7	310.7	0.0200	410.4
Initial 16	K0	14	99.7	106.0	0.0187	205.8
Prospective 8	K6	8	99.5	307.0	0.0196	406.4
Prospective 8	K0	8	99.5	105.2	0.0181	204.7
Pooled 24	K6	24	99.6	309.5	0.0198	409.1
Pooled 24	K0	22	99.6	105.7	0.0185	205.4

Appendix

Table 45: Runtime decomposition in seconds. NB201 K0 timing-only exclusions do not exclude accuracy observations.

Component	CIFAR-10	CIFAR-100	ImageNet16-120
Search initialization	0.63	0.79	1.59
Panel loading	33.98	32.23	29.36
Panel fingerprints	2.34	2.36	0.84
Candidate calibration	195.66	195.72	94.10
Focal BN refresh	28.18	26.46	15.68
Validation scoring	9.52	9.07	9.13

Appendix

Table 46: Instrumented search components, mean seconds over matched transfer seeds.

Block	Seed	K0 valid	K6 valid	Difference	K0 test	K6 test
original	224307	89.314	91.072	1.758	92.415	93.833
original	283331	89.593	90.024	0.431	93.087	93.425
original	524955	89.654	90.063	0.409	92.990	93.137
original	544369	88.731	91.116	2.385	92.140	94.020
original	557081	89.856	91.072	1.216	93.415	93.833
original	636408	90.778	89.471	-1.307	93.730	93.160

Appendix

Table 47: Earlier development pairs, excluded from the current 24-seed principal synthesis.

Seed	Validation	Test	Best	Regret	Unique	Time (s)
738731	69.350	69.630	71.440	2.090	91	410.5
283331	68.800	69.230	71.680	2.880	124	405.3
224307	69.360	69.153	71.490	2.130	131	405.5
544369	67.910	68.040	71.490	3.580	138	401.5
671413	68.873	67.960	71.750	2.877	109	403.5
636408	69.280	69.360	71.490	2.210	109	415.6

Appendix

Table 48: Earlier CIFAR-100 transfer cohort, distinct from the final depth-study seeds.

Seed	Sigma	Validation	Test	Best	Regret	Unique
1203987	0.30	90.010	92.815	90.392	0.382	148
1203987	0.35	89.956	93.020	91.106	1.150	148
1203987	0.40	88.896	92.395	90.841	1.945	168
1203987	0.45	90.305	93.080	90.656	0.351	146
1203987	0.50	89.284	92.205	91.106	1.822	141
1203987	0.55	89.516	92.890	91.188	1.672	139

Appendix

Table 49: Complete four-seed CMA-scale development grid. Validation, not test, informed development.

Seed	K0 valid	K6 valid	Difference	K6 test	K6 best	K6 regret
3170282	88.838	89.345	0.507	92.600	90.712	1.367
3981128	88.426	89.196	0.770	92.920	91.237	2.041
7054363	90.544	91.106	0.562	93.760	91.106	0.000
4010385	90.902	89.108	-1.794	92.020	90.364	1.256
3428960	90.904	90.986	0.082	93.655	91.607	0.621
6505985	89.132	89.336	0.204	92.235	91.106	1.770

Appendix

Table 50: Separate 16-seed v4 study at initial CMA sigma 0.45; not pooled with sigma 0.30.

K	Validation	Best	Regret	Unique	Spearman	Search (s)
0	$89.392\pm 1.284$	$90.738\pm 0.490$	$1.346\pm 0.923$	$207.4\pm 27.6$	$0.372\pm 0.125$	$101.3\pm 0.9$
2	$89.344\pm 1.063$	$90.736\pm 0.618$	$1.391\pm 0.681$	$188.6\pm 28.1$	$0.293\pm 0.140$	$173.1\pm 2.7$
4	$89.828\pm 0.806$	$91.139\pm 0.469$	$1.311\pm 0.626$	$158.8\pm 35.4$	$0.467\pm 0.073$	$241.2\pm 3.0$
6	$89.911\pm 0.813$	$91.045\pm 0.368$	$1.134\pm 0.655$	$156.6\pm 31.2$	$0.463\pm 0.097$	$306.4\pm 6.7$
8	$89.919\pm 0.975$	$91.130\pm 0.491$	$1.211\pm 0.587$	$186.9\pm 51.7$	$0.395\pm 0.118$	$370.8\pm 2.9$

Appendix

Table 51: Secondary eight-seed depth curve at sigma 0.45. Runtime is search-only, not complete-method time.

Family	Authoritative artifact scope
Principal CIFAR-10	Final evidence suite: initial16, prospective8, matched controls, selector replay, runtime tables.
Dense K/W	Merged September 15 analysis: 112 compatible runs; K=0–8 and W=5,10,15,25,35,50.
Conditioning control	September 21 candidate-independent fair-calibration analysis, eight matched pairs.
CIFAR-100	September 9 depth-study per-run rows and aggregate/paired tables.
ImageNet16-120	Independent September 14 transfer analysis and matched dataset diagnostics.
Longer search	Independent E120 budget analysis: prefix and block tables.

Appendix

Table 52: Artifact families used to construct the appendix.

Explore similar work

May 19, 2026cs.CV

Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search

Current neural architecture search (NAS) methods are often limited by their predefined, restrictive search spaces. While recent large language model (LLM)-assisted NAS methods enable open-ended search spaces, they often suffer from inefficient exploration due to biased or low-quality design ideas. To address these issues, we propose to semi-automatically structure model design knowledge to guide the search process. Our approach first defines a high-level structural template of architectural attributes. An LLM then populates this template by analyzing papers, creating a rich and diverse search space that embodies this structured design knowledge. To efficiently explore this vast space, we introduce FairNAD, using a multi-type mutation that enables broad exploration through mutation with fair idea sampling, Pareto-aware mutation, LLM-driven iterative mutation, and a fine-grained feedback loop. We demonstrate the effectiveness of FairNAD in discovering high-performing architectures that yield 0.84, 2.17, and 2.35 points improvement on CIFAR-10, CIFAR-100, and ImageNet16-120, respectively, compared to current state-of-the-art methods.

Yuiko Sakuma, Masakazu Yoshimura, Marcel Gröpl +4

Sony Group Corporation, Tokyo, Japan · ETH Zurich, Switzerland

Neural Architecture Search Lattice Mutation

Ask PDF

Sep 10, 2026cs.LG

CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search

Zero-cost proxies rank architectures cheaply, but their reliability varies across search spaces. We introduce CoRA-NAS (COarse Ranking + Anchor-residual), a two-stage framework combining a static ranking prior with low-cost learning-curve refinement. CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus and a target-free consensus gate. CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model. The refinement uses approximately 1% of the cost of fully training the candidate set. Fully trained architecture-accuracy labels are not used to fit the ranker. One configuration is used across spaces, with space-specific architecture encodings. Across NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101, and NATS-SSS, CoRA-Refine achieves mean Spearman correlations of 0.946, 0.715, 0.786, and 0.894, respectively. Its worst-space correlation of 0.715 is the highest among the compared methods. On NAS-Bench-201/CIFAR-100, its selected architecture reaches 73.32% accuracy, near the reported ground-truth best of 73.37%. On the pure size space, refinement recovers the static prior's shortfall relative to parameter count, while remaining tied with the strongest capacity proxies within noise. The resulting framework combines cross-space ranking robustness with low-cost architecture selection.

Yifan Yang, Zhaoyan Wang, Zheng Gao +2

University of New South Wales · Korea Advanced Institute of Science and Technology

Neural Architecture Search Capability-Speed Trade-Offs Cross-Architecture Generalization Search Algorithms

Ask PDF

Oct 1, 2026cs.LG

Prediction-powered Neural Architecture Search

Evaluating candidate architectures in neural architecture search (NAS) faces an inherent trade-off: on the one hand, reliable performance labels are limited because training and evaluating architectures is costly; on the other hand, zero-cost proxies (ZCPs) are cheap to compute at large scale but can be noisy. Yet, how to effectively combine these two sources of supervision remains unclear. In this paper, we propose PPNAS, a novel prediction-powered inference (PPI) approach for NAS. PPNAS fuses (1) a small set of architectures with observed performance labels and (2) a large set of architectures with ZCP information. To combine these two sources of supervision, PPNAS exploits the ordinal information provided by ZCPs to construct additional pairwise ranking supervision, while PPI debiases systematic discrepancies between ZCP-based and true performance rankings. We evaluate PPNAS in end-to-end predictor-based NAS, where it achieves state-of-the-art under limited evaluation budgets. To the best of our knowledge, PPNAS is the first prediction-powered approach for label-efficient NAS.

Pascal Janetzky, Yuxin Wang, Michael Klar +1

Bosch Center for Artificial Intelligence · LMU Munich Munich Center for Machine Learning(MCML)

Neural Architecture Search

Ask PDF