Prediction-powered Neural Architecture Search

Pascal Janetzky; Yuxin Wang; Michael Klar; Stefan Feuerriegel

cs.LGOct 1, 2026

Prediction-powered Neural Architecture Search

Authors: Pascal Janetzky, Yuxin Wang, Michael Klar, Stefan Feuerriegel

Organizations: Bosch Center for Artificial Intelligence · LMU Munich Munich Center for Machine Learning(MCML)

Abstract

Evaluating candidate architectures in neural architecture search (NAS) faces an inherent trade-off: on the one hand, reliable performance labels are limited because training and evaluating architectures is costly; on the other hand, zero-cost proxies (ZCPs) are cheap to compute at large scale but can be noisy. Yet, how to effectively combine these two sources of supervision remains unclear. In this paper, we propose PPNAS, a novel prediction-powered inference (PPI) approach for NAS. PPNAS fuses (1) a small set of architectures with observed performance labels and (2) a large set of architectures with ZCP information. To combine these two sources of supervision, PPNAS exploits the ordinal information provided by ZCPs to construct additional pairwise ranking supervision, while PPI debiases systematic discrepancies between ZCP-based and true performance rankings. We evaluate PPNAS in end-to-end predictor-based NAS, where it achieves state-of-the-art under limited evaluation budgets. To the best of our knowledge, PPNAS is the first prediction-powered approach for label-efficient NAS.

Figures & tables

Figure 1: Prediction-powered neural architecture search ( ppNAS ). We have access to reliable labeled and noisy zero-cost only datasets (left). From both datasets, we construct pairwise ranking supervision (middle-left). The ranking supervision is then used to learn an architecture ranking model with parameters

\theta

, where the contribution of the noisy data is adaptively weighted (middle-right). The trained model is then used in end-to-end NAS to rank candidate architectures (right).

Figure 2: Main NB201 search results (with

n=10

initial ground-truth-labeled architectures and the nwot ZCP). Curves show best-so-far validation accuracy as a function of consumed ground-truth evaluations. ppNAS reaches strong incumbents early in the search, and can reach a higher validation accuracy several iterations earlier.

Figure 3: Breakdown by ZCPs (on NB201/CIFAR-10). Each cell reports mean AUC $\mid$ mean AUC rank, color is determined by the AUC. Note that ZCP-independent methods have identical values across rows. ppNAS ranks first except for synflow .

Figure 4: Change of

\lambda

over NAS iterations.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Search trajectory on NB201 using grad_norm .

Figure 6: Search trajectory on NB201 using jacov (Jacobian covariance).

Figure 7: Search trajectory on NB201 using grasp .

Figure 8: Search trajectory on NB201 using snip .

Figure 9: Search trajectory on NB201 using synflow .

Figure 10:

\lambda

as a function of the number of ground-truth-labeled architecture acquired during NAS iterations.

Panel of Figure 11 — Figure 11: Evolution of $\lambda$ over time.

Figure 12: grad norm, gold = 5

Figure 13: The effect of different

N

on the final mean AUC.

Figure 14: The effect of different

N

on incumbent curves.

	CIFAR10		Rank $\downarrow$
Method	Final Acc	AUC	Mean rank	Median rank
Random search	94.17 $\pm$ 0.23	94.03 $\pm$ 0.30	9.17	9.00
ProxyBO	94.26 $\pm$ 0.22	94.06 $\pm$ 0.21	8.33	8.00
ZCP only	94.20 $\pm$ 0.36	94.01 $\pm$ 0.34	7.50	9.00
RankNAS	94.42 $\pm$ 0.20	94.18 $\pm$ 0.15	6.67	7.00
ZCPBP Warmup	94.47 $\pm$ 0.18	94.15 $\pm$ 0.23	6.50	5.50

Appendix

Table 6: NB101 CIFAR10 search space.

Method	Final Acc	AUC	Mean rank	Median rank
	AutoEncoder		Rank $\downarrow$
Random search	0.73 $\pm$ 0.02	0.70 $\pm$ 0.02	9.60	10.00
ReNAS	0.74 $\pm$ 0.01	0.71 $\pm$ 0.02	8.00	8.00
RankNAS	0.73 $\pm$ 0.02	0.71 $\pm$ 0.03	7.00	7.00
ZCPBP Warmup	0.73 $\pm$ 0.01	0.71 $\pm$ 0.02	6.40	6.00
PWLNAS	0.74 $\pm$ 0.01	0.71 $\pm$ 0.02	6.00	6.00

Appendix

Table 7: TNB autoencoder search space.

Figure 15: Per-proxy results for CIFAR100 from NB201.

Figure 16: Per-proxy results for ImageNet from NB201.

Figure 17: Spearman ranking correlation between zero-cost proxies and validation accuracy for CIFAR10 from the NB101 benchmark.

Figure 18: Spearman ranking correlation between zero-cost proxies and validation accuracy for CIFAR10 from the NB101 benchmark.

Figure 19: Per-proxy results for CIFAR10 from NB101

Figure 20: Runtimes. ppNAS is comparable to the baselines.

Explore similar work

RiPPLE: Cross-Space Performance Prediction from Early Training for Neural Architecture Search
Sep 14, 2026Yifan Yang, Zhaoyan Wang, Zheng Gao +2Neural Architecture Search Search Algorithms
University of New South Wales · Korea Advanced Institute of Science and Technology
From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search
May 10, 2026Liping Deng, MingQing XiaoNeural Architecture Search Meta-Learning
Department of Mathematics University of California, Riverside Riverside, CA 92507 USA · School of Mathematical and Statistical Sciences Southern Illinois University Carbondale Carbondale, IL 62901 USA
Agentic Neural Architecture Search
Jul 8, 2026Seokhoon Jeong, Mijung Kim, Taehwan KimNeural Architecture Search Agent2Agent
Artificial Intelligence Graduate School, Ulsan National Institute of Science and Engineering, Ulsan, Republic of Korea · Department of Computer Science and Engineering, Ulsan National Institute of Science and Engineering, Ulsan, Republic of Korea

Sep 14, 2026, cs.LGJ/K move · Enter open · S save

RiPPLE: Cross-Space Performance Prediction from Early Training for Neural Architecture Search

Yifan Yang, Zhaoyan Wang, Zheng Gao, Xiaoyu Li+1

University of New South Wales · Korea Advanced Institute of Science and Technology

Neural Architecture Search Search Algorithms Ranking

Open paper AskPDF arXiv

Neural architecture search (NAS) evaluates candidate networks, but fully training enough architectures to rank an entire space is expensive. Zero-cost proxies score architectures at initialization, yet their ranking quality varies across search spaces. Learned predictors reduce evaluation cost but typically require fully trained labels or partial-training features for individual candidates. We introduce

\textbf{RiPPLE}

\underline{\textbf{R}}

anking v

\underline{\textbf{i}}

\underline{\textbf{P}}

refix-

\underline{\textbf{P}}

ropagated

\underline{\textbf{L}}

abel

\underline{\textbf{E}}

xtrapolation, which treats partial training as a source of labels for a small coverage set of anchors. RiPPLE trains these anchors to an early prefix, extrapolates their learning curves to surrogate labels, and propagates the labels over label-free architecture features. The early-training signal remains a label on the anchors rather than a per-candidate feature. Feature, readout, and encoding rules are selected without held-out accuracy and reused across search spaces. We evaluate the method on twelve benchmark cells from four search-space families and on the larger DARTS space. The results examine ranking quality, label efficiency, architecture selection, and the roles of readout, coverage, and propagation. RiPPLE provides a whole-space ranking from a fractional anchor-training budget, with comparisons interpreted under their respective evaluation and cost protocols.

Abstract

Explore similar work

RiPPLE: Cross-Space Performance Prediction from Early Training for Neural Architecture Search

From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search

Agentic Neural Architecture Search