Searching with Sparse Histories: Cross-Task Architecture Search from Incomplete Search Logs and Zero-Cost Proxies

Yuefeng Xu; Jun Yu

cs.LGOct 23, 2026

Searching with Sparse Histories: Cross-Task Architecture Search from Incomplete Search Logs and Zero-Cost Proxies

Authors: Yuefeng Xu, Jun Yu

Organizations: Graduate School of Science and Technology, Niigata University, Niigata, Japan · Institute of Science and Technology, Niigata University, Niigata, Japan

Abstract

Neural architecture search (NAS) is expensive, but cross-task transfer can reduce costs by reusing knowledge from previous tasks. However, existing methods struggle with sparse search histories and task-specific transfer failures. We propose a method, a risk-robust strategy that combines sparse source logs with zero-cost proxies through champion-based initialization and joint pairwise-ranker pretraining. On TransNAS-Bench-101, the proposal mitigates severe worst-case failures, reducing worst-10% ranks from 290 to 43 for Micro autoencoding and from 465 to 45 for Macro room layout. Across 13 cell-space targets, it outperforms a proxy-only ranker on 10 targets in both mean and worst-case performance, while maintaining competitive average performance.

Figures & tables

Figure 1: Overview of SHINE. Sparse source logs (each past task has only

m

evaluated architectures) are imputed per source to build a champion portfolio

\mathcal{Q}_{0}

that initialises the search. A pairwise ranker is pretrained jointly on all sparse source tasks and on zero-cost proxies treated as pseudo-tasks; on the new target it is repeatedly fine-tuned on the evaluated architectures and used to sample the next ones from its predicted top-

k

until the budget

B

is spent.

		6 tasks (mean rank)		4 tasks (rank of mean)		Worst 10%
Method	Source knowledge	Macro	Micro	Macro	Micro	Macro	Micro
Random search	none	59.26	85.61	–	–	–	–
REA [ 21 ]	none	41.03	38.50	–	–	–	–
weakNAS [ 29 ]	none	10.49	20.03	–	–	–	–
Arch-Graph-single	none	12.68	22.15	–	–	–	–
CATCH [ 5 ]	jigsaw + others	37.72	63.49	–	–	–	–

Table 1: Cross-task search on TransNAS-Bench-101 with 50 target evaluations. Global rank of the returned architecture (lower is better). “6 tasks”: mean rank over all six targets. “4 tasks”: rank of the mean metric value over the four targets whose metric matches published tables (object/scene classification, autoencoding, normals), the convention of [ 11 , 15 ] . “Worst 10%”: mean rank of the worst 10% of runs. Published numbers are taken from [ 11 ] ; TRP numbers are converted from the metric values reported in [ 15 ] . Our implementation:

m{=}50

evaluated architectures per source, 20 seeds, leakage-free proxies.

	Macro			Micro
Target	ZC	Src	Both	ZC	Src	Both
Object cls.	3.0	4.2	3.1	16.3	9.9	9.4
Scene cls.	1.8	1.7	2.2	20.2	5.1	3.6
Autoencoding	3.2	2.2	2.0	43.2	34.7	22.3
Normals	1.0	1.4	1.4	10.2	10.1	12.2
Sem. seg.	1.1	1.6	1.2	4.0	9.4	5.0

Table 2: Per-target mean rank of a ranker pretrained on zero-cost proxies only (ZC), on sparse sources only (Src), and on both (SHINE). 20 seeds.

	Mean rank (targets better)		Worst 10%
Variant	Macro	Micro	Macro	Micro
SHINE (full)	4.21	17.84	20.9	88.9
w/o source logs	5.49 (2/6)	25.18 (4/6)	34.8	152.1
w/o proxies	14.26 (4/6)	17.82 (4/6)	120.2	80.5
w/o portfolio init.	6.07 (6/6)	19.64 (4/6)	31.5	102.1
w/o pretraining	8.57 (6/6)	14.07 (1/6)	56.8	66.8

Table 3: Ablations (

B{=}50

m{=}50

, 20 seeds). Mean rank over six targets, in parentheses the number of targets (of six) on which the full method is better after averaging over seeds, and mean rank of the worst 10% of runs.

Figure 2: (a,b) Mean rank over six targets vs . the number of target evaluations

B

(10 seeds for

B\in\{25,100\}

, 20 for

B{=}50

). (c) Mean rank of SHINE vs . the number of evaluated architectures per source

m

(

B{=}50

Method	Regret	Rank	Regret	Rank
	Tiny (768, $B{=}30$ )		Small (5,016, $B{=}50$ )
Random	0.272	195.1	0.253	1256
Target-only ranker	0.221	147.0	0.265	1317
Trees + sources	0.224	161.4	0.229	1122
SHINE w/o ZC, $m{=}50$	0.189	115.7	0.230	1107
SHINE w/o ZC, $m{=}200$	0.202	143.6	0.174	613

Table 4: MedMNIST-NAS-Bench (never used during development). Normalised test regret (lower is better) and test rank of the architecture selected by validation AUROC, averaged over four tasks and 20 seeds. No zero-cost proxies are used here.

	Mean (%)			CVaR 10 (%)
Target	ZC only	Hist. only	SHINE	ZC only	Hist. only	SHINE
TN object cls. †	0.40	0.24	0.23	1.93	0.68	0.66
TN scene cls.	0.49	0.12	0.09	1.42	0.54	0.38
TN autoencoding	1.05	0.85	0.54	7.09	2.64	1.05
TN normals	0.25	0.25	0.30	0.59	0.52	0.77
TN sem. seg.	0.10	0.23	0.12	0.20	0.93	0.24

Table 5: Pooled analysis over 13 Micro targets (analysis plan fixed before the last four targets were run). Rank of the returned architecture as a percentage of the pool (lower is better): mean and CVaR 10 (mean of the worst 10% of runs). TN: TransNAS-Bench-101 (

B{=}50

, 20 seeds, pool 4,096). RT: real training (pool 300, averaged over

B{\in}\{30,50\}

, 30 seeds each). Bottom: number of targets on which SHINE is better, and task-level Wilcoxon

p

-values (Holm-corrected over the two comparisons). † Leakage-free proxies (computed on scene classification).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	TransNAS		MedMNIST
Method	Macro	Micro	Tiny	Small
Pretrained (SHINE)	4.30 / 2.25	17.70 / 9.00	0.189	0.230
No pretraining	9.08 / 2.75	14.45 / 9.00	0.221	0.265
Online gate	5.32 / 2.75	15.52 / 9.50	0.215	0.232
Fixed 0.5 ensemble	5.13 / 2.00	17.29 / 8.00	0.223	0.224

Appendix

Table 6: Exploratory gate (20 seeds; run with the original object-classification proxies). TransNAS: mean rank over six targets / rank of mean value on the four matched targets. MedMNIST: test regret (

m{=}50

, no proxies). The gate did not meet its pre-declared criterion (within 1 rank, resp. 0.01 regret, of the better single model).

Method	Logs	Macro	Micro
ZC-only ranker	–	5.49 / 1.75 / 34.8	25.18 / 13.50 / 152.1
Sources-only ranker	uniform	14.26 / 2.50 / 120.2	17.82 / 12.50 / 80.5
Sources-only ranker	trajectory	3.64 / 2.00 / 18.0	18.74 / 9.00 / 92.4
SHINE	uniform	4.21 / 2.00 / 20.9	17.84 / 9.00 / 88.9
SHINE	trajectory	3.78 / 2.00 / 17.8	20.63 / 11.00 / 110.8

Appendix

Table 7: Uniform vs . search-trajectory source logs (

B{=}50

m{=}50

, 20 seeds). Trajectory logs are the 50 architectures evaluated by an Extra-Trees weak-predictor search on each source. Mean rank over six targets / rank of the mean value on the four matched targets / worst-10%.

	Macro			Micro
Target	SHINE	ZC	Src	SHINE	ZC	Src
Object cls.	9.0	6.0	13.0	27.0	79.0	28.0
Scene cls.	11.0	8.5	5.5	15.5	58.0	22.0
Autoencoding	5.0	5.5	9.0	43.0	290.5	108.0
Normals	3.0	1.0	4.0	31.5	24.0	21.5
Sem. seg.	3.0	2.0	4.0	10.0	8.0	38.0

Appendix

Table 8: Worst-10% rank per target (mean of the two worst of 20 seeds;

B{=}50

m{=}50

) for SHINE, the ZC-only ranker and the sources-only ranker, and pooled CVaR over all 120 runs with the 95% hierarchical bootstrap interval (targets, then seeds) of the difference SHINE

-

ZC.

Macro
Target	$B$	Random	Target-only	ZC only	Hist. only	SHINE
CIFAR-100	30	11.0 (29)	1.1 (2)	1.0 (1)	1.0 (1)	1.0 (1)
	50	7.8 (28)	1.0 (1)	1.0 (1)	1.0 (1)	1.0 (1)
Tiny-IN cls.	30	11.0 (26)	2.0 (2)	2.0 (2)	2.0 (2)	2.0 (2)
	50	6.7 (25)	2.0 (2)	2.0 (2)	2.0 (2)	2.0 (2)
Pets cls.	30	10.6 (37)	3.1 (17)	5.3 (14)	1.5 (6)	2.4 (14)

Appendix

Table 9: Real training, per target. Test rank (lower is better) of the architecture selected on the validation set within a pool of 300 architectures, mean over 30 seeds, worst-10% in parentheses. Metrics: accuracy (classification), SSIM (reconstruction, denoising), PSNR inside the hole (inpainting), mIoU (segmentation).

		vs ZC only		vs Hist. only
Subset	$n$	Mean	CVaR 10	Mean	CVaR 10
All targets	13	10 (.006)	10 (.008)	7 (.69)	6 (.46)
Seen before the plan	9	6 (.10)	6 (.13)	6 (.73)	6 (.43)
Run after the plan	4	4 (.13)	4 (.13)	1 (.88)	0 (.13)
TransNAS only	6	4 (.16)	4 (.16)	4 (.84)	4 (.69)
Real training only	7	6 (.031)	6 (.047)	3 (.58)	2 (.30)

Appendix

Table 10: Sensitivity of the pooled comparison (not part of the fixed plan). Number of targets on which SHINE is better and task-level Wilcoxon

p

(uncorrected), for mean rank and CVaR 10 (% of pool).

Neural Architecture Performance Prediction

See all 13 papers

ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search
Sep 12, 2026Hassan Touayouch, Rabie Najem, Mohammed BenjellounNeural Architecture Search Surrogate-Assisted Optimization
ENSICAEN · ISIA Lab, Université de Mons
QuaNAS: Uncertainty-Aware Reuse of Earlier Searches for Query-Efficient Neural Architecture Search
Oct 22, 2026Brama Seta Putra Satria, Dwi Sunaryono, Misbakhul Munir Irfan Subakti +2Neural Architecture Search Uncertainty-Aware Exploration
Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia · Department of Information Technology, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia · Department of Electrical Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search
Sep 10, 2026Yifan Yang, Zhaoyan Wang, Zheng Gao +2Neural Architecture Search
University of New South Wales · Korea Advanced Institute of Science and Technology

Sep 12, 2026J/K move · Enter open · S save

ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search

Hassan Touayouch, Rabie Najem, Mohammed Benjelloun

ENSICAEN · ISIA Lab, Université de Mons

Neural Architecture Search Surrogate-Assisted Optimization Active Learning

Open paper AskPDF arXiv

Neural Architecture Search (NAS) automates network design, but evaluating a single candidate requires training it to convergence, making exhaustive search intractable. Zero-cost proxies estimate architecture quality at initialization in seconds, yet a single proxy is noisy, and combining several does not straightforwardly help: proxies are strongly correlated, so naive aggregation compounds their shared errors instead of averaging them out. Existing methods exploit either proxy signals or architectural topology - never both within a single active-learning framework. We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap. ZAPS (i) selects a compact, non-redundant proxy subset offline via ProxyFit, a greedy anti-redundancy criterion; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a bootstrapped vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings, queried through an Upper Confidence Bound (UCB) acquisition function. On NAS-Bench-201 under a budget of B=200 evaluations, ZAPS recovers 52.3% of the true top-100 architectures on CIFAR-10 and 65.8% on CIFAR-100, ahead of every baseline we consider - Random Search, Local Search, REA, BANANAS and TPE - and, on CIFAR-10, with less than half the run-to-run standard deviation of the strongest of them. The advantage is largest where evaluations are scarce: on NAS-Bench-201 it narrows as the budget grows, whereas on the harder NAS-Bench-101, which no method comes close to saturating, it widens instead. All methods are scored by a single criterion: how much of the true top-100 lies among the architectures they actually evaluated.

Abstract

Neural Architecture Performance Prediction

Explore similar work

ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search

QuaNAS: Uncertainty-Aware Reuse of Earlier Searches for Query-Efficient Neural Architecture Search

CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search