Neural Architecture Search (NAS) automates network design, but evaluating a single candidate requires training it to convergence, making exhaustive search intractable. Zero-cost proxies estimate architecture quality at initialization in seconds, yet a single proxy is noisy, and combining several does not straightforwardly help: proxies are strongly correlated, so naive aggregation compounds their shared errors instead of averaging them out. Existing methods exploit either proxy signals or architectural topology - never both within a single active-learning framework. We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap. ZAPS (i) selects a compact, non-redundant proxy subset offline via ProxyFit, a greedy anti-redundancy criterion; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a bootstrapped vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings, queried through an Upper Confidence Bound (UCB) acquisition function. On NAS-Bench-201 under a budget of B=200 evaluations, ZAPS recovers 52.3% of the true top-100 architectures on CIFAR-10 and 65.8% on CIFAR-100, ahead of every baseline we consider - Random Search, Local Search, REA, BANANAS and TPE - and, on CIFAR-10, with less than half the run-to-run standard deviation of the strongest of them. The advantage is largest where evaluations are scarce: on NAS-Bench-201 it narrows as the budget grows, whereas on the harder NAS-Bench-101, which no method comes close to saturating, it widens instead. All methods are scored by a single criterion: how much of the true top-100 lies among the architectures they actually evaluated.
Figures & tables
Figure 1: The four stages of ZAPS . ProxyFit runs once offline and returns a proxy subset; the hybrid K-means seeds the labeled set; re-selection, surrogate fitting and UCB querying then alternate for T iterations, each one spending part of the budget on newly evaluated architectures.
Figure 2: Absolute Spearman correlation between proxy pairs on NAS-Bench-201 / CIFAR-10, with proxies ordered by decreasing relevance—the order in which ProxyFit examines them. Redundancy is assessed on ∣ρ∣ , since two strongly anti-correlated proxies are as redundant as two strongly correlated ones. Highly correlated pairs such as flops – params ( ∣ρ∣=0.996 ) are the redundancies ProxyFit is designed to eliminate.
Figure 3: P@100 vs. budget B on NAS-Bench-201 (top: CIFAR-10, bottom: CIFAR-100); means over 200 seeds.
Figure 4: Regret vs. budget B , same setting; geometric means over 200 seeds, floored at 10−5 .
Figure 5: Convergence of ZAPS alone: P@100 vs. budget B on CIFAR-10 and CIFAR-100; means and interquartile bands over 200 seeds.
Figure 6: Regret of ZAPS alone, same setting; geometric means over 200 seeds, floored at 10−5 .
Figure 7: Spearman correlation between the surrogate ranking and the true ranking of all 15,625 architectures of NAS-Bench-201, on CIFAR-10 and CIFAR-100; means and interquartile bands over 200 seeds.
Figure 8: Ablation of ProxyFit across the full budget range on CIFAR-10 and CIFAR-100: the ProxyFit subset of six proxies against the full pool of thirteen. Lines are means over 200 seeds and shaded bands interquartile ranges. Table 6 reports the values at B=200 .
Figure 9: Ablation of the initialization across the full budget range on CIFAR-10 and CIFAR-100: hybrid K-means initialization against uniform random sampling. Lines are means over 200 seeds and shaded bands interquartile ranges. Table 6 reports the values at B=200 .
Surrogate input
Dim.
CIFAR-10
CIFAR-100
Proxy ranks + topology
37 – 40
52.3±6.1
65.8±11.0
proxy ranks only
7 – 10
47.8±8.4
58.7±14.0
topology only
30
31.1±8.0
43.4±11.0
Table 7: Fusion ablation on NAS-Bench-201 at B=200 , 200 seeds. Only the surrogate’s input changes; ProxyFit , initialization, re-selection and UCB acquisition are identical across rows. The proxy block is the online re-selection’s output, whose size grows with the labeled set: 7 proxies at the first fit, 10 thereafter.
Figure 10: Fusion ablation, 200 seeds; only the surrogate’s input differs. Topology-only never closes its gap; proxies-only converges to the full pipeline. Regret on a linear scale, since ZAPS reaches exactly zero.
Figure 11: P@100 (top) and regret (bottom) on NAS-Bench-101 / CIFAR-10; means over 200 seeds, with interquartile bands for P@100 .
CIFAR-100
ImageNet16-120
B
native
transferred
native
transferred
100
24.2
32.7
16.7
14.8
200
65.8
62.0
26.4
27.5
300
76.1
76.5
44.9
48.9
500
88.1
87.1
68.1
70.7
Table 9: P@100 (%) when ProxyFit is calibrated on the target dataset ( native ) versus transferred unchanged from CIFAR-10 ( transferred ). Mean over 200 seeds. A transferred subset uses no ground-truth accuracy from the target dataset.
Figure 12: Cross-dataset transfer: the subset calibrated on CIFAR-10, applied unchanged to CIFAR-100 and ImageNet16-120 ( transferred ), against one calibrated on the target itself ( native ). Values in Table 9 .
Figure 13: Cross-space transfer: the subset calibrated on NAS-Bench-201, applied unchanged to NAS-Bench-101 ( transferred ), against one calibrated on NAS-Bench-101 ( oracle ). They differ by at most 4.2 pp in P@100 and 0.07 pp in regret.