Organizations: Hangzhou Normal University, Hangzhou, P.R. China · Trustworthy and General AI Lab, School of Engineering, Westlake University, Hangzhou, P.R. China · Department of Data Science and Artificial Intelligence, and the Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, China
Modern computer vision requires balancing predictive accuracy with real-time efficiency, yet the high inference cost of large vision models (LVMs) limits deployment on resource-constrained edge devices. Although Evolutionary Neural Architecture Search (ENAS) is well suited for multi-objective optimization, its practical use is hindered by two issues: expensive candidate evaluation and ranking inconsistency among subnetworks. To address them, we propose EvoNAS, an efficient distributed framework for multi-objective evolutionary architecture search. We build a hybrid supernet that integrates Vision State Space and Vision Transformer (VSS-ViT) modules, and optimize it with a Cross-Architecture Dual-Domain Knowledge Distillation (CA-DDKD) strategy. By coupling the computational efficiency of VSS blocks with the semantic expressiveness of ViT modules, CA-DDKD improves the representational capacity of the shared supernet and enhances ranking consistency, enabling reliable fitness estimation during evolution without extra fine-tuning. To reduce the cost of large-scale validation, we further introduce a Distributed Multi-Model Parallel Evaluation (DMMPE) framework based on GPU resource pooling and asynchronous scheduling. Compared with conventional data-parallel evaluation, DMMPE improves efficiency by over 70% through concurrent multi-GPU, multi-model execution. Experiments on COCO, ADE20K, KITTI, and NYU-Depth v2 show that the searched architectures, termed EvoNets, consistently achieve Pareto-optimal trade-offs between accuracy and efficiency. Compared with representative CNN-, ViT-, and Mamba-based models, EvoNets deliver lower inference latency and higher throughput under strict computational budgets while maintaining strong generalization on downstream tasks such as novel view synthesis. Code is available at https://github.com/EMI-Group/evonas
Figures & tables
Fig. 1: The EvoNAS unified visual architecture, featuring a searchable hybrid VSS-ViT encoder, a Spatial Mamba decoder, and optional task heads for detection, segmentation, and MDE. Searchable components within the Evo-Mamba and Evo-Transformer blocks define the encoder’s supernet search space, enabling flexible and general-purpose architecture optimization across diverse vision tasks.
Fig. 2: Overview of the two-stage supernet optimization with dual-domain knowledge distillation. Stage 1 pretrains the VSS-ViT supernet encoder on ImageNet-1K to establish a stable initialization. Stage 2 adapts the supernet to the target task by aligning student VSS/ViT features with teacher ViT features through spatial- and frequency-domain distillation, while jointly applying prediction-level and ground-truth supervision.
Fig. 3: Overview of the EvoNAS pipeline. The framework integrates distributed multi-model parallel evaluation with multi-objective evolutionary search, enabling efficient validation of candidate architectures across GPUs and progressive approximation of the Pareto frontier in terms of accuracy, latency, and computational complexity.
Fig. 4: Performance–efficiency trade-offs of EvoNAS across four benchmarks: (a) COCO, (b) ADE20K, (c) KITTI, and (d) NYU v2. The x-axis denotes MACs, the y-axis represents task-specific accuracy metrics (mAP, mIoU, or AbsRel), bubble size indicates parameter count, and color encodes inference latency.
Model
Params
MACs
APb↑
Lat. ↓
Thrp. ↑
NID ↑
ResNet-50 [ 53 ]
44M
260G
38.2
32.8ms
34
0.87
ConvNeXt-T [ 54 ]
48M
262G
44.2
43.4ms
26
0.92
Swin-T [ 1 ]
48M
267G
42.7
45.1ms
24
0.89
ViT-Adapter-S [ 55 ]
48M
403G
44.7
138.7ms
7
0.93
MPViT-S [ 56 ]
43M
268G
46.4
–
–
1.08
PVTv2-B2 [ 57 ]
45M
–
45.3
86.7ms
7
1.01
TABLE I: Object detection on COCO with Mask R-CNN [ 40 ] . Latency and throughput are measured at a test resolution of 800×1280 . ( ↑ indicates higher is better, ↓ indicates lower is better.)
Fig. 5: Qualitative comparisons of results on COCO dataset.
Model
Params
MACs
mIoU ↑
Lat. ↓
Thrp. ↑
NID ↑
ResNet-50 [ 53 ]
67M
953G
42.1
78.3ms
13
0.63
ConvNeXt-T [ 54 ]
60M
939G
46.0
159.8ms
6
0.77
Swin-T [ 1 ]
60M
945G
44.4
89.0ms
12
0.74
MPViT-S [ 56 ]
52M
943G
48.3
153.2ms
9
0.93
ViT-Adapter-T [ 55 ]
36M
–
42.6
238.2ms
6
1.18
VMamba-T [ 3 ]
62M
949G
48.0
118.5ms
8
0.77
TABLE II: Semantic segmentation on ADE20K with UPerNet [ 41 ] . Latency and throughput are measured at a test resolution of 512×2048 . ( ↑ indicates higher is better, ↓ indicates lower is better.)
Fig. 6: Qualitative comparisons of results on ADE20K dataset.
Model
Params
MACs
Lat. ↓
Thrp. ↑
Abs Rel ↓
δ1↑
NID ↑
BTS [ 64 ]
66.5M
184.3G
32.9ms
35
0.061
0.954
1.43
AdaBins [ 65 ]
78.3M
261.7G
47.2ms
21
0.067
0.949
1.21
P3Depth [ 66 ]
94.3M
401.8G
36.1ms
34
0.071
0.953
1.01
DPT [ 67 ]
123.1M
319.3G
60.1ms
19
0.062
0.959
0.78
iDisc [ 38 ]
40.7M
256.2G
83.4ms
14
0.058
0.968
2.38
IEBins [ 37 ]
90.7M
527.3G
69.5ms
16
0.056
0.970
1.07
TABLE III: Comparison of quantitative results on KITTI Eigen-Split. ( ↑ indicates higher is better, ↓ indicates lower is better.)
Fig. 7: Qualitative comparisons of depth estimation results on KITTI dataset.
Model
Params
MACs
Lat. ↓
Thrp. ↑
Abs Rel ↓
δ1↑
NID ↑
BTS [ 64 ]
66.5M
132.3G
28.2ms
45
0.113
0.871
1.31
LocalBins [ 70 ]
73.6M
186.2G
32.9ms
41
0.099
0.907
1.23
GLPDepth [ 71 ]
62.0M
67.2G
32.6ms
52
0.098
0.915
1.48
AdaBins [ 65 ]
78.3M
187.8G
49.2ms
20
0.103
0.903
1.15
P3Depth [ 66 ]
94.3M
288.3G
35.8ms
35
0.104
0.904
0.96
iDisc [ 38 ]
39.0M
218.8G
53.4ms
25
0.109
0.894
2.29
TABLE IV: Comparison of quantitative results on NYU Depth v2 dataset. ( ↑ indicates higher is better, ↓ indicates lower is better.)
Fig. 8: Qualitative comparisons of depth estimation results on NYU dataset.
Dataset
Method
Config
Time(s)/G
Ratio(%)
NYUv2
Baseline
–
705.1
100.0
Baseline w/ PWs
–
250.0
35.5
DMMPE
(1,4)
236.4
33.5
(4,1)
181.7
25.8
(3,4)
178.2
25.3
(4,3)
172.3
24.4
TABLE V: Efficiency validation of the proposed parallel evaluation framework. PWs denotes Persistent Workers . Config is reported as ( Proc./GPU , Models/Proc. ).
Fig. 9: Evolutionary trajectories of EvoNAS on COCO, ADE20K, KITTI, and NYU v2. Top: 3D distribution of candidate architectures across generations in the accuracy–latency–MACs space. Bottom: normalized hypervolume curves showing the convergence behavior of the Pareto front over 200 generations.
Model
Params
Top-1
NAT-S [ 72 ] (CVPR’2023)
51M
83.7%
VMamba-B [ 3 ] (NeurIPS’2024)
89M
83.9%
MILA-T [ 73 ] (CVPR’2025)
25M
83.5%
GroupMamba-S [ 74 ] (CVPR’2025)
34M
83.9%
EvoNet
17M
84.3%
TABLE VI: Image classification results on ImageNet-1K.
Method
Abs Rel ↓
RMSE ↓
δ1↑
Target: EvoNet-N1
Arch. Pool [ 9 ]
0.118
0.403
0.872
Random Sampling
0.109
0.377
0.886
PST(Ours)
0.095
0.344
0.912
Target: EvoNet-N2
Arch. Pool [ 9 ]
0.096
0.343
0.909
TABLE VII: Ablation analysis of the supernet training strategy.
Model
Method
Abs Rel ↓
Log10 ↓
RMSE ↓
δ1↑
δ2↑
EvoNet-N1
w/o CA-DDKD
0.102
0.044
0.367
0.899
0.982
-
0.095
0.041
0.344
0.912
0.985
EvoNet-N2
w/o CA-DDKD
0.093
0.040
0.332
0.919
0.988
-
0.089
0.038
0.323
0.926
0.988
EvoNet-N3
w/o CA-DDKD
0.090
0.039
0.328
0.922
0.989
-
0.085
0.037
0.310
0.932
0.990
TABLE VIII: Fusion Experiment of Knowledge Distillation Strategy
Fig. 10: Comparison of architecture ranking consistency.
Decoders
MACs
Lat.
Thrp.
Abs Rel
δ1
NewCRFs [ 36 ]
32.09G
37.8ms
76
0.088
0.929
iDisc [ 38 ]
60.84G
38.1ms
53
0.091
0.922
Spatial-Mamba [ 59 ]
33.93G
30.4ms
90
0.085
0.932
TABLE IX: Ablation analysis of the decoder architecture.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Params
Lat.
Thrp.
pixelNeRF [ 76 ]
20.43
0.589
0.550
–
–
–
GPNR [ 77 ]
24.11
0.793
0.255
–
–
–
AttnRend [ 78 ]
24.78
0.820
0.213
–
–
–
MuRF [ 79 ]
26.10
0.858
0.143
–
–
–
pixelSplat [ 80 ]
25.89
0.858
0.142
119M
162ms
–
MVSplat [ 81 ]
26.39
0.869
0.128
12M
62ms
–
TABLE X: NVS on RealEstate10K. Latency and throughput are measured at a test resolution of 256×256 .
Fig. 11: Qualitative comparisons of results on RealEstate10K dataset.
Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves 93.189±0.467% CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately 1/24 of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by 0.577 percentage points while changing best-visited accuracy by only 0.054 points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with 69.615±1.139% and 43.720±1.697% accuracy. Applied without tuning to the larger DARTS space, LESS achieves 96.95±0.14% on CIFAR-10 and 82.43±0.80% on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.
Aviral Gandhi, Jinglue Xu, Jialong Li +1
The University of Tokyo · Sakana AI · Waseda University
We introduce JetViT, a novel family of hybrid-architecture Vision Transformer (ViT) models that match the accuracy of state-of-the-art full-attention vision foundation models while achieving substantially higher inference efficiency on high-resolution images. At the core of our approach is Post-Training Attention Search, a post-training acceleration framework that converts pre-trained full-attention ViTs into efficient hybrid-attention variants by identifying and replacing redundant full-attention blocks with linear or window-attention blocks. By inheriting the MLP and attention weights from the base model, Post-Training Attention Search efficiently explores the architectural design space through three key steps: (1) optimizing the linear-attention block design; (2) finding the best combination of linear-attention and window-attention blocks; and (3) identifying and preserving critical full-attention blocks. We evaluate JetViT on two representative high-resolution vision foundation models, DINOv3 and DepthAnythingV2. On the NVIDIA H100 GPU, JetViT achieves up to 1.79x higher throughput and up to 44.81% lower latency without sacrificing accuracy. We will release our code and accelerated ViT models soon.
Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources. We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures for image classification on a single consumer-grade GPU without LLM fine-tuning. Central to our approach is a historical feedback memory inspired by Markov chains: a sliding window of K=5 recent improvement attempts keeps context size constant while providing sufficient signal for iterative learning. Unlike prior LLM optimizers that discard failure trajectories, each history entry is a structured diagnostic triple -- recording the identified problem, suggested modification, and resulting outcome -- treating code execution failures as first-class learning signals. A dual-LLM specialization reduces per-call cognitive load: a Code Generator produces executable PyTorch architectures while a Prompt Improver handles diagnostic reasoning. Since both the LLM and architecture training share limited VRAM, the search implicitly favors compact, hardware-efficient models suited to edge deployment. We evaluate three frozen instruction-tuned LLMs (≤7B parameters) across up to 2000 iterations in an unconstrained open code space, using one-epoch proxy accuracy on CIFAR-10, CIFAR-100, and ImageNette as a fast ranking signal. On CIFAR-10, DeepSeek-Coder-6.7B improves from 28.2% to 69.2%, Qwen2.5-7B from 50.0% to 71.5%, and GLM-5 from 43.2% to 62.0%. A full 2000-iteration search completes in ≈18 GPU hours on a single RTX~4090, establishing a low-budget, reproducible, and hardware-aware paradigm for LLM-driven NAS without cloud infrastructure.
Xiaojie Gu, Dmitry Ignatov, Radu Timofte
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany