Dual-Domain Representation Alignment: Bridging 2D and 3D Vision via Geometry-Aware Architecture Search

Haoyu Zhang; Zhihao Yu; Rui Wang; Yaochu Jin; Qiqi Liu; Ran Cheng

cs.CVMar 20, 2026

Dual-Domain Representation Alignment: Bridging 2D and 3D Vision via Geometry-Aware Architecture Search

Authors: Haoyu Zhang, Zhihao Yu, Rui Wang, Yaochu Jin, Qiqi Liu, Ran Cheng

Organizations: Hangzhou Normal University, Hangzhou, P.R. China · Trustworthy and General AI Lab, School of Engineering, Westlake University, Hangzhou, P.R. China · Department of Data Science and Artificial Intelligence, and the Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, China

Abstract

Modern computer vision requires balancing predictive accuracy with real-time efficiency, yet the high inference cost of large vision models (LVMs) limits deployment on resource-constrained edge devices. Although Evolutionary Neural Architecture Search (ENAS) is well suited for multi-objective optimization, its practical use is hindered by two issues: expensive candidate evaluation and ranking inconsistency among subnetworks. To address them, we propose EvoNAS, an efficient distributed framework for multi-objective evolutionary architecture search. We build a hybrid supernet that integrates Vision State Space and Vision Transformer (VSS-ViT) modules, and optimize it with a Cross-Architecture Dual-Domain Knowledge Distillation (CA-DDKD) strategy. By coupling the computational efficiency of VSS blocks with the semantic expressiveness of ViT modules, CA-DDKD improves the representational capacity of the shared supernet and enhances ranking consistency, enabling reliable fitness estimation during evolution without extra fine-tuning. To reduce the cost of large-scale validation, we further introduce a Distributed Multi-Model Parallel Evaluation (DMMPE) framework based on GPU resource pooling and asynchronous scheduling. Compared with conventional data-parallel evaluation, DMMPE improves efficiency by over 70% through concurrent multi-GPU, multi-model execution. Experiments on COCO, ADE20K, KITTI, and NYU-Depth v2 show that the searched architectures, termed EvoNets, consistently achieve Pareto-optimal trade-offs between accuracy and efficiency. Compared with representative CNN-, ViT-, and Mamba-based models, EvoNets deliver lower inference latency and higher throughput under strict computational budgets while maintaining strong generalization on downstream tasks such as novel view synthesis. Code is available at https://github.com/EMI-Group/evonas

Figures & tables

Fig. 1: The EvoNAS unified visual architecture, featuring a searchable hybrid VSS-ViT encoder, a Spatial Mamba decoder, and optional task heads for detection, segmentation, and MDE. Searchable components within the Evo-Mamba and Evo-Transformer blocks define the encoder’s supernet search space, enabling flexible and general-purpose architecture optimization across diverse vision tasks.

Fig. 2: Overview of the two-stage supernet optimization with dual-domain knowledge distillation. Stage 1 pretrains the VSS-ViT supernet encoder on ImageNet-1K to establish a stable initialization. Stage 2 adapts the supernet to the target task by aligning student VSS/ViT features with teacher ViT features through spatial- and frequency-domain distillation, while jointly applying prediction-level and ground-truth supervision.

Fig. 3: Overview of the EvoNAS pipeline. The framework integrates distributed multi-model parallel evaluation with multi-objective evolutionary search, enabling efficient validation of candidate architectures across GPUs and progressive approximation of the Pareto frontier in terms of accuracy, latency, and computational complexity.

Fig. 4: Performance–efficiency trade-offs of EvoNAS across four benchmarks: (a) COCO, (b) ADE20K, (c) KITTI, and (d) NYU v2. The x-axis denotes MACs, the y-axis represents task-specific accuracy metrics (mAP, mIoU, or AbsRel), bubble size indicates parameter count, and color encodes inference latency.

Model	Params	MACs	$\mathrm{AP}^{b}\uparrow$	Lat. $\downarrow$	Thrp. $\uparrow$	NID $\uparrow$
ResNet-50 [ 53 ]	44M	260G	38.2	32.8ms	34	0.87
ConvNeXt-T [ 54 ]	48M	262G	44.2	43.4ms	26	0.92
Swin-T [ 1 ]	48M	267G	42.7	45.1ms	24	0.89
ViT-Adapter-S [ 55 ]	48M	403G	44.7	138.7ms	7	0.93
MPViT-S [ 56 ]	43M	268G	46.4	–	–	1.08
PVTv2-B2 [ 57 ]	45M	–	45.3	86.7ms	7	1.01

TABLE I: Object detection on COCO with Mask R-CNN [ 40 ] . Latency and throughput are measured at a test resolution of

800\times 1280

. (

\uparrow

indicates higher is better,

\downarrow

indicates lower is better.)

Fig. 5: Qualitative comparisons of results on COCO dataset.

Model	Params	MACs	mIoU $\uparrow$	Lat. $\downarrow$	Thrp. $\uparrow$	NID $\uparrow$
ResNet-50 [ 53 ]	67M	953G	42.1	78.3ms	13	0.63
ConvNeXt-T [ 54 ]	60M	939G	46.0	159.8ms	6	0.77
Swin-T [ 1 ]	60M	945G	44.4	89.0ms	12	0.74
MPViT-S [ 56 ]	52M	943G	48.3	153.2ms	9	0.93
ViT-Adapter-T [ 55 ]	36M	–	42.6	238.2ms	6	1.18
VMamba-T [ 3 ]	62M	949G	48.0	118.5ms	8	0.77

TABLE II: Semantic segmentation on ADE20K with UPerNet [ 41 ] . Latency and throughput are measured at a test resolution of

512\times 2048

. (

\uparrow

indicates higher is better,

\downarrow

indicates lower is better.)

Fig. 6: Qualitative comparisons of results on ADE20K dataset.

Model	Params	MACs	Lat. $\downarrow$	Thrp. $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	NID $\uparrow$
BTS [ 64 ]	66.5M	184.3G	32.9ms	35	0.061	0.954	1.43
AdaBins [ 65 ]	78.3M	261.7G	47.2ms	21	0.067	0.949	1.21
P3Depth [ 66 ]	94.3M	401.8G	36.1ms	34	0.071	0.953	1.01
DPT [ 67 ]	123.1M	319.3G	60.1ms	19	0.062	0.959	0.78
iDisc [ 38 ]	40.7M	256.2G	83.4ms	14	0.058	0.968	2.38
IEBins [ 37 ]	90.7M	527.3G	69.5ms	16	0.056	0.970	1.07

TABLE III: Comparison of quantitative results on KITTI Eigen-Split. (

\uparrow

indicates higher is better,

\downarrow

indicates lower is better.)

Fig. 7: Qualitative comparisons of depth estimation results on KITTI dataset.

Model	Params	MACs	Lat. $\downarrow$	Thrp. $\uparrow$	Abs Rel $\downarrow$	$\delta_{1}\uparrow$	NID $\uparrow$
BTS [ 64 ]	66.5M	132.3G	28.2ms	45	0.113	0.871	1.31
LocalBins [ 70 ]	73.6M	186.2G	32.9ms	41	0.099	0.907	1.23
GLPDepth [ 71 ]	62.0M	67.2G	32.6ms	52	0.098	0.915	1.48
AdaBins [ 65 ]	78.3M	187.8G	49.2ms	20	0.103	0.903	1.15
P3Depth [ 66 ]	94.3M	288.3G	35.8ms	35	0.104	0.904	0.96
iDisc [ 38 ]	39.0M	218.8G	53.4ms	25	0.109	0.894	2.29

TABLE IV: Comparison of quantitative results on NYU Depth v2 dataset. (

\uparrow

indicates higher is better,

\downarrow

indicates lower is better.)

Fig. 8: Qualitative comparisons of depth estimation results on NYU dataset.

Dataset	Method	Config	$\mathrm{Time(s)}/G$	Ratio(%)
NYUv2	Baseline	–	705.1	100.0
	Baseline w/ PWs	–	250.0	35.5
	DMMPE	(1,4)	236.4	33.5
		(4,1)	181.7	25.8
		(3,4)	178.2	25.3
		(4,3)	172.3	24.4

TABLE V: Efficiency validation of the proposed parallel evaluation framework. PWs denotes Persistent Workers . Config is reported as ( Proc./GPU , Models/Proc. ).

Fig. 9: Evolutionary trajectories of EvoNAS on COCO, ADE20K, KITTI, and NYU v2. Top: 3D distribution of candidate architectures across generations in the accuracy–latency–MACs space. Bottom: normalized hypervolume curves showing the convergence behavior of the Pareto front over 200 generations.

Model	Params	Top-1
NAT-S [ 72 ] (CVPR’2023)	51M	83.7%
VMamba-B [ 3 ] (NeurIPS’2024)	89M	83.9%
MILA-T [ 73 ] (CVPR’2025)	25M	83.5%
GroupMamba-S [ 74 ] (CVPR’2025)	34M	83.9%
EvoNet	17M	84.3%

TABLE VI: Image classification results on ImageNet-1K.

Method	Abs Rel $\downarrow$	RMSE $\downarrow$	$\delta_{1}\uparrow$
Target: EvoNet-N1
Arch. Pool [ 9 ]	0.118	0.403	0.872
Random Sampling	0.109	0.377	0.886
PST(Ours)	0.095	0.344	0.912
Target: EvoNet-N2
Arch. Pool [ 9 ]	0.096	0.343	0.909

TABLE VII: Ablation analysis of the supernet training strategy.

Model	Method	Abs Rel $\downarrow$	Log10 $\downarrow$	RMSE $\downarrow$	$\delta_{1}\uparrow$	$\delta_{2}\uparrow$
EvoNet-N1	w/o CA-DDKD	0.102	0.044	0.367	0.899	0.982
EvoNet-N1	-	0.095	0.041	0.344	0.912	0.985
EvoNet-N2	w/o CA-DDKD	0.093	0.040	0.332	0.919	0.988
EvoNet-N2	-	0.089	0.038	0.323	0.926	0.988
EvoNet-N3	w/o CA-DDKD	0.090	0.039	0.328	0.922	0.989
EvoNet-N3	-	0.085	0.037	0.310	0.932	0.990

TABLE VIII: Fusion Experiment of Knowledge Distillation Strategy

Fig. 10: Comparison of architecture ranking consistency.

Decoders	MACs	Lat.	Thrp.	Abs Rel	$\delta_{1}$
NewCRFs [ 36 ]	32.09G	37.8ms	76	0.088	0.929
iDisc [ 38 ]	60.84G	38.1ms	53	0.091	0.922
Spatial-Mamba [ 59 ]	33.93G	30.4ms	90	0.085	0.932

TABLE IX: Ablation analysis of the decoder architecture.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params	Lat.	Thrp.
pixelNeRF [ 76 ]	20.43	0.589	0.550	–	–	–
GPNR [ 77 ]	24.11	0.793	0.255	–	–	–
AttnRend [ 78 ]	24.78	0.820	0.213	–	–	–
MuRF [ 79 ]	26.10	0.858	0.143	–	–	–
pixelSplat [ 80 ]	25.89	0.858	0.142	119M	162ms	–
MVSplat [ 81 ]	26.39	0.869	0.128	12M	62ms	–

TABLE X: NVS on RealEstate10K. Latency and throughput are measured at a test resolution of

256\times 256

Fig. 11: Qualitative comparisons of results on RealEstate10K dataset.

Neural Architecture Search

Neural Architecture Search automates neural network design for target tasks, resource budgets, and hardware constraints.

See all 132 papers

LESS: Lightweight Evolutionary Supernet Search in Minutes
Oct 1, 2026Aviral Gandhi, Jinglue Xu, Jialong Li +1Evolutionary Optimization Covariance Matrix Adaptation Evolution Strategy
The University of Tokyo · Sakana AI · Waseda University
JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
May 26, 2026Dongyun Zou, Zhuoyang Zhang, Junyu Chen +8Efficient Transformer Inference Vision Foundation Models
MIT · NVIDIA · University of Pennsylvania +1
Resource-Efficient Iterative LLM-Based NAS with Feedback Memory
Mar 12, 2026Xiaojie Gu, Dmitry Ignatov, Radu TimofteLarge Language Model-Guided NAS Neural Architecture Search
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany

Oct 1, 2026J/K move · Enter open · S save

LESS: Lightweight Evolutionary Supernet Search in Minutes

Aviral Gandhi, Jinglue Xu, Jialong Li, Hitoshi Iba

The University of Tokyo · Sakana AI · Waseda University

Evolutionary Optimization Covariance Matrix Adaptation Evolution Strategy Neural Architecture Search Neuroevolution

Open paper AskPDF arXiv

Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves

93.189\pm0.467\%

CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately

1/24

of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by

0.577

percentage points while changing best-visited accuracy by only

0.054

points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with

69.615\pm1.139\%

and

43.720\pm1.697\%

accuracy. Applied without tuning to the larger DARTS space, LESS achieves

96.95\pm0.14\%

on CIFAR-10 and

82.43\pm0.80\%

on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.

Abstract

Neural Architecture Search

Explore similar work

LESS: Lightweight Evolutionary Supernet Search in Minutes

JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search

Resource-Efficient Iterative LLM-Based NAS with Feedback Memory