cs.LGFeb 6, 2026

MIDAS: Mosaic Input-Specific Differentiable Architecture Search

Authors: Konstanty Subbotko

Organizations: Independent Researcher

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, an approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models the two-edge node topology during search. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it discovers optimal or near-optimal architectures. In RDARTS, it achieves the best reported performance among the compared methods on two of four search spaces on CIFAR-10. We further analyze why MIDAS works, showing that patchwise attention improves discrimination among candidate operations and that the resulting input-specific parameter distributions are class-aware.

Figures & tables

Figure 1: Computing input-specific architecture with attention. For a given node, each candidate operation o(j)o^{(j)} applied to an incoming feature x(i)x^{(i)} produces an activation map F(i,j)=o(j)(x(i))F^{(i,j)}=o^{(j)}(x^{(i)}) . We project the node’s concatenated input into a query and the candidate activation maps into keys, and apply dot-product attention to obtain architecture weights. In the mosaic variant, we partition each activation map into P2P^{2} patches, compute attention independently within each patch, and average the patch-level distributions to obtain an image-level architecture. For clarity, the topology-aware search space is not illustrated.
MethodsParams (M)Top1 (%)Top5 (%)
NASNet-A ( Zoph et al., 2018 )5.374.091.6
SNAS ( Xie et al., 2019 )4.372.790.8
DARTS ( Liu et al., 2019 )4.773.391.3
P-DARTS ( Chen et al., 2019 )5.175.392.5
SDARTS-ADV ( Chen and Hsieh, 2020 )5.474.892.2
DOTS ( Gu et al., 2021 )5.275.792.6
Table 3: ImageNet results when transferring cells discovered on CIFAR-10.
Figure 2: Learned input-specific architecture parameters in the first two cells in the DARTS search space on CIFAR-10, averaged over four runs. We compare three variants: no patch (global average pooling only), PS=4 (patch size 4×44\times 4 ), and PS=8 (patch size 8×88\times 8 ). The horizontal line denotes uniform importance across operations. We observe that no patch fails to discriminate among learnable operations, essentially assigning similar weights to all four learnable operations.
Figure 3: (Left) Histogram of pp -values from Hartigan’s dip test across all architecture parameters. The vertical line marks the 0.050.05 rejection threshold. Most pp -values exceed 0.050.05 , so unimodality is not rejected for the majority. (Right) Distribution of an example input-specific parameter, with the vertical line marking the mean value.
Figure 4: Cosine similarity between CIFAR-10 classes derived from input-specific architecture parameters (higher is more similar). Best viewed in color.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Standard deviation of input-specific architecture parameters p(i,j)p^{(i,j)} , averaged over candidate edges, nodes, and four seeds.
Figure 6: Number of non-learnable operations in the decoded normal cell during search (100 epochs).
Figure 7: Architecture found by MIDAS on CIFAR-10.
Figure 8: Architecture found by MIDAS on CIFAR-100.
More in this topic

Neural Architecture Search

Neural Architecture Search automates neural network design for target tasks, resource budgets, and hardware constraints.

See all 132 papers

Explore similar work

CardsList

  1. LP-NAS: Linear Programming-based Neural Architecture Search

    Aug 14, 2026Abhishek Shukla, Ankur Sinha, Faiz HamidNeural Architecture Search

    Department of Management Sciences, IIT Kanpur, India · Krishnamurthy Tandon School of AI, IIM Ahmedabad, India

  2. G-ICSO-NAS: Shifting Gears between Gradient and Swarm for Robust Neural Architecture Search

    Apr 1, 2026Xingbang Du, Enzhi Zhang, Rui Zhong +2Neural Architecture SearchParticle Swarm Optimization

    Information Systems Design Laboratory, Hokkaido University, Sapporo, Japan · Information Initiative Center, Hokkaido University, Sapporo, Japan

  3. Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search

    May 19, 2026Yuiko Sakuma, Masakazu Yoshimura, Marcel Gröpl +4Large Language Model-Guided NASLarge Language Model-Guided Optimization

    Sony Group Corporation, Tokyo, Japan · ETH Zurich, Switzerland