cs.LGMar 12, 2026

Resource-Efficient Iterative LLM-Based NAS with Feedback Memory

Authors: Xiaojie Gu, Dmitry Ignatov, Radu Timofte

Organizations: Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources. We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures for image classification on a single consumer-grade GPU without LLM fine-tuning. Central to our approach is a historical feedback memory inspired by Markov chains: a sliding window of K=5K{=}5 recent improvement attempts keeps context size constant while providing sufficient signal for iterative learning. Unlike prior LLM optimizers that discard failure trajectories, each history entry is a structured diagnostic triple -- recording the identified problem, suggested modification, and resulting outcome -- treating code execution failures as first-class learning signals. A dual-LLM specialization reduces per-call cognitive load: a Code Generator produces executable PyTorch architectures while a Prompt Improver handles diagnostic reasoning. Since both the LLM and architecture training share limited VRAM, the search implicitly favors compact, hardware-efficient models suited to edge deployment. We evaluate three frozen instruction-tuned LLMs (≤7{\leq}7B parameters) across up to 2000 iterations in an unconstrained open code space, using one-epoch proxy accuracy on CIFAR-10, CIFAR-100, and ImageNette as a fast ranking signal. On CIFAR-10, DeepSeek-Coder-6.7B improves from 28.2% to 69.2%, Qwen2.5-7B from 50.0% to 71.5%, and GLM-5 from 43.2% to 62.0%. A full 2000-iteration search completes in ≈18{\approx}18 GPU hours on a single RTX~4090, establishing a low-budget, reproducible, and hardware-aware paradigm for LLM-driven NAS without cloud infrastructure.

Figures & tables

Figure 1: Overview of the iterative NAS pipeline. The Code Generator produces a candidate architecture as executable PyTorch code. The Evaluator validates and trains it using one-epoch proxy evaluation. The Prompt Improver analyzes results with historical feedback memory to generate targeted improvement suggestions for the next iteration.
Figure 2: One-epoch proxy accuracy on CIFAR-10 (top row, a–c), CIFAR-100 (middle row, d–f), and ImageNette (bottom row, g–i) across all iterations. Light curves show per-iteration accuracy (the accuracy of iterations with errors fall back to previous value), dashed lines show the smoothed trend (window w=15w{=}15 ), and bold lines show the best-so-far trajectory. All models exhibit clear upward trends. For DeepSeek-Coder on ImageNette, only the first 30 iterations are plotted because all subsequent iterations resulted in errors.
MethodNAS targetUnconstrained search spaceAutomated failure diagnosisSingle-GPU ( ≤\leq 24 GB)No LLM fine-tuning
OPRO [ 33 ]×\times×\times×\times×\times✓\checkmark
FunSearch [ 25 ]×\times✓\checkmark×\times×\times✓\checkmark
EvoPrompting [ 3 ]✓\checkmark✓\checkmark×\times✓\checkmark×\times
ReEvo [ 34 ]×\times✓\checkmark✓\checkmark×\times✓\checkmark
LLMO [ 38 ]✓\checkmark×\times×\times×\times✓\checkmark
Ours✓\checkmark✓\checkmark✓\checkmark✓\checkmark✓\checkmark
Table 1: Systematic comparison with most similar LLM-based NAS and optimization methods. NAS target: method aims at neural architecture search; Unconstrained search space: search over executable code rather than predefined cell encodings or fixed discrete spaces; Automated failure diagnosis: structured modeling of code execution failures (e.g., diagnostic triples) as first-class feedback signals; Single-GPU ( ≤\leq 24 GB): demonstrated effective with ≤\leq 7B LLMs on a single consumer GPU; No LLM fine-tuning: operates with frozen pretrained LLMs without any parameter updates. Our method uniquely combines all five dimensions.
MetricDeepS.QwenGLM
Total iters20002000100
Succ. evals151937691
Succ. rate76.0%18.8%91.0%
1st acc.28.2%50.0%43.2%
Best acc.69.2%71.5%62.0%
Improve.+41.0%+21.5%+18.7%
Table 2: Comparison of the iterative NAS pipeline across three LLMs on CIFAR-10, CIFAR-100, and ImageNette datasets. The models exhibit statistically significant improvement over single-shot generation. On ImageNette, correlation metrics for DeepSeek are not meaningful due to insufficient successful evaluations. On CIFAR-100, Qwen2.5 shows weaker correlation due to extended error loops reducing effective search diversity.
Figure 3: Ablation study of DeepSeek-Coder-6.7B-Instruct on CIFAR-10, CIFAR-100, and ImageNette datasets. The results highlight the effectiveness of the complete iterative loop with historical feedback memory compared to its ablated variants.
More in this topic

Large Language Model-Guided NAS

Large language model-guided neural architecture search uses language models to generate and optimize neural network designs.

See all 24 papers

Explore similar work

CardsList
Apr 17, 2026

LLM as a Tool, Not an Agent: Code-Mined Tree Transformations for Neural Architecture Search

Neural Architecture Search (NAS) aims to automatically discover high-performing deep neural network (DNN) architectures. However, conventional algorithm-driven NAS relies on carefully hand-crafted search spaces to ensure executability, which restricts open-ended exploration. Recent coding-based agentic approaches using large language models (LLMs) reduce manual design, but current LLMs struggle to reliably generate complex, valid architectures, and their proposals are often biased toward a narrow set of patterns observed in their training data. To bridge reliable algorithmic search with powerful LLM assistance, we propose LLMasTool, a hierarchical tree-based NAS framework for stable and open-ended model evolution. Our method automatically extracts reusable modules from arbitrary source code and represents full architectures as hierarchical trees, enabling evolution through reliable tree transformations rather than code generation. At each evolution step, coarse-level planning is governed by a diversity-guided algorithm that leverages Bayesian modeling to improve exploration efficiency, while the LLM resolves the remaining degrees of freedom to ensure a meaningful evolutionary trajectory and an executable generated architecture. With this formulation, instead of fully agentic LLM approaches, our method explores diverse directions beyond the inherent biases in the LLM. Our method improves over existing NAS methods by 0.69, 1.83, and 2.68 points on CIFAR-10, CIFAR-100, and ImageNet16-120, demonstrating its effectiveness.

Masakazu Yoshimura, Zitang Sun, Yuiko Sakuma +3

Sony Group Corporation, Tokyo, Japan

Large Language Model-Guided NASLarge Language Model-Guided OptimizationNeural Architecture Search
AskPDF
May 19, 2026

Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search

Current neural architecture search (NAS) methods are often limited by their predefined, restrictive search spaces. While recent large language model (LLM)-assisted NAS methods enable open-ended search spaces, they often suffer from inefficient exploration due to biased or low-quality design ideas. To address these issues, we propose to semi-automatically structure model design knowledge to guide the search process. Our approach first defines a high-level structural template of architectural attributes. An LLM then populates this template by analyzing papers, creating a rich and diverse search space that embodies this structured design knowledge. To efficiently explore this vast space, we introduce FairNAD, using a multi-type mutation that enables broad exploration through mutation with fair idea sampling, Pareto-aware mutation, LLM-driven iterative mutation, and a fine-grained feedback loop. We demonstrate the effectiveness of FairNAD in discovering high-performing architectures that yield 0.84, 2.17, and 2.35 points improvement on CIFAR-10, CIFAR-100, and ImageNet16-120, respectively, compared to current state-of-the-art methods.

Yuiko Sakuma, Masakazu Yoshimura, Marcel Gröpl +4

Sony Group Corporation, Tokyo, Japan · ETH Zurich, Switzerland

Large Language Model-Guided NASLarge Language Model-Guided OptimizationNeural Architecture Search
AskPDF
Apr 10, 2026

Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Search

This paper focuses on a key challenge in Neural Architecture Search (NAS): integrating established architectural knowledge while exploring new designs under expensive evaluations. Large language models (LLMs) are a promising assistant for NAS because they can translate rich architectural and coding priors into executable code edits. However, in practice, seemingly local revisions often propagate into non-local behavioral and performance shifts because a single edit can inadvertently couple multiple interacting functional factors, a phenomenon we refer to as functional entanglement. To make LLM knowledge usable under such entanglement, we propose Structured Progressive Knowledge Activation (SPARK), which activates relevant priors by explicitly selecting the functional factor to modify and conditioning the edit on that factor. This factor-conditioned editing reduces entangled side effects and yields more targeted, reliable architecture modifications. On CLRS-DFS, SPARK achieves a 28.1x sample-efficient architecture evolution speedup and yields a 22.9% relative improvement in OOD accuracy. Our code is available at https://github.com/AIM-ResearchLab/SPARK.

Zhen Liu, Yuhan Liu, Jinjun Wang +3

State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University. · Zhongguancun Academy, Beijing, China. · MiLM Plus, Xiaomi Inc. +1

Large Language Model-Guided NASLarge Language Model-Guided OptimizationNeural Architecture Search
AskPDF