cs.AIMay 18, 2025

NeuronGen: Neural Network Parameter Generation via Large Language Models

Authors: Jiaqi Wang, Yusen Zhang, Xi Li

Organizations: The Pennsylvania State University · Auburn University · University of Alabama at Birmingham

Abstract

Acquiring the parameters of neural networks (NNs) has been one of the most important problems in machine learning since the inception of NNs. Traditional approaches, such as backpropagation and forward-only optimization, acquire parameters via iterative data fitting to gradually optimize them. This paper aims to explore the feasibility of a new direction: acquiring NN parameters via large language model generation. We propose NeuronGen, a generalized and easy-to-implement two-stage approach for NN parameter generation conditioned on descriptions of the data, task, and network architecture. Stage one is Parameter Reference Knowledge Injection, where LLMs are pretrained on NN checkpoints to build foundational understanding of parameter space, whereas stage two is Context-Enhanced Instruction Tuning, enabling LLMs to adapt to specific tasks through enriched, task-aware prompts. Experimental results demonstrate that NeuronGen effectively generates usable NN parameters. Our findings highlight the feasibility of LLM-based NN parameter generation and suggest a promising new paradigm where LLMs and lightweight NNs can coexist synergistically

Figures & tables

Explore similar work

CardsList
  1. Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

    Sep 16, 2026Jinli Hu, Ross M. Clarke, Yichuan Zhang +1Continual Learning for LLMsHypernetworks

  2. Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs

    May 6, 2026Santosh Premi Adhikari, Radu Timofte, Dmitry IgnatovLarge Language Model-Guided NASLLM Fine-Tuning

  3. Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts

    May 24, 2026Xiangtian Ji, Yuxin Chen, Zhengzhou Cai +3Fine-TuningLLM Interpretability