Agentic Discovery with Active Hypothesis Exploration for Visual Recognition

Jaywon Koo; Jefferson Hernandez; Ruozhen He; Hanjie Chen; Chen Wei; Vicente Ordonez

cs.CVApr 14, 2026

Agentic Discovery with Active Hypothesis Exploration for Visual Recognition

Authors: Jaywon Koo, Jefferson Hernandez, Ruozhen He, Hanjie Chen, Chen Wei, Vicente Ordonez

Organizations: Rice University

Abstract

We introduce HypoExplore, an agentic framework that formulates neural architecture discovery for visual recognition as a hypothesis-driven scientific inquiry. Given a human-specified high-level research direction, HypoExplore ideates, implements, evaluates, and improves neural architectures through evolutionary branching. New hypotheses are created using a large language model by selecting a parent hypothesis to build upon, guided by a dual strategy that balances exploiting validated principles with resolving uncertain ones. Our proposed framework maintains a Trajectory Tree that records the lineage of all proposed architectures, and a Hypothesis Memory Bank that actively tracks confidence scores acquired through experimental evidence. After each experiment, multiple feedback agents analyze the results from different perspectives and consolidate their findings into hypothesis confidence updates. Our framework is tested on discovering lightweight vision architectures on CIFAR-10, with the best achieving 94.11% accuracy evolved from a root node baseline that starts at 18.91%, and generalizes to CIFAR-100 and Tiny-ImageNet. We further demonstrate applicability to a specialized domain by conducting independent architecture discovery runs on MedMNIST, which yield a state-of-the-art performance. We show that hypothesis confidence scores grow increasingly predictive as evidence accumulates, and that the learned principles transfer across independent evolutionary lineages, suggesting that HypoExplore not only discovers stronger architectures, but can help build a genuine understanding of the design space.

Figures & tables

Figure 1: High-level Overview of HypoExplore. Starting from a research direction, HypoExplore initializes a discovery state with a Trajectory Tree Memory and Hypothesis Memory Bank (Step 0 $\rightarrow$ Step 1). At each subsequent step, the current discovery state selects a parent node and hypothesis to guide the Research Cycle, producing an updated discovery state with enriched memory (Step t $\rightarrow$ Step t+1).

Figure 2: HypoExplore finds a lightweight Global Shape Token Network (GSTN) that introduces a small bank of learned global vectors. This network using less parameters closely matches or surpasses other manually engineered networks.

Figure 3: Overview of Per-Node Research Cycle. The Idea Agent proposes a neural architecture, which the Coding Agent implements with iterative hyperparameter tuning. A Redundancy Filtering Agent checks against the Tree Memory to prevent re-generation of concepts already explored. The Executor trains and evaluates each architecture, and the results are analyzed by four specialized Feedback Agents (right), each providing a distinct analytical perspective. The Hypothesis Synthesis Agent consolidates these multi-perspective analyses to update the Hypothesis Memory Bank, while experimental trajectories are stored in the Tree Memory to guide future exploration.

Panel of Figure 4 — Figure 4: HypoExplore discovers high-performing architectures via hypothesis-guided evolutionary branching. The trajectory tree (left) records the full lineage of architectural experiments across 5 root branches, where node color indicates accuracy. While most branches improve gradually, one branch achieves a decisive accuracy leap by iteratively applying accumulated hypotheses, evolving from a poorly-performing root architecture to ultimately discover the best-performing design (right).

Figure 5: Accumulated best accuracy over 50 iterations on CIFAR-10. All methods share the same five root architectures (iteration 0, best 81.2%). Left: Component ablation. Each curve removes one contribution from the full system. Right: Parent selection strategy comparison. All methods have hypothesis memory, hypothesis selection, and multi-agent feedback, and only the parent selection mechanism varies.

Model	Params (M)	CIFAR-10		CIFAR-100		Tiny-ImageNet
		Acc@1	Acc@5	Acc@1	Acc@5	Acc@1	Acc@5
ResNet-18	11.7	95.4	99.8	78.5	94.1	69.3	84.8
MobileNet V3	2.5	95.5	99.9	73.0	92.5	58.5	83.4
ShuffleNet V2	1.4	90.1	99.4	67.5	85.2	50.9	73.4
SqueezeNet	1.2	91.1	99.7	67.3	83.8	54.7	77.1
GSTN (ours)	0.9	94.1	99.6	72.6	91.7	58.1	81.7

Table 1: We report results on CIFAR-100 and Tiny-ImageNet for GSTN which was discovered on CIFAR-10. Results from manually engineered models of similar and larger capacity are included for reference.

Method	Dermal MNIST	Tissue MNIST	Breast MNIST
ResNet-18 [ 18 , 34 ]	75.4	68.1	83.3
ResNet-50 [ 18 , 34 ]	73.1	68.3	82.8
ViT [ 14 , 11 ]	73.9	–	–
Swin [ 31 , 11 ]	75.3	–	–
MedViTV1-L [ 33 ]	77.3	68.3	88.5
MedMamba-B [ 61 ]	75.7	–	89.1

Table 2: Domain-specific architecture discovery on MedMNIST classification. We report top-1 accuracy (%) on DermalMNIST, TissueMNIST, and BreastMNIST. “–” indicates the result is not reported. Best in bold and second-best underlined .

Figure 6: Analysis of hypothesis memory over 50 iterations on CIFAR-10. Left: Hypothesis prediction accuracy. Higher-confidence hypotheses predict experimental outcomes more accurately. Right: Knowledge accumulation curve. Best accuracy achieved and validated hypothesis count grows together.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Resolution	$32\times 32$ (native)
Normalisation	$\mu=(0.4914,0.4822,0.4465)$ , $\sigma=(0.2470,0.2435,0.2616)$
Augmentation	Random horizontal flip (50%), random translation ( $\pm 4$ px, reflect pad), Cutout ( $12\times 12$ )
Data loader	GPU-resident (entire dataset in VRAM); all augmentation on-GPU; FP16, channels-last
Optimiser	SGD (lr ${}_{\text{default}}$ = 0.1, momentum = 0.9, weight decay = $5{\times}10^{-4}$ , Nesterov)
LR schedule	Cosine annealing with linear warmup (5 epochs)

Appendix

Table 3: Default CIFAR-10 training configuration. All values may be overridden by the Coding Agent via the generated config.py . Additionally, new hyperparameters can be added by the Coding Agent.

Setting	Value
Datasets	DermaMNIST (7 cls)
Resolution	$224\times 224$ (aligned with the MedViTV2 SOTA protocol)
Channel conversion	Grayscale $\to$ RGB ( as_rgb=True )
Normalisation	$\mu=(0.5,0.5,0.5)$ , $\sigma=(0.5,0.5,0.5)$
Augmentation	RandomResizedCrop(224) , AugMix (sev.=3, width=3, $\alpha=0.4$ ), RandomHorizontalFlip ( $p=0.4$ )
Test/val transform	Resize(224) + normalisation only

Appendix

Table 4: Default MedMNIST training configuration. All values may be overridden by the Coding Agent via the generated config.py . Additionally, new hyperparameters can be added by the Coding Agent.

Figure 8: HyporExplore’s run using Gemini-3.1-pro

Neural Architecture Search

Neural Architecture Search automates neural network design for target tasks, resource budgets, and hardware constraints.

See all 132 papers

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery
Sep 24, 2026Lishan Yu, Derek Jiu, Qizhen Lan +1Large Language Model-Guided NAS Evolutionary Optimization
Department of Health Data Science and AI McWilliams School of Biomedical Informatics University of Texas Health Science Center at Houston
Neural Architecture Discovery via Autonomous Evolution
Jul 24, 2025Weixian Xu, Yixiu Liu, Yang Nan +4Autonomous Research Agents AI Agents for Scientific Discovery
Shanghai Jiao Tong University · SII · GAIR +1
Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study
Aug 3, 2026Aon Safdar, Mohamed SaadeldinAI Agents for Scientific Discovery Neural Architecture Search
University College Dublin, Ireland

Sep 24, 2026J/K move · Enter open · S save

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

Lishan Yu, Derek Jiu, Qizhen Lan, Xiaoqian Jiang

Department of Health Data Science and AI McWilliams School of Biomedical Informatics University of Texas Health Science Center at Houston

Large Language Model-Guided NAS Evolutionary Optimization Neural Architecture Search

Open paper AskPDF arXiv

AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this process through iterative generation and evaluation. Yet these iterations alone do not ensure cumulative progress or establish which directions to pursue next. Costly evaluation further constrains the scope of exploration. Neural architecture discovery brings these challenges together, coupling open-ended design with resource-intensive experimentation. We introduce EvoTreeNAD, a genealogy-guided evolutionary algorithm that constructs trainable architectures without a supplied seed or a hand-specified search space. Starting from an empty root, it grows a persistent genealogy in which each new node represents a complete architecture. Top-percentile values computed from each node and its descendants guide lineage selection. Using the selected design history, an Idea Agent proposes a variant and a Code Agent implements it. Each evaluated variant becomes a child node, expanding the genealogy while providing evidence for subsequent lineage selection. Our theoretical analysis establishes the existence of stationary variation regimes as the genealogy grows. Under specified variation assumptions, sustained top-percentile family values quantify the probability of generating high-reward architectures in these regimes. EvoTreeNAD discovers architectures that outperform the compared NAS and NAD baselines, achieving CIFAR-10/100 test errors of

2.05{\pm}0.06\%

and

15.09{\pm}0.22\%

. On all six MedMNIST-v2 tasks, the discovered architectures surpass the strongest listed baselines. A controlled CIFAR-10 study further shows that EvoTreeNAD outperforms direct generation, best-of-

N

greedy continuation, and full-family-mean routing.

Abstract

Neural Architecture Search

Explore similar work

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

Neural Architecture Discovery via Autonomous Evolution

Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study