Planning to Explore: Curiosity-Driven Planning for LLM Test Generation

Alfonso Amayuelas; Firas Laakom; Piotr Piękos; Wenyi Wang; Yifan Xu; Yuhui Wang; Jürgen Schmidhuber; William Wang

cs.SEApr 6, 2026

Planning to Explore: Curiosity-Driven Planning for LLM Test Generation

Authors: Alfonso Amayuelas, Firas Laakom, Piotr Piękos, Wenyi Wang, Yifan Xu, Yuhui Wang, Jürgen Schmidhuber, William Wang

Organizations: University of California, Santa Barbara · King Abdullah University of Science and Technology

Abstract

The use of LLMs for code generation has naturally extended to code testing and evaluation. As codebases grow in size and complexity, so does the need for automated test generation. Current approaches for LLM-based test generation rely on strategies that maximize immediate coverage gain, a greedy approach that plateaus on code where reaching deep branches requires setup steps that individually yield zero new coverage. Drawing on principles of Bayesian exploration, we treat the program's branch structure as an unknown environment, and an evolving coverage map as a proxy probabilistic posterior representing what the LLM has discovered so far. Our method, CovQValue, feeds the coverage map back to the LLM, generates diverse candidate plans in parallel, and selects the most informative plan by LLM-estimated Q-values, seeking actions that balance immediate branch discovery with future reachability. Our method outperforms greedy selection on TestGenEval Lite, achieving 51-77% higher branch coverage across three popular LLMs and winning on 77-84% of targets. In addition, we build a benchmark for iterative test generation, RepoExploreBench, where they achieve 40-74%. These results show the potential of curiosity-driven planning methods for LLM-based exploration, enabling more effective discovery of program behavior through sequential interaction

Figures & tables

Definitions	Symbol	Test Generation
Unknown environment	$\Theta$	Program’s branch reachability structure
Prior	$p(\Theta)$	No branches observed
History	$h=(a_{1},o_{1})\cdots(a_{t},o_{t})$	Sequence of (test, coverage result) pairs
Exploration state	$p(\Theta\|h)$	Coverage map $\mathcal{C}_{t}$ (sufficient statistic after $t$ executions)
Action	$a$	Test plan (sequence of $S$ scripts)
Observation	$o$	Branches hit by executing the plan

Table 1: Formal mapping from Sun et al. (2011) optimal Bayesian exploration framework to LLM-based test generation.

Figure 1: Overview of CovQvalue . At each round, the LLM generates $K$ diverse candidate plans conditioned on the coverage map. Each plan is scored by its estimated Q-value, information gain plus discounted future reachability (Equation 2 ), and the highest scoring plan is executed. The resulting branch coverage updates the coverage map for next round.

	RepoExploreBench			TestGenEval Lite
	Gemini 3	GPT-5.4	Mistral	Gemini 3	GPT-5.4	Mistral
	Flash	mini	Large 3	Flash	mini	Large 3
Random	43.2 $\pm$ 4.5	31.2 $\pm$ 3.0	41.2 $\pm$ 4.2	64.4 $\pm$ 6.3	50.4 $\pm$ 4.6	65.9 $\pm$ 5.9
Greedy	41.8 $\pm$ 3.9	28.6 $\pm$ 2.8	42.3 $\pm$ 4.4	61.5 $\pm$ 6.0	53.2 $\pm$ 4.8	65.0 $\pm$ 5.8
CovGreedy	51.7 $\pm$ 5.7	36.5 $\pm$ 3.5	40.0 $\pm$ 4.5	77.6 $\pm$ 7.3	66.0 $\pm$ 6.2	74.7 $\pm$ 7.1
CovQValue	72.4 $\pm$ 6.5	49.8 $\pm$ 5.0	59.1 $\pm$ 6.0	108.9 $\pm$ 9.0	80.5 $\pm$ 7.2	100.0 $\pm$ 8.4

Table 2: Mean per-module branch coverage (

\pm

standard error) for all 3 models. CovQValue vs. Greedy comparisons:

p<0.0001

(paired

t

-test).

d

: Cohen’s

d

effect size.

Panel of Figure 2 — Figure 2: Cumulative branch coverage over execution steps, averaged across three models (Gemini 3 Flash, GPT-5.4 Mini, Mistral Large). CovQValue (ours) continues discovering branches while Random and Greedy stall earlier. Shaded region indicates $\pm$ 1 SE.

Figure 3: Mean branch coverage by repository on TestGenEval Lite, averaged across three models. CovQValue outperforms other methods on every repository, with the largest gains over Greedy on sympy (+55%), matplotlib (+52%), and astropy (+55%).

Figure 4: Trade-off between test pass rate and branch coverage. Large dots represent the mean across models, while individual models and benchmark results are represent with transparent dots. Coverage-map strategies discover more branches by generating larger tests at a small cost in pass rate.

Panel of Figure 5 — Figure 5: Ablation studies on RepoExploreBench (93 targets, Gemini 3 Flash). (a) CovQValue scales steadily with budget. (b) Longer plans yield higher coverage, confirming the benefit of multi-step lookahead. (c) Top: discount factor sensitivity (default in bold). Bottom: component contributions.

Figure 6: Corridor navigation on four RepoExploreBench modules (Gemini 3 Flash). It showcases how Random or Greedy remain flat while CovQValue traverses setup corridors to reach deep branches as the number of steps increase.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Package	Domain	Mod.	Lines	Corridor Structure
click	CLI framework	10	8,293	Command setup → option parsing → type coercion
flask	Web framework	10	5,369	App creation → config → routing → request context
httpx	Async HTTP client	10	6,925	Client config → auth → transport → URL parsing
jinja2	Template engine	12	12,751	Env setup → lexer → parser → AST → compiler
pydantic	Data validation	10	11,727	Model definition → schema → validation → serialization
requests	HTTP client	6	4,375	Session setup → auth → adapters → connection pool

Appendix

Table 3: RepoExploreBench composition. Packages selected from top 150 PyPI downloads (as of March 2026). Corridor structure describes the sequential dependencies that gate access to deep branches.

RepoExploreBench
	Gemini 3 Flash		GPT-5.4 Mini		Mistral Large
Strategy	Lines	Pass%	Lines	Pass%	Lines	Pass%
Random	146	84%	113	95%	142	79%
Greedy	142	83%	109	95%	144	80%
CovGreedy	157	57%	123	86%	129	48%
CovQValue	197	71%	151	87%	171	67%

Appendix

Table 4: Line coverage and pass rate. Results consistent with branch coverage.

Figure 7: Mean branch coverage by repository on RepoExploreBench, averaged for all models. CovQValue outperforms all baselines on every repository.

	Gemini Flash				GPT-5.4 Mini				Mistral Large 3
Repo	Rnd	Gdy	CG	CQ	Rnd	Gdy	CG	CQ	Rnd	Gdy	CG	CQ
astropy	37	23	49	80	13	18	19	51	35	35	47	76
django	69	71	83	114	61	65	73	90	81	74	76	109
matplotlib	93	93	101	133	53	68	81	103	86	89	106	119
seaborn	10	20	10	10	9	9	10	10	8	8	9	9
flask	10	18	23	23	19	17	21	25	15	18	8	21

Appendix

Table 5: Per-repository branch coverage on TestGenEval Lite by model. Each cell averages across modules in that repo. Mean row shows the per-module average. Best per group in bold.

	Gemini Flash				GPT-5.4 Mini				Mistral Large 3
Repo	Rnd	Gdy	CG	CQ	Rnd	Gdy	CG	CQ	Rnd	Gdy	CG	CQ
click	38	28	32	68	23	24	27	29	29	27	26	51
flask	19	24	31	53	28	18	28	39	20	20	17	37
httpx	37	38	45	67	33	33	39	54	35	33	44	54
jinja2	51	46	57	84	29	30	32	56	43	50	40	61
pydantic	70	58	83	88	52	48	58	80	72	66	76	87

Appendix

Table 6: Per-repository branch coverage on RepoExploreBench by model. Each cell averages across modules in that repo. Mean row shows the per-module average Best per group in bold.

Automated Test Generation

Automated test generation creates inputs, scenarios, and checks to expose software and system defects with less manual effort.

See all 131 papers

FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback
May 2, 2026Kushal Jasti, Tejamani Prashanth Sahu, Rishitha Pentyala +2Multi-Agent LLM Systems Automated Software Testing
Department of Computer Science and Engineering, SRM University AP, India
Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics
Apr 20, 2026Khang Tran, Khoa Nguyen, Cristian Borcea +1LLM-Based Program Synthesis Automated Test Generation
New Jersey Institute of Technology, Newark, NJ, USA
Evaluating LLM-Based Test Generation Under Software Evolution
Mar 24, 2026Sabaat Haroon, Mohammad Taha Khan, Muhammad Ali GulzarAutomated Test Generation Automated Software Testing
Virginia Tech, USA · Carnegie Mellon University, USA

May 2, 2026J/K move · Enter open · S save

FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback

Kushal Jasti, Tejamani Prashanth Sahu, Rishitha Pentyala, Muvvala Mohit+1

Department of Computer Science and Engineering, SRM University AP, India

Multi-Agent LLM Systems Automated Software Testing Automated Test Generation

Open paper AskPDF arXiv

Traditional approaches to test case generation often involve manual effort and incur significant computational overhead. Additionally, these approaches are not scalable, and hence, unsuitable for complex software systems. Recently, Large Language Models (LLMs) have been applied to software testing. However, single-shot prompt engineering-based approaches tend to hallucinate and generate redundant test cases, resulting in fewer branches. To handle the above-mentioned limitations, in this paper, we propose FeedbackLLM, a novel automated language-agnostic test case generation framework based on tightly coupled two-stage approach. In the first stage, FeedbackLLM extracts the input constraints by parsing source code and generates the possible test cases. The quality of the test cases is evaluated in the second stage by the following two specialized LLM feedback agents: (i) Line Feedback Agent: extracts the metadata related to missed line executions and (ii) Branch Feedback Agent: extracts the metadata of the unexecuted branch conditions. The above agents operate in a two-stage process, communicating in tandem, and this procedure is repeated for k-steps. Further, we also introduced a redundancy prevention cache to avoid duplicate API requests and avoid unnecessary execution cycles. The performance of the proposed architecture is evaluated on the standard benchmark programs related to C and Python programs. FeedbackLLM demonstrated more line and branch coverage than baseline tools while scaling linearly in execution time.

Abstract

Automated Test Generation

Explore similar work

FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback

Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics

Evaluating LLM-Based Test Generation Under Software Evolution