AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments

Rui Wang; Zishen Lai; Jiayu Liu; Cheng Qian; Jeonghwan Kim; Heng Wang; Xuejun Zhang; Jiateng Liu

cs.CLOct 19, 2026

AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments

Authors: Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu,

Organizations: HKUST · HKUST-UGAIL · HKUST(GZ) · UIUC ·

Abstract

Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.

Figures & tables

Benchmark / Work	Proactive Exploration	Adaptive Exploration	Env-learning Exploration	In-Process Evaluation	Heterogeneous Rules	Scalable Generation
InductionBench ( Hua et al., 2025 )	✗	✗	✗	✗	✓	✓
HERO’S JOURNEY ( Zheng et al., 2026a )	✗	✗	✗	✗	✓	✓
RULEARN ( He et al., 2025 )	✓	✗	✓	✓	✓	✗
NewtonBench ( Zheng et al., 2026b )	✓	✗	✓	✗	✓	✓
WILT ( Banatt et al., 2024 )	✓	✗	✓	✓	✓	✗
OdysseyArena ( Xu et al., 2025 )	✓	✗	✓	✓	✓	✓

Table 1: Positioning of AdaExploreBench. For each work, the table reports whether each trait is fully ( ✓ ), partially ( ✓ ) or not ( ✗ ) addressed. Detailed explanations of each terminology are provided in Appendix A .

Figure 1: Overview of AdaExploreBench. Data construction shows the generation pipeline. Runtime protocol shows how agents explore and how are they assessed. Example Trajectory shows a complete example of an episode.

Model Name	End-to-end Outcome		Exploration Behavior		Adaptation Performance
Model Name	EA (%) $\uparrow$	CA (%) $\uparrow$	Mean Uni. Probes	Mean Re-probes	Factor Rec. (%) $\uparrow$	Change Rec. (%) $\uparrow$
Llama-3.1-8B	30.67	8.13	4.77	0.24	52.54	6.30
Qwen3.5-9B	32.17	12.31	7.81	1.04	39.40	13.80
Qwen3.8-27B	45.95	37.73	42.20	5.46	65.87	25.93
Gemma-4-12B	37.98	20.59	12.52	0.13	53.21	10.34
DeepSeek-V4.1-Flash	58.01	52.87	113.42	16.04	78.07	44.52

Table 2: Main evaluation results in the default setting. Scores in bold indicate the best performance among all models, and underlined scores denote the second-best performance. Exploration Behavior metrics omit arrows and highlights, as neither direction is inherently better. Metrics marked with (%) are reported in percentage form.

Figure 2: Calibrated Accuracy under default and the three other settings. Boxed annotations give the average CA gain over Default across all models. The full results and significance analysis provided in Appendix I.2 and I.3 .

Figure 3: Calibrated Accuracy per evaluation batch under default , with batch 0 marking the last pre-change batch.

			Spearman $\rho$
Model	Re-probes	Change Rec.(%)	count	rate
Llama-3.1-8B	0.24	6.30	0.05	0.05
Qwen3.5-9B	1.04	13.80	0.01	0.03
Qwen3.8-27B	5.46	25.93	0.66	0.64
Gemma-4-12B	0.13	10.34	0.05	0.05
DeepSeek-V4.1-Flash	16.04	44.52	0.25	0.30

Table 3: Correlation between Mean Re-probes and Change Recognition on default , where rate normalizes re-probes by distinct pre-change probe number. Bold and underlined values indicate

p<0.001

and

p<0.01

	Factor Rec.		Change Rec.
Model	P	R	P	R
Llama-3.1-8B	38.2	84.5	10.5	6.5
Qwen3.5-9B	43.9	35.9	19.2	14.9
Qwen3.8-27B	81.1	55.6	66.9	23.1
Gemma-4-12B	52.8	53.5	18.6	10.5
DeepSeek-V4.1-Flash	86.9	70.8	77.8	39.3

Table 4: Precision and recall for Factor Rec. and Change Rec. on default . The larger value is bolded .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Motivation of AdaExploreBench. An agent proactively explore and get confused when rule changes.

Figure 5: Illustrative examples of the four rule families considered in this work. Given the same input vector, the additive rule computes a weighted sum, the conjunctive and disjunctive rules return the minimum and maximum threshold margins (

x_{i}-\tau_{i}

), respectively, and the lexicographic rule returns the margin of the first threshold-satisfying input in priority order.

Dimension	Mean $\pm$ SD	Pos. (%)	Maj. (%)
Scenario Plausibility	4.95 $\pm$ 0.22	100.0	100.0
Information Replacement	4.56 $\pm$ 0.61	94.2	100.0
Effect Reversal	4.09 $\pm$ 0.73	77.5	90.0

Appendix

Table 5: Human evaluation of generated task plausibility. Positive denotes the percentage of ratings

\geq 4

, while Majority denotes the percentage of cases rated

\geq 4

by at least two annotators.

Setting	Value
Episode layout
Action batches per episode	12
Cases per action batch	8
Candidate actions per case	4
Input fields per case	8
rule drift after batch	5, 6, or 7

Appendix

Table 6: Interaction and failure-handling configuration of the benchmark runtime. Every value is fixed across episodes, models, and settings. The fallbacks that apply once a limit is exhausted are described in the text.

Model Name	Effort	Thinking Budget	Max Output Tokens
Llama-3.1-8B	default	–	8192
Qwen3.5-9B	default	4096	8192
Qwen3.8-27B	medium	–	8192
Gemma-4-12B	high	4096	8192
DeepSeek-V4.1-Flash	high	–	32768
GPT-6-Luna	high	–	32768

Appendix

Table 7: Configuration snapshot of every reported run, frozen into the run directory at launch. A dash means the setting is unset.

	Pearson $r$			Spearman $\rho$
Metric	EA	CA	Avg.	EA	CA	Avg.
Mean Uni. Probes	0.912	0.916	0.914	0.881	0.905	0.893
Factor Rec.	0.913	0.922	0.917	0.929	0.952	0.940
Change Rec.	0.937	0.928	0.932	0.833	0.905	0.869

Appendix

Table 8: Pearson

r

and Spearman

\rho

of Mean Unique Probes and the two recognition metrics with the end-to-end outcomes, computed across the eight model means. Bold marks the strongest correlation in each column.

	Stationary		Stationary-IID
Model	EA	CA	EA	CA
DeepSeek-V4.1-Flash	69.19	67.60	68.85	67.39
Qwen3.8-27B	54.74	50.57	55.60	51.37
GPT-6-Luna	50.26	41.14	50.82	42.23
Gemma-4-12B	39.37	22.78	40.32	24.86
Qwen3.5-9B	32.15	12.41	33.29	13.86

Appendix

Table 9: Performance results comparing stationary and stationary setting with IID sampling .

Model	Factor Replacement	Direction Reversal
Llama-3.1-8B	6.9	5.9
Qwen3.5-9B	21.1	2.4
Qwen3.8-27B	31.7	6.0
Gemma-4-12B	14.7	2.0
DeepSeek-V4.1-Flash	49.2	19.6
GPT-6-Luna	13.1	1.6

Appendix

Table 10: Change Recognition decomposed by atom type on default . The larger of the two rates is in bold. The replacement rate exceeds the reversal rate at

p<10^{-70}

under a paired Wilcoxon signed-rank test.

	End-to-end Outcome		Exploration Behavior
Model Name	EA (%) $\uparrow$	CA (%) $\uparrow$	Mean Uni. Probes
Llama-3.1-8B	30.90	8.74	4.53
Qwen3.5-9B	32.15	12.41	6.76
Qwen3.8-27B	54.74	50.57	33.31
Gemma-4-12B	39.37	22.78	12.28
DeepSeek-V4.1-Flash	69.19	67.60	85.36

Appendix

Table 11: Evaluation results under the Stationary control, where the rule never changes. Mean Re-probes and Adaptation Performance are undefined in this setting. Gemini-3.8-Flash has no diagnostic runs and is omitted.

	Pre-change (%)				Post-change (%)
Model	Exact	Omit only	Over-report only	Mixed	Exact	Omit only	Over-report only	Mixed	$n$ (pre/post)
Llama-3.1-8B	0.4	0.0	60.0	39.6	0.0	0.4	77.0	22.7	255/256
Qwen3.5-9B	0.0	32.4	12.3	55.3	0.0	29.7	10.5	59.8	253/256
Qwen3.8-27B	29.1	65.7	1.6	3.5	2.4	44.0	4.0	49.6	254/252
Gemma-4-12B	10.9	48.8	18.0	22.3	0.4	17.3	19.2	63.1	256/255
DeepSeek-V4.1-Flash	46.1	49.6	2.0	2.3	18.0	30.2	9.4	42.4	256/255

Appendix

Table 12: Factor Recognition decomposed into per-assessment outcomes on the dynamic branch of Default, reported as shares of completed assessments before and after rule drift. Each assessment falls into exactly one of the four columns.

n

lists the completed assessments in each phase, and the pooled row aggregates assessments across models.

Model	Both	Replacement only	Reversal only	Neither	Unchanged report	$n$
Llama-3.1-8B	0.0	2.7	5.9	91.4	33.3	255
Qwen3.5-9B	0.0	11.9	2.4	85.8	29.6	253
Qwen3.8-27B	2.4	22.3	3.6	71.7	54.2	251
Gemma-4-12B	0.0	4.3	2.0	93.7	44.3	255
DeepSeek-V4.1-Flash	14.5	19.2	5.1	61.2	29.4	255
GPT-6-Luna	0.0	5.1	1.6	93.4	65.6	256

Appendix

Table 13: Episode-level decomposition of Change Recognition outcomes on the dynamic branch of Default, reported as shares of eligible episodes. Each episode falls into exactly one of the first four columns, while unchanged report overlaps with the neither column.

n

counts eligible episodes, and the pooled row aggregates episodes across models.

	End-to-end Outcome		Exploration Behavior		Adaptation Performance
Model Name	EA (%) $\uparrow$	CA (%) $\uparrow$	Mean Uni. Probes	Mean Re-probes	Factor Rec. (%) $\uparrow$	Change Rec. (%) $\uparrow$
Llama-3.1-8B	34.13	13.78	4.87	0.41	52.55	5.32
Qwen3.5-9B	43.30	28.18	4.72	0.29	40.91	15.39
Qwen3.8-27B	35.73	33.78	17.36	0.08	62.65	13.85
Gemma-4-12B	48.10	36.52	7.00	0.00	50.34	12.10
DeepSeek-V4.1-Flash	71.51	70.34	119.28	6.38	79.92	43.65

Appendix

Table 14: Evaluation results under the Evidence Given setting.

	End-to-end Outcome		Exploration Behavior		Adaptation Performance
Model Name	EA (%) $\uparrow$	CA (%) $\uparrow$	Mean Uni. Probes	Mean Re-probes	Factor Rec. (%) $\uparrow$	Change Rec. (%) $\uparrow$
Llama-3.1-8B	27.83	4.07	19.81	2.05	51.10	7.74
Qwen3.5-9B	33.59	14.32	17.46	4.41	38.66	12.76
Qwen3.8-27B	47.98	38.69	58.09	8.01	66.34	38.70
Gemma-4-12B	38.96	22.02	26.25	5.37	52.10	9.59
DeepSeek-V4.1-Flash	57.98	53.27	133.05	18.68	78.54	48.91

Appendix

Table 15: Evaluation results under the Forced-Probe setting.

	Stationary vs. Default		Evidence Given vs. Default		Forced-Probe vs. Default
Model Name	$\Delta$ CA	$p$	$\Delta$ CA	$p$	$\Delta$ CA	$p$
Llama-3.1-8B	$+0.62$	$0.294$	$+5.65$	$<0.001^{***}$	$-4.06$	$<0.001^{***}$
Qwen3.5-9B	$+0.10$	$0.893$	$+15.87$	$<0.001^{***}$	$+2.01$	$0.066$
Qwen3.8-27B	$+12.85$	$<0.001^{***}$	$-3.94$	$0.007^{**}$	$+0.96$	$0.618$
Gemma-4-12B	$+2.19$	$0.003^{**}$	$+15.93$	$<0.001^{***}$	$+1.43$	$0.132$
DeepSeek-V4.1-Flash	$+14.74$	$<0.001^{***}$	$+17.47$	$<0.001^{***}$	$+0.40$	$0.711$

Appendix

Table 16: Paired Wilcoxon signed-rank tests on per-episode Calibrated Accuracy between each diagnostic setting and Default, over 256 paired episodes per model.

\Delta

CA is the mean difference in percentage points, positive when the diagnostic setting scores higher. The Mean row averages the per-model deltas and tests them at the model level. All seven models with diagnostic runs are reported, while the main text figure shows six of them for space.

{}^{*}\,p<0.05

{}^{**}\,p<0.01

{}^{***}\,p<0.001

LLM Agent Evaluation

Large language model agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently.

See all 1,194 papers

Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents
Aug 14, 2026Zhizhao Guan, Chen Huang, Ziming Liu +5RL Exploration LLM Agents
Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence · National University of Singapore +1
Look Before You Leap: Autonomous Exploration for LLM Agents
May 15, 2026Ziang Ye, Wentao Shi, Yuxin Liu +6RL Exploration LLM Agents
University of Science and Technology of China · Meituan
EvoProbe: Evaluation-Driven Benchmark Evolution for LLM Agents
Oct 14, 2026Qianhong Guo, Wei Xie, Xiaofang Cai +3LLM Agent Evaluation AI Agent Benchmarks
College of Computer Science and Technology, National University of Defense Technology · Institute of Information Engineering, Chinese Academy of Sciences

Aug 14, 2026J/K move · Enter open · S save

Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents

Zhizhao Guan, Chen Huang, Ziming Liu, Hongru Liang+4

Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence · National University of Singapore+1

RL Exploration LLM Agents LLM Agent Training RL for LLM Agents Proactive Agents

Open paper AskPDF arXiv

We study proactive exploration in LLM agents, i.e., the ability to explore an environment to acquire information that improves future decision-making. In this regard, we first identify two fundamental bottlenecks that hinder this capability and then propose \ours, a novel method designed to instill and refine proactive exploration. Specifically, \ours\ consists of two components: (1) Exploratory Data Construction, which synthesizes exploration-rich trajectories to mitigate the hindsight bias of standard demonstrations; and (2) RL Optimization with Contrastive Signal Guidance, which leverages contrastive trajectory pairs to distinguish productive exploration from redundant wandering. Extensive experiments demonstrate the effectiveness of \ours\ and provide insights into the characteristics of proactive exploration. Our code is available at: https://github.com/GuanZhizhao/SAFARI.

Abstract

LLM Agent Evaluation

Explore similar work

Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents

Look Before You Leap: Autonomous Exploration for LLM Agents

EvoProbe: Evaluation-Driven Benchmark Evolution for LLM Agents