Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.
Figures & tables
Benchmark / Work
Proactive Exploration
Adaptive Exploration
Env-learning Exploration
In-Process Evaluation
Heterogeneous Rules
Scalable Generation
InductionBench ( Hua et al., 2025 )
✗
✗
✗
✗
✓
✓
HERO’S JOURNEY ( Zheng et al., 2026a )
✗
✗
✗
✗
✓
✓
RULEARN ( He et al., 2025 )
✓
✗
✓
✓
✓
✗
NewtonBench ( Zheng et al., 2026b )
✓
✗
✓
✗
✓
✓
WILT ( Banatt et al., 2024 )
✓
✗
✓
✓
✓
✗
OdysseyArena ( Xu et al., 2025 )
✓
✗
✓
✓
✓
✓
Table 1: Positioning of AdaExploreBench. For each work, the table reports whether each trait is fully ( ✓ ), partially ( ✓ ) or not ( ✗ ) addressed. Detailed explanations of each terminology are provided in Appendix A .
Figure 1: Overview of AdaExploreBench. Data construction shows the generation pipeline. Runtime protocol shows how agents explore and how are they assessed. Example Trajectory shows a complete example of an episode.
Model Name
End-to-end Outcome
Exploration Behavior
Adaptation Performance
EA (%) ↑
CA (%) ↑
Mean Uni. Probes
Mean Re-probes
Factor Rec. (%) ↑
Change Rec. (%) ↑
Llama-3.1-8B
30.67
8.13
4.77
0.24
52.54
6.30
Qwen3.5-9B
32.17
12.31
7.81
1.04
39.40
13.80
Qwen3.8-27B
45.95
37.73
42.20
5.46
65.87
25.93
Gemma-4-12B
37.98
20.59
12.52
0.13
53.21
10.34
DeepSeek-V4.1-Flash
58.01
52.87
113.42
16.04
78.07
44.52
Table 2: Main evaluation results in the default setting. Scores in bold indicate the best performance among all models, and underlined scores denote the second-best performance. Exploration Behavior metrics omit arrows and highlights, as neither direction is inherently better. Metrics marked with (%) are reported in percentage form.
Figure 2: Calibrated Accuracy under default and the three other settings. Boxed annotations give the average CA gain over Default across all models. The full results and significance analysis provided in Appendix I.2 and I.3 .
Figure 3: Calibrated Accuracy per evaluation batch under default , with batch 0 marking the last pre-change batch.
Spearman ρ
Model
Re-probes
Change Rec.(%)
count
rate
Llama-3.1-8B
0.24
6.30
0.05
0.05
Qwen3.5-9B
1.04
13.80
0.01
0.03
Qwen3.8-27B
5.46
25.93
0.66
0.64
Gemma-4-12B
0.13
10.34
0.05
0.05
DeepSeek-V4.1-Flash
16.04
44.52
0.25
0.30
Table 3: Correlation between Mean Re-probes and Change Recognition on default , where rate normalizes re-probes by distinct pre-change probe number. Bold and underlined values indicate p<0.001 and p<0.01 .
Factor Rec.
Change Rec.
Model
P
R
P
R
Llama-3.1-8B
38.2
84.5
10.5
6.5
Qwen3.5-9B
43.9
35.9
19.2
14.9
Qwen3.8-27B
81.1
55.6
66.9
23.1
Gemma-4-12B
52.8
53.5
18.6
10.5
DeepSeek-V4.1-Flash
86.9
70.8
77.8
39.3
Table 4: Precision and recall for Factor Rec. and Change Rec. on default . The larger value is bolded .
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Motivation of AdaExploreBench. An agent proactively explore and get confused when rule changes.
Figure 5: Illustrative examples of the four rule families considered in this work. Given the same input vector, the additive rule computes a weighted sum, the conjunctive and disjunctive rules return the minimum and maximum threshold margins ( xi−τi ), respectively, and the lexicographic rule returns the margin of the first threshold-satisfying input in priority order.
Dimension
Mean ± SD
Pos. (%)
Maj. (%)
Scenario Plausibility
4.95 ± 0.22
100.0
100.0
Information Replacement
4.56 ± 0.61
94.2
100.0
Effect Reversal
4.09 ± 0.73
77.5
90.0
Appendix
Table 5: Human evaluation of generated task plausibility. Positive denotes the percentage of ratings ≥4 , while Majority denotes the percentage of cases rated ≥4 by at least two annotators.
Setting
Value
Episode layout
Action batches per episode
12
Cases per action batch
8
Candidate actions per case
4
Input fields per case
8
rule drift after batch
5, 6, or 7
Appendix
Table 6: Interaction and failure-handling configuration of the benchmark runtime. Every value is fixed across episodes, models, and settings. The fallbacks that apply once a limit is exhausted are described in the text.
Model Name
Effort
Thinking Budget
Max Output Tokens
Llama-3.1-8B
default
–
8192
Qwen3.5-9B
default
4096
8192
Qwen3.8-27B
medium
–
8192
Gemma-4-12B
high
4096
8192
DeepSeek-V4.1-Flash
high
–
32768
GPT-6-Luna
high
–
32768
Appendix
Table 7: Configuration snapshot of every reported run, frozen into the run directory at launch. A dash means the setting is unset.
Pearson r
Spearman ρ
Metric
EA
CA
Avg.
EA
CA
Avg.
Mean Uni. Probes
0.912
0.916
0.914
0.881
0.905
0.893
Factor Rec.
0.913
0.922
0.917
0.929
0.952
0.940
Change Rec.
0.937
0.928
0.932
0.833
0.905
0.869
Appendix
Table 8: Pearson r and Spearman ρ of Mean Unique Probes and the two recognition metrics with the end-to-end outcomes, computed across the eight model means. Bold marks the strongest correlation in each column.
Stationary
Stationary-IID
Model
EA
CA
EA
CA
DeepSeek-V4.1-Flash
69.19
67.60
68.85
67.39
Qwen3.8-27B
54.74
50.57
55.60
51.37
GPT-6-Luna
50.26
41.14
50.82
42.23
Gemma-4-12B
39.37
22.78
40.32
24.86
Qwen3.5-9B
32.15
12.41
33.29
13.86
Appendix
Table 9: Performance results comparing stationary and stationary setting with IID sampling .
Model
Factor Replacement
Direction Reversal
Llama-3.1-8B
6.9
5.9
Qwen3.5-9B
21.1
2.4
Qwen3.8-27B
31.7
6.0
Gemma-4-12B
14.7
2.0
DeepSeek-V4.1-Flash
49.2
19.6
GPT-6-Luna
13.1
1.6
Appendix
Table 10: Change Recognition decomposed by atom type on default . The larger of the two rates is in bold. The replacement rate exceeds the reversal rate at p<10−70 under a paired Wilcoxon signed-rank test.
End-to-end Outcome
Exploration Behavior
Model Name
EA (%) ↑
CA (%) ↑
Mean Uni. Probes
Llama-3.1-8B
30.90
8.74
4.53
Qwen3.5-9B
32.15
12.41
6.76
Qwen3.8-27B
54.74
50.57
33.31
Gemma-4-12B
39.37
22.78
12.28
DeepSeek-V4.1-Flash
69.19
67.60
85.36
Appendix
Table 11: Evaluation results under the Stationary control, where the rule never changes. Mean Re-probes and Adaptation Performance are undefined in this setting. Gemini-3.8-Flash has no diagnostic runs and is omitted.
Pre-change (%)
Post-change (%)
Model
Exact
Omit only
Over-report only
Mixed
Exact
Omit only
Over-report only
Mixed
n (pre/post)
Llama-3.1-8B
0.4
0.0
60.0
39.6
0.0
0.4
77.0
22.7
255/256
Qwen3.5-9B
0.0
32.4
12.3
55.3
0.0
29.7
10.5
59.8
253/256
Qwen3.8-27B
29.1
65.7
1.6
3.5
2.4
44.0
4.0
49.6
254/252
Gemma-4-12B
10.9
48.8
18.0
22.3
0.4
17.3
19.2
63.1
256/255
DeepSeek-V4.1-Flash
46.1
49.6
2.0
2.3
18.0
30.2
9.4
42.4
256/255
Appendix
Table 12: Factor Recognition decomposed into per-assessment outcomes on the dynamic branch of Default, reported as shares of completed assessments before and after rule drift. Each assessment falls into exactly one of the four columns. n lists the completed assessments in each phase, and the pooled row aggregates assessments across models.
Model
Both
Replacement only
Reversal only
Neither
Unchanged report
n
Llama-3.1-8B
0.0
2.7
5.9
91.4
33.3
255
Qwen3.5-9B
0.0
11.9
2.4
85.8
29.6
253
Qwen3.8-27B
2.4
22.3
3.6
71.7
54.2
251
Gemma-4-12B
0.0
4.3
2.0
93.7
44.3
255
DeepSeek-V4.1-Flash
14.5
19.2
5.1
61.2
29.4
255
GPT-6-Luna
0.0
5.1
1.6
93.4
65.6
256
Appendix
Table 13: Episode-level decomposition of Change Recognition outcomes on the dynamic branch of Default, reported as shares of eligible episodes. Each episode falls into exactly one of the first four columns, while unchanged report overlaps with the neither column. n counts eligible episodes, and the pooled row aggregates episodes across models.
End-to-end Outcome
Exploration Behavior
Adaptation Performance
Model Name
EA (%) ↑
CA (%) ↑
Mean Uni. Probes
Mean Re-probes
Factor Rec. (%) ↑
Change Rec. (%) ↑
Llama-3.1-8B
34.13
13.78
4.87
0.41
52.55
5.32
Qwen3.5-9B
43.30
28.18
4.72
0.29
40.91
15.39
Qwen3.8-27B
35.73
33.78
17.36
0.08
62.65
13.85
Gemma-4-12B
48.10
36.52
7.00
0.00
50.34
12.10
DeepSeek-V4.1-Flash
71.51
70.34
119.28
6.38
79.92
43.65
Appendix
Table 14: Evaluation results under the Evidence Given setting.
End-to-end Outcome
Exploration Behavior
Adaptation Performance
Model Name
EA (%) ↑
CA (%) ↑
Mean Uni. Probes
Mean Re-probes
Factor Rec. (%) ↑
Change Rec. (%) ↑
Llama-3.1-8B
27.83
4.07
19.81
2.05
51.10
7.74
Qwen3.5-9B
33.59
14.32
17.46
4.41
38.66
12.76
Qwen3.8-27B
47.98
38.69
58.09
8.01
66.34
38.70
Gemma-4-12B
38.96
22.02
26.25
5.37
52.10
9.59
DeepSeek-V4.1-Flash
57.98
53.27
133.05
18.68
78.54
48.91
Appendix
Table 15: Evaluation results under the Forced-Probe setting.
Stationary vs. Default
Evidence Given vs. Default
Forced-Probe vs. Default
Model Name
Δ CA
p
Δ CA
p
Δ CA
p
Llama-3.1-8B
+0.62
0.294
+5.65
<0.001∗∗∗
−4.06
<0.001∗∗∗
Qwen3.5-9B
+0.10
0.893
+15.87
<0.001∗∗∗
+2.01
0.066
Qwen3.8-27B
+12.85
<0.001∗∗∗
−3.94
0.007∗∗
+0.96
0.618
Gemma-4-12B
+2.19
0.003∗∗
+15.93
<0.001∗∗∗
+1.43
0.132
DeepSeek-V4.1-Flash
+14.74
<0.001∗∗∗
+17.47
<0.001∗∗∗
+0.40
0.711
Appendix
Table 16: Paired Wilcoxon signed-rank tests on per-episode Calibrated Accuracy between each diagnostic setting and Default, over 256 paired episodes per model. Δ CA is the mean difference in percentage points, positive when the diagnostic setting scores higher. The Mean row averages the per-model deltas and tests them at the model level. All seven models with diagnostic runs are reported, while the main text figure shows six of them for space. ∗p<0.05 , ∗∗p<0.01 , ∗∗∗p<0.001 .
We study proactive exploration in LLM agents, i.e., the ability to explore an environment to acquire information that improves future decision-making. In this regard, we first identify two fundamental bottlenecks that hinder this capability and then propose \ours, a novel method designed to instill and refine proactive exploration. Specifically, \ours\ consists of two components: (1) Exploratory Data Construction, which synthesizes exploration-rich trajectories to mitigate the hindsight bias of standard demonstrations; and (2) RL Optimization with Contrastive Signal Guidance, which leverages contrastive trajectory pairs to distinguish productive exploration from redundant wandering. Extensive experiments demonstrate the effectiveness of \ours\ and provide insights into the characteristics of proactive exploration. Our code is available at: https://github.com/GuanZhizhao/SAFARI.
Zhizhao Guan, Chen Huang, Ziming Liu +5
Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence · National University of Singapore +1
Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents.
Ziang Ye, Wentao Shi, Yuxin Liu +6
University of Science and Technology of China · Meituan
Static benchmarks for large language model (LLM) agents face saturation and data contamination risks, motivating continual updates to evaluation tasks. However, aggregate scores and success rates offer limited insight into specific problem-solving bottlenecks and provide limited guidance for subsequent task design. We propose EvoProbe, an evaluation-driven benchmark evolution framework that integrates task generation, diagnostic evaluation, and feedback-driven regeneration into a closed loop. It combines verifiable subtask outcome scoring with evidence-supported multi-judge trajectory evaluation and distills phase-level bottlenecks, success patterns, and failure patterns into diagnostic feedback for generating new executable tasks. Across two rounds of cybersecurity experiments, evaluations of eight models on 66 tasks reveal phase-level performance differences and common failure patterns beyond aggregate scores. In a cross-round comparison with the Examiner model set and solver configuration held fixed, all eight models achieve lower Outcome scores on the updated tasks, with relative decreases of 11.8% in mean Outcome and 40.0% in mean Pass\textasciicircum{}3. These results support using fine-grained diagnostics to guide iterative task updates and probe agents' current capabilities and limitations.
Qianhong Guo, Wei Xie, Xiaofang Cai +3
College of Computer Science and Technology, National University of Defense Technology · Institute of Information Engineering, Chinese Academy of Sciences