cs.CRNov 17, 2026

Stopping Is Not the Limit: Understanding and Eliciting Agent Capability in Long-Horizon Vulnerability Exploitation

Authors: Deokhyung Kang, Minjae Gwon, Sangdon Park

Organizations: Safe AI Research Center, UNIST, Republic of Korea · Department of Computer Science and Engineering, POSTECH, Republic of Korea · Graduate School of Artificial Intelligence, POSTECH, Republic of Korea

Abstract

Vulnerability exploitation tests whether language model agents can develop working exploits from known software flaws. Sequential test-time scaling can improve exploitation performance, but larger budgets do not ensure sustained exploration: agents often stop with budget remaining. Yet what happens when execution continues beyond stopping remains underexplored. We investigate this gap on ExploitBench using DeepSeek-V4-Flash-0731 and MiMo-V2.6-Pro. We find additional verified gains after the first stop response, but gains diminish over longer execution. We identify two patterns associated with less productive continuation: an increasing proportion of stop responses within the same context and earlier stopping associated with summaries retaining impossibility judgments about unachieved capabilities. To test whether context management can elicit further capability, we propose SCoRe, which compacts context at stop bursts and marks these judgments as unproven. SCoRe improves verified capability over summary-based continuation and other evaluated context-management baselines. With SCoRe, MiMo reaches a capability coverage score of 67.0% at 2,000 turns, exceeding both 32.4% under the original stopping policy and the 58.6% computed for GPT-5.5 from ExploitBench's 300-turn reference results on the same tasks. These findings show that stopping need not mark an agent's exploitation limit, and that continued execution with context management can elicit substantial further capability.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents

    May 13, 2026Seunghyun Lee, David BrumleyAI Agent Security BenchmarksLLMs for Cybersecurity

  2. Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities

    Apr 6, 2026Charafeddine MouzouniLLM Agent SecurityLLM Agent Safety

  3. Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing

    Jun 24, 2026Liwei Yu, Shuo Li, Ming Zhou +2LLM Agent EvaluationAI Agent Security Benchmarks