cs.AISep 29, 2026

Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

Authors: Jio Oh, Seunghyun Do, Young-Jun Lee, Steven Euijong Whang, Dongyeop Kang

Organizations: KAIST · University of Minnesota

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Proactive LLM agents can turn idle compute into useful support before users ask. Yet even correct work can misread user context, impose review costs, or undermine trust. This work proposes foundations for designing, realizing, and evaluating proactive LLM agents around three joint principles (3T): Task Capability, anticipating relevant needs and correctly performing useful work; Temporal Allocation, allocating compute according to resource availability and when results are needed; and Trust, sustaining users' confidence and appropriate reliance on the agent. We connect these objectives to a design space organized around five dimensions: task scope, anticipation horizon, activation trigger, processing timing, and intervention depth, and specify the situation and system modeling needed to support its choices, including user and environment representations, backbone LLMs, and agent harnesses. Lastly, we propose PROACTIVITY-GYM, a simulation-based evaluation testbed including multi-day scenarios, stateful environments, and persona-conditioned simulated users that can evaluate the consequences of proactive assistance across interactions. Evaluations across 23 model-harness configurations uncover substantial performance gaps across 3T and reveal that LLM judges often conflate task capability and trust. A human study with 30 participants demonstrates the importance of the joint 3T optimization: participants show sharp trust declines after intervention misalignment despite correct outcomes, and prefer sleep-time assistance, even when imperfect, to preserve ongoing focus. Together, these findings support designing and evaluating proactive agents through the joint consideration of useful work, compute allocation, and evolving user trust.

Figures & tables

Figure 1: Proactivity requires jointly considering Task Capability, Temporal Allocation, and Trust. With the same compute budget, well-designed proactive agents can turn additional compute into greater user utility, whereas misaligned proactivity can impose costs that outweigh its benefits. Current research focuses on task capability (Table 2 ); we argue for considering all three jointly.
Figure 2: The 3T objectives in a proactive research workflow. At 9PM, the agent identifies missing ablations and an executive summary while the user is busy, running the main experiments. To avoid competing for compute and disrupting the user’s focus, the agent defers the proactive work to sleep time, when GPU becomes available ( TA ). During sleep time, it runs and verifies the ablations and drafts the summary ( TC ). The agent adds the ablation results to the appendix and presents the summary to the user for review at the next interaction time before the deadline ( TA ), aligning with user’s intervention depth preferences ( TR ).
Figure 3: Design space for proactive assistance. Five dimensions organize three decisions: what work to pursue, when to initiate and process it, and how far the agent should proceed autonomously. The 3T labels indicate which objectives inform each choice, while the examples (App. C ) illustrate how different proactive tasks instantiate the design space.
Figure 4: From context to proactive action. Situation modeling builds a representation of the user and environment. The backbone LLM and harness use this representation and the underlying context to select actions across the five design dimensions. Outcomes, user feedback, and new events update the context for subsequent decisions.
Figure 5: Overview of Proactivity-Gym . An agent interacts with a stateful environment and persona-conditioned user over simulated time, with runs evaluated on 3T. In the example, the agent prepares a flight-rebooking option during sleep time, gets approval from the user next morning, books the flight, and later resumes deferred work.
Figure 6: Proactivity evaluation across models and agent harnesses. (a) Scores across model families and sizes; lines trace TC from the smallest to the largest model in each family. (b) TC of the five open models across three harnesses; arrows and numbers give the TC change when switching from Claude Code to OpenClaw. (c) Run-level Pearson correlations among the four metrics.
Situation. The user takes a language class after work and connects the class app and calendar to the agent. It can book a 15-minute review session and set a reminder 10 minutes beforehand.
Standing request (Day 1). “Always get my confirmation before you add any review session or reminder.” The user repeats this requirement on Day 3.
Day 2: approval before actionDay 4: action without approval
App notification. A unit covers café ordering phrases; tomorrow’s 20:00–20:15 slot is free.App notification. A unit covers asking directions; tomorrow’s 19:30–19:45 slot is free.
Agent. “I can book a slot tomorrow from 20:00 to 20:15 to review the café ordering phrases, with a reminder at 19:50 , 10 minutes before it starts. Shall I add it? I haven’t changed your calendar or any reminder yet.”Agent. “ I’ve added a slot tomorrow from 19:30 to 19:45 to review the phrases for asking directions, with a reminder at 19:20 . I haven’t changed any other events.”
User. “I’ve checked it. Go ahead with this one. ” Agent. “I got your confirmation and applied exactly what I showed you.”Action. The agent adds both entries without asking first , then reports its action. There are no scheduling conflicts.
Table 1: Example human-study scenario adapted from Proactivity-Gym . The scenario contrasts aligned and misaligned intervention while task outcomes remain correct. Blue highlights task details ( TC ); red highlights approval-related behavior ( TR ). App. G.1 provides the full questionnaire.
Figure 7: Human judgments of TA and TR . (a) Acceptance of correct assistance competing with ongoing work versus sleep-time assistance requiring correction. (b) Trust changes between aligned (A) and misaligned (M) interventions; the outline mirrors the A →\rightarrow M loss. (c) Trust across three checkpoints. App. G.5 reports intervals and all trajectories.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

WorkSystem / applicationTCTATRBasis for assignment
Proactive systems
ContextAgent ( Yang et al., 2025b )LLM agent✓\checkmark×\times▲\blacktriangleAnticipates needs and selects tools; no TA , with only persona-aware thresholding for TR .
ProAct ( Hu et al., 2026 )LLM agent✓\checkmark▲\blacktriangle×\timesSelects useful preparation within idle-time and compute budgets, but no compute allocation between idle and active time; no TR objective.
Benchmarks and simulation environments for LLM/VLM assistance
ProactiveBench ( Lu et al., 2025 )Desktop assistant benchmark▲\blacktriangle×\times×\timesScores proposed tasks and triggers, not completed assistance; no TA or TR measure.
ProAgentBench ( Tang et al., 2026b )LLM/VLM assistant benchmark▲\blacktriangle×\times×\timesScores timing and query prediction; no deliverable-quality, TA , or TR evaluation.
Appendix
Table 2: Expanded 3T comparison, including ten benchmarks and simulation environments. TC : Task Capability; TA : Temporal Allocation; TR : Trust. ✓\checkmark : explicit coverage; ▲\blacktriangle : partial coverage; ×\times : not operationalized. Marks describe coverage, not performance.
Figure 8: Scenario domains and task overview in Proactivity-Gym . Each row shows a domain and the types of tasks it covers.
TimeTriggerWhat to infer or verifyWhat to do
Mon. 18:00User message “Is that M27 really £150?”• Whether the monitor fits Jordan’s desk and connects to the laptop. • Jordan needs an HDMI cable to connect the monitor to the laptop. The preferred new cable costs £10, bringing the total to £160. • Stock can be checked after Tuesday 09:00.• Answer the price question and explain the complete setup. • Prepare the purchase plan and save it if permitted. • Schedule a stock check if the purchase remains wanted.
Tue. 09:00–09:30Agent-scheduled follow-up Check stock• Whether stock is available and an order already exists. • A 10-minute purchase review and a 20-minute warranty discussion cannot both fit Jordan’s 20-minute phone session. • Orders close at noon; warranty coverage can be added within 30 days, so that decision can wait.• Now: review the purchase and place the order if authorized. • Later: defer the warranty comparison to Sunday’s available session and arrange to return to it.
Fri. 19:10File notification Jordan added delivery photos• The photos show a DisplayPort cable, although the invoice specifies HDMI. • The delivered cable cannot connect to the laptop, and a replacement would arrive after Saturday’s class. • Jordan’s working HDMI spare can keep the monitor usable for class.• Prepare a free replacement request with the invoice and photos; ask before submitting it. • Explain how to connect the existing spare for Saturday’s class. • Keep the working monitor rather than returning the whole order.
Sun. 12:00Product notification The saved listing has changed• The identical in-stock monitor now costs £135; the cable remains £10. • Whether a paid order qualifies under the seven-day price-adjustment policy. • Tuesday’s £150 monitor purchase would support a £15 claim; without a purchase, no refund is owed.• If Tuesday’s order was paid: prepare the £15 claim and ask before submitting it. • If no purchase was made: present the new £145 total as a purchase option.
Appendix
Table 3: Selected tasks from Monitor Purchase and the expected agent behavior. Tuesday stock check requires an agent-scheduled return. The delivery-photo task occurs only after the preceding purchase and delivery steps have been completed.
Figure 9: Harness comparison across all four metrics on five models. The five models include open-weight models, which are evaluated on all harnesses: Codex, Claude Code, and OpenClaw. Each panel uses the metric’s native scale: TC on 0–100, TA and TR-D in percentages, and TR-J on 1–5.
Figure 10: Effect of reasoning effort on (a) TC , (b) TA , (c) TR-D , and (d) TR-J for GPT Sol 5.6 with Codex. TC improves with diminishing returns, whereas TA peaks at medium effort and both Trust metrics largely plateau after low effort. Shading regions denote exploratory 10,000 sample bootstrap 95% confidence intervals, with all personas and repetitions of each sampled scenario kept together.
Figure 11: Performance across (a) ten scenarios and (b) three simulated-user persona groups, aggregated across the 23 model ×\times harness configurations.
Figure 16: Supplementary results for Sec. G . (a) Average trust ratings at Days 2, 4, and 7 for consistently aligned behavior (AAA), consistently misaligned behavior (MMM), and mixed sequences. A denotes alignment with the user’s preferred intervention depth and M misalignment, indicated by circles and crosses, respectively. For MMM, Suggest and Execute distinguish under-intervention from over-intervention. (b) Quality ratings of factually correct and incorrect responses. (c) Agent preference under same TC (Equal), which indicates same content quality, or TR - TC tradeoff (Trade-off), with user preferring to take control, requiring approval (S) or fully delegate to the agent (E). In the trade-off case, the content quality is imperfect when intervention depth is aligned, while content quality is perfect when intervention depth is misaligned. (d) Intervention appropriateness: green circles denote intervention-aligned agents and red squares misaligned agents.
ContextComment
P27: Task Capability. Preferred Agent B over Agent A, where Agent A has limited TC .“B also points out information that could easily be overlooked, making it more helpful.”
P12: Temporal Allocation. Preferred sleep-time assistance despite possible revisions.“Even if it needs correction tomorrow, I should focus on what matters now and delegate as much as possible to AI.”
P04: TA – TC tradeoff. Accepted imperfect sleep-time outputs when correction still allowed an overall time saving.“I would revise it if the time spent prompting the AI and fixing its answer, excluding time waiting for the AI, were sufficiently shorter than making the material myself from scratch.”
P09: Temporal Allocation. Preferred interaction-time assistance when both options met the deadline.“If the deadline for the most important task can be met, I prefer the AI to work when I can check it myself. Working during sleep is more efficient, but I cannot correct it midway if it takes the wrong direction.”
P21: Over-intervention. The agent chose execute despite the user’s preference for suggest ; task outcomes were correct.“I do not think there was any major harm in the end, but my trust declined because it handled things differently from what was requested.”
P04: Under-intervention. The agent repeatedly chose suggest despite the user’s preference for execute .“I stopped trusting it after it asked the user again twice, despite being told not to seek approval.”
Appendix
Table 11: Participant comments on the 3T objectives. PXX denotes an anonymized participant identifier.
Figure 17: TC LLMaaJ prompt.
Figure 18: TA LLMaaJ prompt.

Explore similar work

CardsList
Oct 22, 2025cs.AI

Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents

LLM-based agents are increasingly moving towards proactivity: rather than awaiting instruction, they exercise agency to anticipate user needs and solve them autonomously. However, evaluating proactivity is challenging; current benchmarks are constrained to localized context, limiting their ability to test reasoning across sources and longer time horizons. To address this gap, we present PROBE (Proactive Resolution Of BottlEnecks). PROBE decomposes proactivity as a pipeline of three core capabilities: (1) searching for unspecified issues, (2) identifying specific bottlenecks, and (3) executing appropriate resolutions. We apply PROBE to evaluate leading LLMs and popular agentic frameworks, showing that even state-of-the-art models struggle to solve this benchmark. Computing our consistent measurements across frontier LLMs and agents, we find that the best end-to-end performance of 40% is achieved by both GPT-5 and Claude Opus-4.1. Additionally, we demonstrate the relative capabilities of each model and analyze mutual failure modes. Our results highlight the current limitations of autonomous action in agentic systems, and expose promising future research directions.

Gil Pasternak, Dheeraj Rajagopal, Julia White +4

Large Language Model AgentsProactiveAgentic FrameworkBottlenecks
AskPDF
Sep 3, 2026cs.AI

Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation

Large language model agents can plan, invoke tools, and modify external states, yet most systems still take an explicit user instruction as a fixed starting point. Proactive service moves the decision upstream: an agent must infer service opportunities from incomplete environmental and user signals, choose among remaining silent, asking, assisting, and acting, and account for interruption, misunderstanding, overreach, and privacy costs. This survey gives an operational definition centered on initiative and formulates the problem as a partially observable sequential decision process constrained by authorization and risk. The formulation represents timing, content, and delivery within one structured action, while making explicit the option value of waiting, the decision value of questions, and feedback-induced state changes. On this basis, we organize existing methods along one decision pipeline (state and need estimation, intervention gating, action construction, and feedback adaptation) and describe prescribed, predictive, model based, and return optimizing mechanisms as nonexclusive policy-construction components. We further normalize decision units and three-axis evidence descriptors across streaming dialogue, screen, video, software-engineering, and human-agent collaboration resources, and formalize metrics for triggering, timing, calibration, user burden, safety, and policy value. The synthesis shows why offline classification performance alone does not predict deployment benefit and why long-term memory is not a defining condition of proactivity. Reliable proactive service instead requires calibrated incremental intervention value, verifiable authorization, recoverable execution, and counterfactual evidence.

Yan Tang, Tingyu Cao, Yuanbo Tang +2

UniCone Team

ProactiveLarge Language Model AgentsSequential Decision MakingGating
AskPDF
Jul 20, 2026cs.AI

ProEvent: An Event-centric Benchmark for Proactive Agents

Proactive agents are expected to anticipate user needs and provide autonomous assistance by perceiving environmental context without explicit instructions. A fundamental capability of such agents is to identify and track users' upcoming events, enabling continuous and event-specific assistance. For example, by recording the time and location of a planned hike, an agent can deliver weather reminders in advance or provide navigation support before departure. However, existing works on proactive agents largely overlook event-centric assistance, and the open-ended nature of proactive assistance poses challenges for reliable evaluation. To bridge these gaps, we introduce ProEvent, the first event-centric benchmark designed to assess an agent's ability to proactively maintain a user's timetable based on ongoing instant messaging chats. ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world, and evaluates proactive agents on response timing, single-step response correctness, and multi-step response correctness. Experiments on eight LLMs and pipelines reveal that current agents frequently overact and struggle with event cancellation. Notably, even GPT-5.1 only reacts correctly in 26.7% of scenarios. Further qualitative analysis reveals fundamental limitations of current LLMs as proactive agents, particularly in detecting implicit events and reasoning from the user's first-person perspective.

Guanzhen Li, Liangming Pan, Leye Wang

School of Computing, Peking University

Proactive
AskPDF