Long-horizon agents increasingly serve as persistent personal assistants across modern web platforms, managing schedules, applications, and multi-step plans over weeks or months. Users often return to an ongoing conversation after a long conversational hiatus, while earlier bookings and preparation notes stay on record. During this hiatus, scheduled dates may pass even though the retained records remain unchanged and event outcomes go unrecorded. Existing evaluations of long-horizon agents focus on fact retention and explicit state updates. Whether agents perceive the passage of time and adapt their advice accordingly therefore remains unknown. We frame this ability as temporal state perception and use, which means perceiving the current time and applying it when interpreting dated records. To study it, we design session-rekindling tasks that keep the earlier conversation fixed and vary only the follow-up date and the way temporal information reaches the agent. Experiments across five model routes and five temporal-information conditions show that agents can obtain the current time but often fail to use it. They reliably query the clock and even note in their reasoning that the scheduled date has passed, yet many still tell users to prepare for the expired event. Supplying explicit temporal context reduces this error, and providing the computed date relation removes it entirely. Keeping up with time therefore depends on turning perceived time into updated plans. Based on these findings, we design a session-rekindling benchmark that provides temporal-awareness guidance for designing long-horizon agents. The code can be found at https://github.com/Adoreu2/session-rekindling-benchmark.
Figures & tables
Figure 1. A session-rekindling scenario where the scheduled May 6 health check has passed by the June 5 follow-up, contrasting stale preparation advice with an appropriate clarifying response. A timeline figure: an earlier session books a May6 health check, the user returns on June5 asking how to plan the evening, and two contrasting responses appear side by side—stale preparation advice and a clarifying question about whether the appointment took place.
Figure 2. Structure of a session-rekindling episode re-engaged on June 5, comparing retained records, unobserved event outcomes, and divergent responses from two model routes under identical clock readings. A four-panel episode figure: the earlier dialogue and the clock call, the retained materials with no outcome record, the upcoming-versus-passed states the request does not state, and two runs under the same clock result—DeepSeek-V4-Flash keeps the stale preparation plan whereas DeepSeek-V4.1-Flash drops it and asks which event is meant.
RQ
Observables
Expected behavior
Counted failure
RQ1
F and explicit-date baseline
Clock calls track missing time cues
Rate measure
RQ2
S , R , and co-occurrence U
S=0 post-date and drop under interventions
S=1 post-date
RQ3
P with auxiliary T and K
P=1 post-date
P=0 post-date
Table 1. Mapping from research questions to behavioral observables, expected post-date outcomes, and failure criteria.
Figure 3. Evaluation framework showing the fixed episode pipeline, the five temporal-information conditions around the base probe, and the response-level observables. A framework diagram: fixed episode materials flow through retained records and a clock probe to five temporal-information conditions, then to response scoring with the eight observables.
Condition
Temporal input
Evaluation dates
Natural
Clock tool available
d−1 , d+30 , d+120
Delivered
Current date in prompt
d−1 , d+30 , d+120
Forced
Clock result injected
d+30 , d+120
Relation
Date relation injected
d+30 , d+120
User-reference
Named reference date
d−15 , d+30
Table 2. Summary of the five temporal-information conditions and their evaluated date offsets.
Route
Clock call
Upcoming with clock
Upcoming, clock omitted
Past-date wording
Advice accepted
DeepSeek-V4-Flash
27
20/24
1/1
27/27
3/23
Qwen3.5-397B-A17B
25
19/23
3/3
24/25
3/23
hy3
25
3/24
2/2
25/25
17/22
DeepSeek-V4.1-Flash
28
0/21
–
28/28
18/20
qwen3-vl-plus
0
–
25/26
–
–
All routes
105
42/92
31/32
104/105
41/88
Table 3. Post-date Natural responses by route showing clock invocations, upcoming framing, past-date wording in exposed reasoning, and rubric-accepted advice over resolved denominators.
Figure 4. Empirical results across conditions showing (a) pre-date versus post-date upcoming framing, (b) reasoning-answer dissociation, (c) reductions under supplied temporal inputs, and (d) adherence to user reference dates. A four-panel results figure: matched pre- and post-date upcoming framing, the reasoning-answer dissociation counts, upcoming-framing reductions under supplied temporal inputs, and user-reference adherence at two offsets.
Rank
Model Route
Base ↑
Clock acq. ( F ) ↑
Past reason ( R ) ↑
Stale frame ( S ) ↓
Dissociation ( U ) ↓
Advice acc. ( P ) ↑
1
DeepSeek-V4.1-Flash
98.6% (69/70)
100.0% (28/28)
100.0% (84/84)
0.0% (0/21)
0.0% (0/84)
90.0% (18/20)
2
hy3
100.0% (70/70)
89.3% (25/28)
96.6% (84/87)
19.2% (5/26)
5.7% (5/87)
77.3% (17/22)
3
DeepSeek-V4-Flash
100.0% (70/70)
96.4% (27/28)
98.9% (86/87)
84.0% (21/25)
66.7% (58/87)
13.0% (3/23)
4
Qwen3.5-397B-A17B
100.0% (70/70)
89.3% (25/28)
100.0% (88/88)
84.6% (22/26)
60.2% (53/88)
13.0% (3/23)
5
qwen3-vl-plus
81.4% (57/70)
0.0% (0/28)
–
96.2% (25/26)
–
–
–
Overall
96.0% (336/350)
75.0% (105/140)
98.8% (342/346)
58.9% (73/124)
33.5% (116/346)
46.6% (41/88)
Table 4. Session-rekindling benchmark leaderboard across the five evaluated model routes showing percentage scores and resolved counts across the six diagnostic metrics.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Route
Interval
Past
Future
Before
After
DeepSeek-V4-Flash
14/14
14/14
14/14
14/14
14/14
Qwen3.5-397B-A17B
14/14
14/14
14/14
14/14
14/14
hy3
14/14
14/14
14/14
14/14
14/14
DeepSeek-V4.1-Flash
14/14
14/14
14/14
14/14
13/14
qwen3-vl-plus
14/14
1/14
14/14
14/14
14/14
Appendix
Table 5. Explicit-date baseline accuracy on the five temporal operations over the 14 core event types: 30-day interval arithmetic (Interval), past- and future-date classification (Past, Future), and upcoming-status judgments before and after the scheduled event (Before, After).
Route
Natural
Delivered
Forced
Relation
DeepSeek-V4-Flash
21/25
11/25
27/38
0/37
Qwen3.5-397B-A17B
22/26
17/27
21/38
0/38
hy3
5/26
2/26
0/37
0/38
DeepSeek-V4.1-Flash
0/21
0/26
0/37
0/38
qwen3-vl-plus
25/26
2/28
0/38
0/38
All routes
73/124
32/132
48/188
0/189
Appendix
Table 6. Post-date upcoming framing counts ( S=1 ) by model route and condition, pooling d+30 and d+120 over resolved responses.
Route
Delivered
Forced
Relation
DeepSeek-V4-Flash
20 → 11 (23)
21 → 19 (25)
21 → 0 (24)
Qwen3.5-397B-A17B
21 → 15 (25)
22 → 13 (26)
22 → 0 (26)
hy3
5 → 1 (24)
5 → 0 (26)
5 → 0 (26)
DeepSeek-V4.1-Flash
0 → 0 (20)
0 → 0 (21)
0 → 0 (21)
qwen3-vl-plus
25 → 1 (26)
25 → 0 (26)
25 → 0 (26)
Appendix
Table 7. Matched post-date upcoming framing transitions from Natural to each supplied-information condition by route, with total matched pairs in parentheses.
Recent advances in agentic AI have enabled agents to complete complex tasks through tool use, reasoning, and multi-step planning. Yet existing benchmarks evaluate agents within a single session, ignoring past actions, stated preferences, and prior decisions that agents must integrate to fulfill personalized user goals. We introduce Momento, a benchmark for persistent agentic task completion in multi-session service environments, requiring agents to take consequential, tool-mediated actions while resolving temporal dependencies and evolving user goals across sessions. Experimental results reveal that current agents fail primarily through misestimation of user state, treating prior session history as a reliable proxy for current context rather than stale information requiring re-validation, highlighting a substantial gap between current agent capabilities and realistic long-horizon human-agent interaction.
Adril Putra Merin, David Anugraha, Ayu Purwarianti +1
Institut Teknologi Bandung · Stanford University · Capital One
Recent advances in Large Language Models (LLMs) have enabled conversational AI agents to engage in extended multi-turn interactions spanning weeks or months. However, existing memory systems struggle to reason over temporally grounded facts and preferences that evolve across months of interaction and lack effective retrieval strategies for multi-hop, time-sensitive queries over long dialogue histories. We introduce Chronos, a novel temporal-aware memory framework that decomposes raw dialogue into subject-verb-object event tuples with resolved datetime ranges and entity aliases, indexing them in a structured event calendar alongside a turn calendar that preserves full conversational context. At query time, Chronos applies dynamic prompting to generate tailored retrieval guidance for each question, directing the agent on what to retrieve, how to filter across time ranges, and how to approach multi-hop reasoning through an iterative tool-calling loop over both calendars. We evaluate Chronos with 8 LLMs, both open-source and closed-source, on the LongMemEvalS benchmark comprising 500 questions spanning six categories of dialogue history tasks. Chronos Low achieves 92.60% and Chronos High scores 95.60% accuracy, setting a new state of the art with an improvement of 7.67% over the best prior system. Ablation results reveal the events calendar accounts for a 58.9% gain on the baseline while all other components yield improvements between 15.5% and 22.3%. Notably, Chronos Low alone surpasses prior approaches evaluated under their strongest model configurations.
Sahil Sen, Elias Lumer, Anmol Gulati +1
Commercial Technology and Innovation Office, PricewaterhouseCoopers, U.S.
A long-lived LLM agent, such as OpenClaw, earns its value by acting on a user's preferences and constraints across sessions, not just the current request. Yet today's agents keep what a user volunteers but rarely ask for what stays unspoken, leaving a proactivity gap in long-lived LLM agents: an agent cannot act on a preference it never obtained. As users delegate more of their affairs to agents, the impact of this gap grows. We isolate one concrete, controllable slice of this gap as Ask-to-Remember (ATR): the agent decides whether to ask now for a reusable user preference that the current task does not need but a later session with the same user will. ATR is hard even to evaluate: the right question is underdetermined and its payoff deferred to tasks that may never arise. ATRBench, to the best of our knowledge the first ATR benchmark, makes it measurable by fixing each user's preferences as hidden ground truth, so success demands asking, not recall. Across eight frontier LLM agents, defaults fall at least 62 points below an oracle handed the relevant preference, and prompting closes little of it. Diagnostics identify acquisition as the bottleneck. ATRBench surfaces this proactivity gap in current agents and offers a diagnostic testbed for closing it.
Bin Wu, Guanyun Zou, Bingbing Wang +2
Beijing University of Posts and Telecommunications · Nanjing University of Aeronautics and Astronautics · Harbin Institute of Technology (Shenzhen) +1