Chronocooked: A Benchmark for Interval Timing in Reinforcement Learning Agents

Amrapali Pednekar; Alvaro Garrido-Perez; Yara Khaluf; Pieter Simoens

cs.AIAug 17, 2026

Chronocooked: A Benchmark for Interval Timing in Reinforcement Learning Agents

Authors: Amrapali Pednekar, Alvaro Garrido-Perez, Yara Khaluf, Pieter Simoens

Organizations: Department of Information Technology IDLab, Ghent University - imec · Computer Science Department Vrije Universiteit Amsterdam

Abstract

Interval timing is extensively studied as an important aspect of human behaviour. As artificial agents are increasingly designed to function alongside humans, their interval timing abilities also needs to be studied. However, research in this area remains limited and scattered. This paper presents Chronocooked, a reinforcement learning (RL) benchmark environment that enables a systematic study of interval timing abilities in RL agents. Inspired by Overcooked, the suite comprises cooking scenarios involving interval timing tasks drawn from the psychology literature. The tasks and reward functions are designed such that temporal information is unobserved but critical for optimal performance. The environment is intentionally kept simple to enable controlled experiments and support biologically plausible models. Each task is accompanied by evaluation metrics to study different aspects of interval timing in RL agents, namely, task performance, human-like timing and scalability. We report baselines using a non-recurrent architecture (CNN), a recurrent architecture (LSTM), and a biologically inspired recurrent architecture (CTRNN). The baseline model analysis shows that, although RL agents can successfully perform time-dependent tasks, they do not necessarily process and perceive time in the same way as humans. Understanding these differences is important for anticipating their impact on human-robot interactions (HRI).

Figures & tables

Panel of Figure 1 — Figure 1 . Chronocooked grid layout for the two core tasks. All other tasks discussed in this study are modifications of these two tasks to evaluate different temporal processing abilities of an RL agent. The Chronocooked grid layout consisting of an onion dispenser(in pink), an oven(in yellow), a delivery counter (in orange or green) for the FI timing task and the temporal bisection task

Figure 2 . FI timing (task performance): First oven check (FOC) and number of timing policies (top and bottom respectively) across model types (shown in different colours) and TD (x-axis). The standard deviation is shown as error bars on corresponding barplots. Missing barplots indicate that models did not converge in the soup delivery task in any of the runs. In the top plot, the barplots with a yellow border indicate cases where mean FOC was not significantly different from the corresponding TD. The mean FOC of the high memory recurrent models especially LSTM models closely tracks the target duration. These models also exhibit variable action sequences in the timing phase. Together these results indicate a tendency of implicit time-keeping in high memory recurrent models

Figure 3 . FI timing (human-like timing): The plots show results for the three regularities of scalar property. The leftmost plot also shows evidence for FOC validity because the mean FOC increases linearly with TDs. However, none of the models follow all scalar property regularities. The corresponding regression statistics for each plot are shown in blue at the bottom. LSTM125, LSTM256 and CTRNN256 showed strong positive correlation between the SD and mean FOC. The CV (SD/mean) showed no significant relationship with TD for all models except LSTM256, consistent with the constant CV regularity expected under scalar timing. Taken together, while some models showed partial conformity to individual regularities of the scalar property, no model satisfied all three regularities simultaneously.

Figure 4 . Multi-timer categorical variant (task performance): The plots show the average reward of the different models for two TD configurations (row titles) in the last two CL stages, namely, 4-timer and 5-timer setting (column titles) for the multi-timer task with buffer. The error bars depict the SD across the different model runs and seeds. The red dotted line indicates the CL performance threshold. All CTRNN models perform better than the LSTM models in both TD configurations. LSTM8 shows the worst performance in the 5-timer setting.

Figure 5 . Temporal bisection task (task performance): Psychometric curves of the three model architectures (rows) for the different anchors (columns). Each plot shows P(long) plotted against the different probe durations. The arithmetic mean (AM) and geometric mean (GM) of each anchor are shown at the bottom of each column. The grey dots represent p(long) of each run and the solid grey line shows the corresponding logistic fit. Similarly, the black dots represent the average P(long) across all runs and the red line shows the logistic fit on the average P(long). The vertical blue dotted lines corresponds to durations with P(long)

=

25% and P(long)

=

75% respectively. The region between the two blue dotted lines is called the Just Noticeable Difference (JND). Smaller JND corresponds to higher discriminability and vice versa. The bisection points (BPs) (duration at which P(long)

=

50%) are shown in the respective plots. The RL agents with sufficient memory and a recurrent architecture can both converge on the soup delivery task and perform the bisection task including OOD. They also exhibit an s-shaped response curve which is qualitatively similar to human and animal data.

Model	Soup delivery (%)	Bisection success (%)
CNN	100.0	8.3
CTRNN-125	85.4	75.0
CTRNN-256	97.9	93.8
LSTM-256	100.0	97.9
LSTM-125	100.0	100.0
CTRNN-8	16.7	100

Table 1 . Temporal bisection task (task performance)

Panel of Figure 6 — Figure 6 . Temporal bisection task (Human-like timing): (a) The plots show three BP regularities (y-axis) as a function of spread (x-axis). Both architectures show that BP tracks GM and stays around 1.2xGM even at larger spreads, unlike humans, where BP track the AM. (b) The plots show Weber ratio against spread (first column) and against AM for spreads of 2 and 3 (second and third column, respectively). Both architectures show decreasing discriminablity with spread, consistent with human data. For a specific spread, both architectures show a stronger positive trend in Weber ratio than human data (r=0.02) and CTRNN violates Weber’s law at spread 3. (a-b) Each plot shows the corresponding statistics of the regression fits,namely, the Pearson correlation coefficient (r), p-value (p) and number of samples (n). The results for corresponding human data taken from Kopec and Brody [2010] are shown at the bottom of each columns. The BP plots show that for both architectures, BP tracks GM and stays around 1.2xGM even at larger spreads, unlike humans, where BP track the AM. The weber ratio plots show that, both architectures exhibit decreasing discriminablity with spread, consistent with human data. For a specific spread, both architectures show a stronger positive trend in Weber ratio than human data (r=0.02) and CTRNN violates Weber’s law at spread 3

Effect	LSTM	CTRNN	Comments (contrast from human data)
BP: sub-geometric $\rightarrow$ sub-arithmetic	$\times$	–	Opposite direction (both models).
BP: BP/AM vs log2(L/S) (BP track AM)	$\times$	$\times$	Same trend as humans, but with a steeper decrease (both models)
BP: sub-arithmetic for small spreads	$\times$	$\checkmark$	However, CTRNN declines with spread rather than remaining at a constant fraction of AM.
BP: BP/GM vs log2(L/S)	$\times$	$\times$	Same trend as humans, but with a less steep increase (both models)
BP: supra-geometric for high spreads	$\times$	$\times$	BP remains below 1.2 times GM (both models)
Weber ratio (discriminability)	$\checkmark$	$\checkmark$	Same trend as humans, but with a steeper increase (both models)

Table 2 . Temporal bisection task (Human-like timing): Summary of BP and Weber ratio regularities relative to human data.

Figure 7 . Multi-Anchor categorization Task in Chronocooked. The example shows 4 delivery counters (in different colours) each for a different anchor duration. The grid layout for Multi-Anchor categorization Task in Chronocooked. It is similar to the temporal bisection task layout except that it contains more delivery counters. The example shows 4 delivery counters (in different colours) each for a different anchor duration.

Figure 8 . Multi-anchor (Temporal categorization ability): Visualization of 3-anchor (left) and 4-anchor(right) setting. Each subplot shows a stacked bar plot indicating the percentage of deliveries in each delivery counter (indicated by a different colour), aggregated across four runs for a given RL agent (rows) and anchor configuration (columns). Delivery counter 1 corresponds to the smallest anchor duration, delivery counter 2 to the second smallest, and so on. A smooth transition from delivery counter 1 to delivery counter 2 and then 3 and 4 indicates a good performing model. Grey bars denote failed cases, in which the agent did not deliver soup to any counter. The figure shows a clear transition from delivery counter 1 to 2, then 3, and then 4, as probe durations move from one extreme to the other. Importantly, this includes OOD probe durations. CTRNN models exhibit better performance as compared to LSTM models in the 4-anchor setting (last CL stage).

Panel of Figure 9 — Figure 9 . Multi-anchor (Task performance): The left panel shows the model-level task performance (as described in Section 8.1) of each recurrent model. Specifically, the plots show the percentage of incorrect deliveries aggregated across runs and main anchor configurations for the different n-anchor settings (indicated by different colours). The red stars on the x-axis denote probe duration categories where % of incorrect deliveries in the 4-anchor setting was significantly worse than the corresponding 3-anchor setting. The Wilcoxon signed-rank test was used for the comparison. The right panel shows the architecture-level comparison for the 4-anchor setting (last CL stage) task performance. Data across the different main anchor configurations and the two memory sizes within each architecture is combined. The red stars on x-axis in this case denote the probe duration category where LSTM task performance was significantly worse than CTRNN. A Mann-Whitney U test was used for the comparison. For both plots, the error bars show the SD and the p-values are corrected using the Holm–Bonferroni method. Refer Appendix B for details on all significance tests. CTRNN models showed a better task performance than LSTM models both for the model-level and architecture-level analysis.

Figure 10 . Asynchronous Multi-Oven Bisection Task in Chronocooked. The example shows 3 oven each with a different start time and different TD. The example shows the grid layout for multi-oven bisection task. The layout is similar to the temporal bisection task, except that there are 3 ovens in the last row instead of just 1 oven.

Figure 11 . Asynchronous Multi-Oven Bisection (task performance): The plots show the success rate (successful bisection tasks) across all ovens, probe durations (including OOD durations) and runs, for each model (shown in different colours) and anchor configuration (columns), in 2-oven and 3-oven setting (x-axis). While the asynchronous multi-oven task was in general difficult, performance improved as distance between

S

and

L

anchor increased. In the asynchronous variant, performance improved as distance between

S

and

L

increased. In the 3-oven setting, models achieved at most a 50% success rate for the 4–18 anchor configuration (maximum distance between

S

and

L

), while performance was lower for all other anchor configurations.

Figure 12 . Asynchronous multi-oven bisection (Scalability): The plot shows the training time steps required for each CL stage (n-oven setting) shown in different colours for each model (rows) and anchor configurations (columns). The numbers in the bars represent the corresponding CL-efficiency ratios. All models needed significantly more than 1x the time steps in 1-oven setting to converge in the 2-oven setting (CL-efficiency ratio >1). All models reach the 3-oven setting only for anchor configuration 4-18 (largest spread). Only one run from

LSTM125

reaches 3-oven setting for anchor configuration 4-8 (smallest spread). All models need significantly more than 1x time steps of previous CL stage to complete the current CL stage.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 13 . Multi-timer exact variant (task performance): The plots show the average reward of the different models (x-axis) in the different CL stages (columns) for the 2 TD configurations (rows) of the multi-timing exact-timer variant. CTRNN models fail already in the 1-timer stage. The LSTM models perform better in anchor configuration with smaller distance between consecutive anchors. LSTM8 reached up to 5-timer stage surpassing the larger memory models. This could be attributed to a fix timing policy followed by this model.

Figure 14 . Multi-timer, exact-timer variant (# timing policies): The plots show the number of unique timing policies in the different seeds, averaged across runs, for the different RL agents (shown in different colours) for the anchor configuration 2-4-6-8-10 in the exact-timer variant. The plot titles indicate the different CL stages. As in the FI timing task, LSTM8 showed a lower number of timing action sequences compared to the two higher memory LSTM models (see Figure \ref{fig:multitimer_num_timing_actions_exact}), suggesting greater reliance on embodied timing (a fixed action sequence) rather than implicit time-keeping. This could be one of the possible reasons for LSTM8 model to successfully reach 5-timer setting in the more difficult exact-timer variant of the multi-timer task.

Figure 15 . Multi-timer categorical variant (CL-efficiency): The training time steps required by the different runs of the the models (rows) for the TD configurations (columns), split by CL stages (shown in different colours) for categorical variant. The numbers in the bars represent the corresponding CL-efficiency ratios. Except for CNN, at least one run from all models reach the last CL stage (5-timer) in the categorical timer variant

Figure 16 . Multi-timer exact-timer variant (CL-efficiency): The training time steps required by the different runs of the the models (rows) for the TD configurations (columns), split by CL stages (shown in different colours) for exact-timer variant. The numbers in the bars represent the corresponding CL-efficiency ratios. All runs in the CTRNN models fail already in the 1-timer stage. The LSTM models perform better in anchor configuration with smaller distance between consecutive anchors. 2 runs from LSTM8 reached up to 5-timer stage surpassing the larger memory models. This could be attributed to a fix timing policy followed by this model

Figure 17 . Multi-anchor categorization task performance: The plots show the percentage of incorrect deliveries across runs and seeds for a given probe duration category, with error bars indicating +- standard deviation. The final column shows the mean and standard deviation across runs and anchor configurations. Lower values indicate better performance; Models should be compared along each probe-duration interval (<=A1, A1–A2, etc.) to isolate the effect of probe durations. The red stars in the average column indicates the cases when anchor 4 performance was significantly worse than corresponding anchor 3 performance. At an architecture level, for 4 anchor setting, CTRNN yielded significantly fewer incorrect deliveries than LSTM in 3 of the 5 probe categories: Probe category 0 (F = 14.93, p < .001), Probe category 1 (F = 4.48, p = .041), and Probe category 4 (F = 9.27, p = .004) At an architecture level, for 4 anchor setting, CTRNN yielded significantly fewer incorrect deliveries than LSTM in 3 of the 5 probe categories

Figure 18 . Multi-anchor (CL efficiency): Training time steps required by each run of the models (rows) for the different CL stages (shown in different colours) across all main anchor configurations (columns). The numbers in the bars represent the corresponding CL-efficiency ratios. The last column shows an average taken across the main anchor configurations for each model. Most runs from all models reached the last CL stage (4-anchor) across all anchor configurations.

Model	Probe category	$N$	4-anchor Median [IQR]	3-anchor Median [IQR]	$W$	$p$	$p_{\mathrm{Holm}}$	Reject $H_{0}$
LSTM125	<=A1	15	0.19 [0.23]	0.00 [0.16]	76.00	0.016	0.033	Yes
LSTM125	A1-A2	15	0.11 [0.42]	0.00 [0.01]	75.00	0.002	0.007	Yes
LSTM125	A2-A3	15	0.15 [0.58]	0.00 [0.00]	102.00	< .001	0.004	Yes
LSTM125	A3-A4	15	0.09 [0.33]	0.00 [0.10]	82.00	0.032	0.033	Yes
LSTM256	<=A1	12	0.27 [0.69]	0.05 [0.10]	43.00	0.006	0.023	Yes
LSTM256	A1-A2	13	0.08 [0.19]	0.02 [0.04]	49.00	0.087	0.087	No

Appendix

Table 3 . Multi-anchor (Task performance): Wilcoxon signed-rank comparisons between the 3-anchor and 4-anchor conditions.

p

-values were corrected using the Holm method.

Model	Probe category	$N$ 125 / 256	Memory=125 Median [IQR]	Memory=256 Median [IQR]	Statistic	$p$ -value	$p_{\mathrm{Holm}}$	Reject $H_{0}$
LSTM	<=A1	15 / 12	0.19 [0.23]	0.27 [0.69]	76.50	0.524	1.000	No
LSTM	A1-A2	15 / 13	0.11 [0.42]	0.08 [0.19]	106.50	0.689	1.000	No
LSTM	A2-A3	15 / 12	0.15 [0.58]	0.18 [0.40]	84.00	0.788	1.000	No
LSTM	A3-A4	15 / 12	0.09 [0.33]	0.53 [0.87]	79.00	0.606	1.000	No
LSTM	>=A4	15 / 11	0.59 [0.86]	1.00 [0.51]	67.00	0.418	1.000	No
CTRNN	<=A1	13 / 11	0.00 [0.00]	0.00 [0.18]	52.50	0.159	0.648	No

Appendix

Table 4 . Multi-anchor (task performance): Mann–Whitney U test comparing the percentage incorrect deliveries in 4-anchor setting between the two memory sizes (125 and 256) of a specific model architecture.

Panel of Figure 19 — Figure 19 . Multi-oven bisection (task performance): The top plot depicts the asynchronous task performance and bottom plot shows the synchronous task performance. The plots show the success rate (successful bisection tasks) across all ovens, probe durations (including OOD durations) and runs, for each model (shown in different colours) and anchor configuration (columns), in 2-oven and 3-oven setting (x-axis). While the asynchronous multi-oven task was in general difficult, performance improved as distance between $S$ and $L$ anchor increased. All the RL agents achieved good performance in the synchronous variant of the task. In the asynchronous variant, performance improved as distance between $S$ and $L$ increased. In the 3-oven setting, models achieved at most a 50% success rate for the 4–18 anchor configuration (maximum distance between $S$ and $L$ ), while performance was lower for all other anchor configurations.

Figure 20 . Asynchronous multi-oven bisection (psychometric curves): The plot shows the psychometric curve of each model (rows) for the different ovens in the 2-oven (left) and 3-oven setting (right), for the anchor configuration 4-18 which was the best performing anchor (also the one with the largest distance between the anchors). The primary oven (oven 1) shows the best performance among the different ovens, especially in the 3-oven setting. The primary oven (oven 1) shows the best performance among the different ovens, especially in the 3-oven setting.

Panel of Figure 21 — Figure 21 . Asynchronous Multi-oven bisection (psychometric curve): The plot shows the psychometric curve for the different ovens for 2-oven and 3-oven setting for anchor 4-12 (top) and 4-8(bottom). All model performance degrades in the 3-oven setting. The primary oven (oven 1) shows the best performance among the different ovens. The performance in the 3-oven stage for the asynchronous timing task is better in the 4-12 anchor configuration rather than the 4-8 configuration. All model performance degrades in the 3-oven setting. The primary oven (oven 1) shows the best performance among the different ovens.

Figure 22 . Synchronous multi-oven bisection (scalability): The plots show the training time steps required for each CL stage (n-oven setting) shown in different colours for each model (rows) and anchor configuration (columns) for the synchronous variant. The numbers in the bars depict CL-efficiency ratios. At an architecture level, the CL-efficiency ratio of both architectures was significantly more than 1. In the synchronous variant most model runs reached the last CL stage (3-oven).

TD	Model	N	M	SD	t	p
4	LSTM8	36	3.39	0.87	-4.21	0.000
4	LSTM125	48	3.98	0.53	-0.27	0.785
4	LSTM256	48	3.96	0.68	-0.42	0.674
4	CTRNN8	24	0.00	0.00	-inf	0.000
4	CTRNN125	42	2.52	1.44	-6.66	0.000
4	CTRNN256	36	1.97	1.63	-7.46	0.000

Appendix

Table 5 . FI timing: Significance test comparing FOC with the corresponding TD for each model

Figure 23 . Psychometric curves of the models (rows) for the different anchors (columns). Y-axis shows percentage of long - P(long) and x-axis shows probe (test) durations along the the arithmetic mean (AM) and geometric mean (GM) of each anchor. The grey dots represent p(long) of each run (with 11 seeds each). The black dots is the average P(long) across all runs. The red line shows the sigmoid fit on the average P(long). The bisection points (BP) are shown in the respective plots. The smaller memory sizes in the recurrent models do not perform as good as the corresponding high memory models

Explore similar work

Oct 7, 2026cs.AI

AgentTime: Can Agents Estimate and Control Their Own Runtime?

An essential control of AI agents is their ability to manage runtime. This ability requires a sense of time-awareness, to predict and estimate wall-clock time and to control their own actions. Prior work has focused on time-awareness, but duration-following and control in native agent harnesses remain unexplored. We present AgentTime, a benchmark for testing whether agents can work for a requested duration, predict their runtime, and estimate elapsed time afterward. It comprises 222 tasks from 18 sources spanning coding, computer use, agentic work, and automated research. Duration-following experiments append a single instruction specifying how long to work, with requests ranging from about a minute to multiple days. Accuracy on these instructions varies substantially: Fable 5.1 in Claude Code deviates from requested runtimes by a typical factor of 2.9

\times

, compared with only 1.2

\times

for GPT-6 Astra in Codex. However, matching the requested runtime does not, by itself, establish continued work on the task. Among 158 reviewed Astra runs with classifiable transcripts, 14 explicitly slept after appearing to finish. In forecasting experiments, predictions tend to overestimate natural runtimes. In retrospective experiments, removing temporal information more than doubles deviation for Sol and Astra and nearly doubles it for Fable. An agent's ability to complete a task does not guarantee that it can control its own time or work for the whole requested duration. For agents to run reliably, safely, and autonomously over long horizons, we require the evaluation of both.

Michael Ofengenden, Maksym Andriushchenko

MATS · ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, Tübingen AI Center

Topic pending

Ask PDF

Sep 28, 2026cs.AI

ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning

A goal that is close in space can be far away in time. Obstacles, terrain, and the agent's own capabilities determine how long it takes to get there. Yet, critics in contrastive and survival reinforcement learning do not measure the distances in their representation space in units of time. We therefore introduce ChronoSRL, which gives the critic's embeddings an explicit temporal geometry. The distance between state-action and goal embeddings is trained to match the time that the agent takes to reach the goal (goal-reaching time), while goals that were not reached, and goals from other trajectories, are pushed at least one discount horizon away. Furthermore, reaching a goal quickly once does not mean that reaching it is reliable in general, so the policy should not follow the temporal distance directly. Instead, we build on survival reinforcement learning and predict from our temporal embeddings not only the full distribution of goal-reaching times but also the time spent near the goal. Thereby, the policy is trained to favor actions that reach the goal sooner and more reliably and that keep the agent near it. ChronoSRL learns faster and reaches higher performance than contrastive, action-chunked contrastive, and survival reinforcement learning baselines on seven standard locomotion and navigation benchmarks, even with much smaller networks. To test the limits of self-supervised reinforcement learning, we introduce velocity tracking, goal-position reaching, and box climbing tasks with a quadruped robot in a realistic sim-to-real locomotion setup, and show how the shaping terms that are typical for robotics can be naturally incorporated into our framework. ChronoSRL is the only one of the tested self-supervised reinforcement learning methods that learns to stay at the commanded velocities and goal positions, and climbs the highest boxes.

Nico Bohlinger, Jan Peters

Technical University of Darmstadt, Germany · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI

Goal-Conditioned Reinforcement Learning Offline Reinforcement Learning Chronos

Ask PDF

May 24, 2026cs.AI

ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents

Proactive task-oriented agents must autonomously anticipate user needs, identify actionable opportunities, and trigger software actions at appropriate moments - fundamentally shifting from reactive systems that await explicit instructions. However, existing approaches lack generalizable end-to-end solutions for measuring and optimizing such anticipatory behaviors. This paper introduces ProActor, a unified framework for conversational task scheduling that integrates: (1) a domain-agnostic automated annotation methodology that enables scalable proactiveness reinforcement learning (RL) by generating full opportunity time windows instead of rigid point labels, (2) systematic proactiveness metrics capturing both timing quality and reference action alignment, and (3) RL optimization using GRPO with various reward designs. Our insight is that RULER-based rewards with proactiveness rubrics are crucial for improving timing quality, and that proactiveness optimization enabled by stage-aware composite rewards is key to balancing timing quality and reference action alignment. Timing-aware RL requires extensive exploration, demanding efficient infrastructure. We develop ART-F, an adaptive framework combining request-adaptive inference clusters with DDP-based training on single-node multi-GPU systems, enabling LoRA training of 4-bit Qwen2.5-14B-ProActor-Q4 with 4-8x speedups. Experiments on two newly auto-annotated datasets demonstrate significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art (SOTA) baselines. Ablations validate the effectiveness of distinct composite reward variations.

Lei Ding, Bin He, Chenguang Wang +1

University of California, Santa Cruz · 2Zillow Group

Proactive Agentic Reinforcement Learning

Ask PDF