Organizations: Myeongseongsimjae AX Institute, Yeongyang-gun, Gyeongsangbuk-do, Republic of Korea · Department of Computer Science and Software Engineering, Korea University, Seoul, Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea
Real-time agents are often evaluated with the world paused, or with decision time rounded to whole ticks (tick conversion). Tick conversion predicts almost no loss for a learned decider whose hit accuracy falls by 14.8 percentage points (pp) on the same games in asynchronous play. We instead split the decider's gap to a zero-latency teacher into time and quality components. In ViZDoom, we train a small decider by imitating a scripted teacher, run the game on wall-clock time, and add a control in which the teacher waits for a call to the decider's server before deciding. On a Windows host, three preregistered studies put the time component at 6.5-8.7 pp and the quality component at 6.3-8.6 pp. On a Linux host that skips 0.08-0.09% of ticks, the time component disappeared (-0.1 pp; paired 95% bootstrap interval [-0.4, 0.0]) while the quality component remained (5.6 pp [3.7, 7.5]). Retraining on teacher-labelled states from the decider's own play (DAgger) improved it on new games on both hosts (2.6 pp [0.4, 4.9] and 3.1 pp [1.1, 5.2]), a preregistered partial success. Imposing one delay schedule on every arm in live play left a quality component of 5.8 pp [4.0, 7.5] on 100 new games, and retraining cut the decider's disagreement with the teacher on its own states from 22.7% to 14.9%. In a conveyor simulator with a 400 ms deadline, delay past it erased the quality component. A latency-matched control whose delays match the decider's shows which remedy to try.
Figures & tables
Figure 1: Time and quality components in three studies on host 1 (v4+v4b, v5, v6b; 18, 18 and 56 distinct games). Left: time component T0 − S. Right: quality component S − decider for O, N and N32, including the input change and, in v5, imperfect latency matching; arrows: reduction from O. Bars: paired 95% game-bootstrap intervals. Symbols: Table 1 .
Symbol
Meaning
T0
teacher with no added latency
S
the same teacher waiting for the decider’s server before acting (latency-matched teacher)
SS
the teacher deciding from the state sentence the decider reads (v10)
D
a learned decider on the server, in Sections 4 and 5 (O, N, N32 or M)
O
original imitation decider
N
decider after two on-policy DAgger rounds, in which the decider plays, the teacher labels the states it reaches, and the decider is retrained on them; N1 after one round
Table 1: Arms and symbols. Colours of these arms in the data figures: T0 dark gray, S gray, O orange, N dark blue, N32 light blue; other series use colours defined in their own figures.
Figure 2: The evaluation loop. The engine sends exact coordinates to the teacher arms and a state sentence to the decider’s server. Tick conversion holds the previous action for m(L) ticks; asynchronous play runs at 35 ticks per second and repeats the last action while a decision is pending (hatched bar). Dashed: offline training data. Symbols: Table 1 .
Figure 3: The three arms of the latency-matched decomposition. S acts like T0 but calls the decider’s server and waits for, then discards, the answer; D is the learned decider on that server. T0 to S adds the wait (gray: time component); S to D swaps in the decider and its input (highlighted: quality component). Box widths do not encode time.
Figure 4: Primary contrasts of all studies on one axis: time components, quality components of O, DAgger gains and two other contrasts, in hit-accuracy points with 95% game-bootstrap intervals. Filled: preregistered; open: reanalysis (tick conversion over distinct games). Host 2: v12 and v18; all others host 1.
Arm or contrast
Hit accuracy (%)
95% interval
or difference (pp)
game bootstrap
run-level t
T0
95.5%
–
–
S
86.8%
–
–
O
80.0%
–
–
Time, T0 − S
8.7 pp
[6.6, 10.9]
–
Quality, S − O
6.8 pp
[4.5, 8.8]
[1.2, 12.4]
Table 2: Latency-matched decomposition of hit accuracy
Figure 5: Live play with one delay schedule imposed on every arm (v18; host 2; 100 distinct games, two sessions). Time: T0 − teacher on the schedule; quality: teacher − decider on the same schedule; DAgger: N or N32 − O. Paired 95% game-bootstrap intervals; dotted: 2 pp strong-success line.
Figure 6: A direct view of covariate shift (exploratory). (a) Share of state sentences, with an enemy in view and ammunition left, that never occur among the 40,440 distinct training sentences: 8.0% of the teacher’s validation sentences, 49.2% and 47.8% of those met in the two DAgger rounds. (b) How often each state occurs in the training data.
Figure 7: Disagreement with the teacher on the decider’s own states against its quality loss, per game (v18; 100 distinct games). Lines: least-squares fit per decider.
Table 5: Host, scene and decision-period contrasts
Figure 8: Time (T0 − S), quality (S − O) and DAgger (N − O) contrasts with 95% game-bootstrap intervals; filled: interval excludes zero; gray: host 1, black: host 2. Top: defend_the_center, hit accuracy, 56 distinct games per host (v6b, v12). Bottom: deadly_corridor, mean reward, 94 games on host 1 and 200 on host 2 (v11).
Figure 9: A deadline makes the time component a cliff (conveyor simulator, line A, asynchronous). (a) Cost per part by injected delay: exact teacher, teacher with ± 50 ms jitter, decider D over HTTP; dots: episodes (10 per cell, 500 parts); dotted: 400 ms deadline. (b) 340–400 ms. (c) Late decisions: four episodes at once (preregistered), one at a time (exploratory).
Table S1: Decider agreement with the teacher’s actions
Figure S1: Five metrics in the replication study (v6b; 100 seeds, 56 distinct games). Dots: means; bars: 95% intervals from resampling distinct games. Lone-enemy reaction: ticks from the first tick with one enemy in view to the first shot aimed within 6 degrees of it. Hits taken depends on survival time and is not ranked.
Figure S2: Hit accuracy by run in the three studies on host 1: runs (v4+v4b: 5 S/O pairs; v5: 5 S/N/O triads; 18 games each) or 25-seed blocks (v6b: 4 blocks, 56 games). Squares: the three earlier-session T0 runs; filled diamonds: pooled, with 95% game-bootstrap intervals; dashed: the two v5 triads outside the skip band 0.8–1.25.
Figure S3: Slope of hit accuracy on skipped-tick share, run by run, with 95% t -intervals; n: runs (v4+v4b, v5: 18 games) or 25-seed blocks (v6b: 56 games). Filled circles: intervals that exclude zero. T0 is omitted because it skips almost no ticks. Scatter: Figure S4 .
Figure S4: Run-level data behind Figure S3 : hit accuracy over distinct games against the skipped-tick share of each run (v4+v4b, v5) or 25-seed block (v6b), with least-squares lines per arm and the slope with its 95% t-interval in each legend entry; hollow v5 markers are the triads that failed the S/N skip match. T0 is omitted because it skips almost no ticks.
Figure S5: How often to decide depends on the scene: O on host 2 holds each action for k ticks (v15). (a) Hit accuracy (defend_the_center) and mean reward (deadly_corridor) by k ; dashed: O acting every tick on a decision 8 ticks (228.6 ms) old. (b) Differences from k=0 and fresh versus stale (square), with 95% game-bootstrap intervals.
Figure S6: The decomposition on two scales (exploratory for b). (a) Hit accuracy of T0, S and O in the decomposition study (v4+v4b; 95.5, 86.8, 80.0%) and the replication (v6b; 94.3, 87.9, 81.6%). (b) Hits per game in the same studies (24.7, 22.1, 18.9 and 24.3, 22.4, 19.4). Arrows mark the time and quality components.
Figure S7: The data-volume control (v7; 56 distinct games, 4 blocks): N adds 92,725 rows from its own asynchronous play, labelled by the teacher; M adds 92,725 rows from new teacher-played games. Gray segments: blocks; brackets: paired 95% game-bootstrap interval of the pooled difference. Dotted: O, N and S from v6b; T0 lies off the axis at 94.3.
Figure S8: Tick conversion against asynchronous play for the same decider on the 18 distinct games of seeds 850000–850029, sorted by the loss. Pooled hit accuracy: 98.6% under tick conversion and 83.8% in asynchronous play.
Figure S9: On-policy errors in the DAgger rounds of v5 (enemy in view, ammunition left), driven by O and N1; row percentages (sums 99–100, errors on one 0–25% scale). Right: TR → TL rate, O 20.7% [18.7, 22.7], N1 18.2% [16.0, 20.5], difference − 2.4 [ − 5.5, 0.6] (descriptive). No F row: the teacher never chose it.
Figure S10: Decision time on the two hosts (exploratory): cumulative share of episodes by the median decision time within the episode, 1,300 decider episodes on host 1 and 860 on host 2 (log scale). The dotted line marks one engine tick (28.6 ms).
Figure S11: All 28 studies of this paper (25,550 episodes in total): episodes played, host and scene. Blue: preregistered and run; lilac: exploratory or without a frozen hash; orange: failed its own calibration or fidelity check; gray: withdrawn. Bars at 1: not executed (v8, v14b) or counted in another row (v4 in v4b, v6b+ in v6b).
Figure S12: Time and quality components on identical synthetic skip schedules imposed on T, O and N (56 distinct games, 100 seeds). Time: T without skipping minus T on the schedule; quality: T minus O and T minus N on the same schedule. Paired 95% game-bootstrap intervals.
Contrast (host-1 recorded schedules, 29 games)
Original decider O
DAgger decider N
Teacher on S schedule − teacher on own schedule (bias term)
− 2.7 [ − 6.2, 0.5]
− 2.2 [ − 6.2, 1.6]
Quality, schedules matched exactly
7.1 [3.7, 10.6]
− 0.2 [ − 2.5, 2.1]
Quality, shares matched (original design)
4.4 [0.2, 8.9]
− 2.5 [ − 6.7, 1.8]
DAgger gain N − O on N’s schedule
4.8 [1.8, 7.6]
Appendix
Table S2: Recorded host-1 schedules replayed identically
Run
Acc. dist.
Acc. raw
Kills
Shots
Hits taken
Ticks
Lone react.
Skip %
S1
85.3
87.3
22.50
25.7
5.53
1250
29.2
11.3
D4
80.7
81.6
19.37
23.7
5.20
1112
35.0
7.1
S2
86.6
87.9
22.87
26.0
6.10
1235
25.5
5.9
D5
83.7
81.9
18.83
23.0
6.00
1072
38.7
9.6
S3
86.3
86.7
22.13
25.5
5.53
1243
25.9
6.9
D6
83.2
82.4
19.17
23.3
6.03
1087
36.0
6.7
Appendix
Table S3: Decomposition study (v4, v4b): five S/D pairs in execution order, seeds 850000–850029.
Run
Acc. dist.
Acc. raw
Kills
Shots
Hits taken
Ticks
Lone react.
Skip %
S1
86.5
87.2
22.47
25.8
5.53
1234
27.0
7.4
N1
79.5
80.3
20.50
25.5
5.57
1159
32.9
15.7
O1
74.5
74.2
17.60
23.7
5.70
1050
35.1
19.0
O2
72.4
71.1
16.10
22.6
5.17
997
38.8
25.3
N2
82.0
83.3
21.37
25.6
5.47
1192
27.2
11.1
S2
89.5
88.2
22.40
25.4
6.00
1245
30.1
6.8
Appendix
Table S4: Intervention study (v5): five triads in execution order (triads 2 and 4 in reverse order), seeds 850000–850029.
Triad
Skip S
Skip N
Skip O
S − N
S − N adj.
N − O
N − O adj.
1
7.4
15.7
19.0
7.0
1.9
5.1
3.0
2
6.8
11.1
25.3
7.5
4.8
9.6
1.0
3
8.0
8.1
7.2
2.3
2.2
4.9
5.5
4
9.6
8.3
6.7
3.0
3.8
1.5
2.5
5
9.9
9.4
9.3
2.7
3.0
-0.1
-0.0
Appendix
Table S5: Skip-adjusted contrasts per triad of the intervention study (exploratory). Adjusted contrasts move each decider to the skipped-tick share of the arm it is compared with using the pooled slope κ of Section S1.4 (Proposition 1 ). Values in percentage points, skipped-tick shares in percent.
Figure S13: Episode hit accuracy against the share of skipped ticks in that episode for the 450 episodes of the intervention study.
Round
Driver
Seeds
Rows
Engaged
Skip %
Minutes
1
a814587e03
720000–720059
44,820
43,707
11.5
31
2
240805651d
720100–720159
47,905
46,725
11.1
33
Appendix
Table S6: DAgger collection rounds. “Engaged” counts decisions with an enemy in view.
Teacher
n
TL
TR
F
SL
SR
FSL
FSR
Round 1 (original decider in control) , 24,547 decisions
TL
11,635
93.2
4.0
0.0
0.6
0.4
1.0
0.9
TR
7,190
20.7
73.5
0.0
1.3
1.2
1.3
2.0
SL
992
0.2
0.2
0.0
98.8
0.1
0.7
0.0
SR
819
0.2
0.2
0.0
0.0
99.5
0.0
0.0
FSL
1,658
0.1
0.1
0.0
0.2
0.0
99.1
0.5
Appendix
Table S7: On-policy confusion (rows: teacher action, columns: decider action, % of row). TL/TR turn left/right, F fire, SL/SR strafe, FSL/FSR fire while strafing.
Figure S14: Validation Brier score during training for the original decider and after each DAgger round (seed 31). The validation split of the later rounds includes on-policy rows.
Figure S15: The evaluation loop of the earlier tick-converted evaluations, including the language-model comparator arms (labelled A–F there; in the main text D denotes the original decider). The engine sends exact coordinates to the scripted teacher and a state sentence, written by a perception script, to the learned arms. One arm plays each run, under tick conversion (which waits for each decision and holds the previous action for m(L) ticks) or asynchronous play on wall-clock time.
Figure S16: The decider scores each candidate action against the state sentence. A request carries the state sentence and the seven candidate actions with their descriptions. The encoder turns each candidate, with the sentence, into 1,785 hashed word and character n-gram and overlap features; a shared layer (1,785 to 128, ReLU) embeds each candidate; set pooling (mean and max, 256) summarises all candidates; a context layer (the candidate’s 128 values with the 256 pooled values, 384 to 96, ReLU) and a linear score (96 to 1) rate each candidate; a softmax with a fitted temperature returns the choice and a probability for every candidate. 265,665 parameters; about 2 ms per decision on one CPU core; the same code serves both scenes and only the weights change. Training uses the teacher’s labels with a sampled policy-gradient estimate of the Brier score, keeps the step with the best validation Brier score and fits the temperature on a calibration split. No pretrained language model is used.
Figure S17: Under tick conversion, injected latency leaves the teacher’s skill intact below one tick and removes it quickly above, and slower controllers fall on the degraded side. Comparator arms are placed at the median of their per-episode median latency. They are context rather than latency-only comparisons, because their policies differ. The arms are the distilled decider; a 1.7B-parameter open-weight large language model (LLM) choosing every action (small LLM); the same model returning 16 actions per call (action queue); and a command-line language-model agent (LLM agent), on the 20 raw seeds of the comparator evaluation. The gray band is an axis break: 0 ms is plotted to its left. Teacher with fixed injected latency (one run per latency on the 100 seeds of the second tick-converted evaluation; points are measured values joined by lines, not a fitted step); These tick-converted runs were preregistered as single runs without intervals.
Figure S18: Distinct teacher games as seeds are added. The dotted line is the case in which every seed gives a new game.
Figure S19: Qualifying the metrics on 200 seeds before any learning.
Figure S20: Kills per game against median latency of one model call, under tick conversion (peripheral comparison).
ViZDoom 1.3.1 (asynchronous player mode, 35 ticks per second), NumPy 2.5.3; Python 3.14.0 (host 1), 3.12.3 (host 2)
Training
PyTorch 2.13.0 CPU (host 1), 2.14.0 CPU (host 2); 2,500 steps, training seed 31 (and 32 for N32)
Decider
hashed n-gram encoder to 1,785 features, 265,665 parameters, served over local HTTP; original O a814587e03, after one DAgger round 240805651d, after two rounds N 8ce6b41475, seed-32 replicate 2f4785728c, data-volume control M 2061ae8b; second scene O a164d90514, N 11409b5c60
Open-weight comparator
qwen3:1.7b through Ollama on host 1 (model layer 3d0b790534)
Commercial comparator
gpt-6-sol through the Codex command-line tool 0.156.1, default settings, run on 2026-09-23
Appendix
Table S8: Experimental environment. Weight digests are the first 10 hexadecimal digits of SHA-256.
Recorded arm
Exact matches
Hit accuracy (%)
Skipped ticks (%)
recorded
replayed
recorded
replayed
N, DAgger decider
0 of 50
87.8
99.2
8.2
8.2
O, original decider
0 of 50
80.9
98.8
8.6
8.7
S, teacher
0 of 50
85.2
99.3
9.6
9.6
Appendix
Table S9: Fidelity of trace replay (v9, B1). Accuracy and skipped-tick share pooled over the 50 episodes of each arm.
Figure S21: Schedule adherence in v18: share of decisions by target and realised gap (ticks), per waiting arm.
Figure S22: Skipping in v18: (a) share of skipped runs at least as long as a given length; (b) cumulative share of episodes by skipped-tick share.
Figure S23: Quality component of O per game in v18: (a) teacher and O on the same schedule, one line per game; (b) teacher − O sorted, dashed: mean.
Figure S24: Session 1 against session 2 in v18, per game: (a) teacher − O; (b) N − O. Dashed: equality.
Figure S25: T0, teacher, O and N in 20 of the v18 games, from the smallest to the largest quality component of O.
Figure S26: Quality components in live play (v18, host 2, 100 games) and in the synchronous emulator on host-1 schedules (v17 rerun, 29 games); different games.
Figure S27: Accuracy against disagreement with the teacher per v18 episode, in quartiles of the skipped-tick share; colours as in Figure 7 .
Figure S28: Per-episode skipped-tick share and hit accuracy in defend_the_center, same 100 seeds on both hosts (v6b, v12). On host 1 the arms that wait for a server skip 5–12% of ticks in 90% of episodes (5th–95th percentile), with single episodes up to 56%; the zero-latency teacher and every arm on host 2 skip below 0.4%. Note the different horizontal ranges: the hosts differ by two orders of magnitude.
Figure S29: Episode-level distributions of three metrics by arm and host (defend_the_center, v6b and v12; boxes: quartiles; whiskers: 1.5 interquartile ranges; outliers hidden). On host 2 the latency-matched teacher S is indistinguishable from T0 on all three metrics, while the ordering S > N > O in accuracy and kills persists.
Figure S30: Episode rewards in deadly_corridor (v11 on both hosts, v15 on host 2); diamonds: episode means. Episode rewards vary far more between games than between arms, which is why the game bootstrap intervals of this scene are wide (Table 5 ).
Figure S31: Decision time: distribution over episodes of the median decision time (log scale) for the local decider on host 2 and the latency-matched teachers on both hosts, which wait for a real call to the decider’s server.
Figure S32: Confidence of the original deciders on all held-out teacher states. Left: agreement with the teacher by confidence bin (bins with at least 20 states); the dotted line is perfect calibration. Right: agreement on the states the decider would handle alone when it takes the most confident ones first; the remaining states would be handed to a slower judge (Table S10 ).
Scene and threshold
Automated (%)
Agreement (%)
Handed off (%)
defend_the_center (30,385 states)
none
100.0
96.1
0.0
≥ 0.8
95.1
97.8
4.9
≥ 0.9
90.8
98.4
9.2
≥ 0.95
84.8
98.8
15.2
≥ 0.99
63.4
99.4
36.6
Appendix
Table S10: Confidence thresholds on the validation split (all states). Automated: share of states at or above the threshold; agreement: teacher agreement on those states; handed off: the rest.
ID
Study
Host
Scene
Episodes
Frozen text
Verdict
S1/E1
first real-time demonstration: paused, tick-converted and asynchronous play
1
defend
320
yes
see text
E2
competing methods (hybrid, action queue, language models) on one real-time ruler
1
defend
1,365
amended
see text
v3
asynchronous replication of teacher and decider
1
defend
210
yes
–
v4
latency-matched decomposition, three pairs (episodes in v4b: its first three pairs)
1
defend
0
yes
–
v4b
extension to five pairs; run- and game-level intervals
1
defend
300
yes
claim of quality component < 5 pp allowed: no
v5
on-policy asynchronous DAgger, five triads
1
defend
450
yes
partial success
Appendix
Table S11: All studies. Host 1: Windows desktop; host 2: Linux workstation. Scene: defend = defend_the_center, corridor = deadly_corridor. Verdict: the preregistered verdict keys stored in each analysis file.
ID
Content
Evaluation seeds
Hash
v3
zero-latency teacher (T0) and decider, asynchronous replication
850000–850029
299dafd18395
v4
latency-matched decomposition, three pairs
850000–850029
d4c13a3bcfce
v4b
extension to five pairs; run-level and game-level intervals
850000–850029
a317edf316e7
v5
on-policy asynchronous distillation, five triads
850000–850029
404982264335
v6a
calibration of core-pinned concurrent evaluation
850000–850029
b96753c944f2
v6b
replication with contemporaneous T0 and seed-32 replicate
860000–860099
1ab89caac991
Appendix
Table S12: Preregistrations, seeds and frozen hashes.
Line A
Line B
Line A
δ (ms)
teacher
teacher ± 50
D
teacher
D
D, solo
0
0.03 (0.0)
0.03 (0.0)
0.88 (0.0)
0.03 (0.0)
0.88 (0.0)
–
200
0.03 (0.0)
0.03 (0.0)
0.88 (0.0)
0.03 (0.0)
0.88 (0.0)
–
340
0.03 (0.0)
0.85 (9.6)
1.37 (7.3)
0.03 (0.0)
0.90 (0.2)
0.92 (0.2)
360
0.03 (0.0)
2.06 (30.4)
1.83 (13.1)
0.03 (0.0)
2.99 (31.2)
0.88 (0.4)
370
0.03 (0.0)
3.16 (40.0)
1.25 (6.0)
0.03 (0.0)
1.38 (6.7)
0.88 (0.3)
Appendix
Table S13: LineTempo, asynchronous mode: mean cost per part and, in parentheses, late decisions (%), by injected delay. Teacher: exact teacher; teacher ± 50: with ± 50 ms uniform jitter; D: the decider over live HTTP, four concurrent episodes; D, solo: exploratory rerun of D one episode at a time (5 seeds). Other cells: 10 episodes of 500 parts.
Figure S33: (a) ViZDoom from the player’s view, with an opposite-turn error of Supplementary Figure S34 (gray arrow teacher, orange arrow decider). (b) The conveyor simulator of Section 6.8 , a frame of a logged 50-part run of the decider on line B. A camera inspects each part; a diverter 0.4 m downstream (400 ms at 1 m/s) sorts it.
Figure S34: First observed opposite-turn errors of the original decider, one per game: the teacher turns toward the nearest enemy and the decider turns away. Key: gray arrow teacher, orange arrow decider, white circle the nearest enemy; in the state sentence, a is the angle bin in degrees and d the distance bin in 100-unit steps. Illustrative, not a measurement: the first opposite-turn decision with ammunition left in each of the first six games from seed 860000, under synchronous replay with 10% i.i.d. skipped ticks. The frame is shown for the reader; the decider itself received only the state sentence printed below it.
Delay (ms)
T0
S
O
S skipped (%)
O skipped (%)
0
61.6
61.6
63.0
1.6
1.6
30
61.6
89.4
91.5
48.6
48.6
60
61.6
82.4
84.5
63.1
63.1
Appendix
Table S14: Injected-delay pilot, deadly_corridor, host 2 (exploratory). T0 has no server call and is the same at every delay.
Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls take time. In continuously changing environments, an otherwise correct action may arrive too late. We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference. Paired evaluations show that GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running. We propose Two-Timescale Offline Recursive Self-Improvement (TORSI): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted. Two rounds raise Jev's average test score from 13.51 to 24.19 on a 0--100 progress scale, improving all six game types. The instructions also improve average scores for three other models without retuning. These findings point toward real-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales.
Ningxin Pan, Hanyu Li, Yeqing Shen +3
Peking University · Samsung Research, Beijing, China
Deliberating takes time. In real-time settings, that time is not free. Standard reinforcement learning (RL) sidesteps this as the environment waits indefinitely for the agent's decision. Instead, we study real-time RL environments where the environment progresses while waiting for the agent's action. Building on prior real-time formalizations, we introduce variable-delay real-time RL, where the agent chooses how long to deliberate at each decision point since the environment progresses. For the planning agents we use, the right delay is state-dependent, and naively planning how long to plan can paralyze the agent. We instead approach this setting by training a lightweight gating policy on top of a planner to select state-dependent planning budgets. Across real-time Pac-Man, Tetris, Snake, Speed Hex, and Speed Go, our gating policy outperforms fixed-budget and heuristic baselines, and transfers to a real-time setup where the environment and agent run on two different GPUs.
Aneesh Muppidi, Firas Darwish, Dylan Cope +2
British Open-ended Learning and Discovery Lab (BOLD), University of Oxford · Department of Statistics, University of Oxford · VGG, University of Oxford
Real-time interaction requires agents to perceive evolving environments and act under temporal constraints, yet existing evaluations of real-time streaming models focus on conversational quality and streaming perception, leaving effective action under strict temporal constraints untested. We introduce Realtime-Bench, a suite of carefully curated dynamic tasks where delayed actions directly affect success. Under Wall Clock, where environments advance according to actual inference latency, the best success rate reaches only 44% and the best Pass^3 is only 36%, demonstrating that real-time interaction remains challenging. To separate serving latency from models' real-time streaming capability, we compare Wall Clock with Paused Clock, which removes inference delays as an idealized reference, and introduce Token Clock, which advances simulated time according to generated token counts, testing temporal awareness independently of physical serving speed. We find that temporal costs limit success in evolving environments and that higher reasoning effort has mixed effects under time pressure, reflecting a trade-off between decision quality and timely execution. Based on these results, we discuss future directions for improving real-time multimodal models, including adaptive reasoning with temporal awareness and continuous perception and action.
Jingyu Tang, Xu Yang, Zhehao Zhang +7
University of Texas at Austin · The Ohio State University · University of Illinois at Urbana-Champaign +3