Old Is Not False: Valid-Time Memory for Interactive Video World Models
Organizations: The Hong Kong University of Science and Technology · Beihang University · Monash University · Sun Yat-sen University ·
Abstract
Interactive video world models draw each frame from a memory bank of earlier frames stamped with their capture times. With frames, weights and noise fixed, reversing the timestamps of two views flips the state that generators of two families draw: they take the newest timestamp as the truth. That reading is right as long as states never return. After an agent undoes an action, the newest view of a region is outdated and the only view of the present is older; memories for changing worlds delete that older view, and memories that keep it hand the generator the past. Temporal databases separate when a record is stored from when its content holds, its valid time. ReDate keeps every frame and dates each retrieved token by its valid time, read from the agent's action log. The frozen generator then reads a reverted region as one that has just reached a new state, so ReDate needs no training. We prove that trusting the newest timestamp is right exactly when every current view is dated after every outdated one, that without a view taken after the return only the action log tells a reverted region from an up-to-date one, and that a generator able to draw from its own prediction gains more from re-dating than from deletion when the moved tokens keep little weight. After an undo, ReDate raises the share of FAR's steps showing the current state from 0.36 to 0.62 and HY-WorldPlay's from 0.003 to 0.19. We also propose a calibrated state metric to test any memory after undos.
Figures & tables
| Memory | Up to date | Reverted | Crossing | Other poses |
|---|---|---|---|---|
| Deletion | ✓ | ✗ | ½ | ✗ |
| Shared time or none | ½ | ½ | ½ | ✗ |
| Capture order | ✓ | ✗ | ½ | ✗ |
| Sink at the present | ½ | ½ | ½ | ✗ |
| State filtering | (✓) | (✓) | ✗ | ✗ |
| ReDate per frame | ✓ | (✓) | ½ | ✗ |
| Reverted steps | All steps | |||||||||
| Memory rule | Acc.↑ | At 0.2↑ | Seen↑ | Unseen↑ | Outd.↓ | Low q.↓ | LPIPS acc.↑ | PSNR↑ | Repeat↑ | New↑ |
| FAR [ 26 ] | 0.36 | 0.32 | 0.54 | 0.05 | 0.27 | 0.22 | 0.62 | 36.5 | 0.58 | 0.43 |
| View-overlap retrieval [ 75 ] | 0.08 | 0.08 | 0.12 | 0.01 | 0.03 | 0.88 | 0.14 | 22.5 | 0.09 | 0.01 |
| Geometric retrieval [ 66 ] | 0.03 | 0.01 | 0.03 | 0.02 | 0.09 | 0.82 | 0.20 | 23.3 | 0.26 | 0.23 |
| Decay by age [ 82 ] | 0.40 | 0.37 | 0.61 | 0.06 | 0.24 | 0.19 | 0.64 | 37.5 | 0.60 | 0.44 |
| Re-timing in the true order [ 52 ] | 0.26 | 0.23 | 0.39 | 0.06 | 0.36 | 0.24 | 0.53 | 35.7 | 0.53 | 0.44 |
| Recalled frames | Outdated tokens | Acc.↑ | At 0.2↑ | LPIPS acc.↑ | PSNR↑ | Outd.↓ | Low q.↓ | New↑ |
|---|---|---|---|---|---|---|---|---|
| capture time | kept | 0.39 | 0.37 | 0.65 | 37.3 | 0.25 | 0.19 | 0.44 |
| capture time | hidden | 0.52 | 0.48 | 0.70 | 34.0 | 0.01 | 0.39 | 0.40 |
| capture time | moved before the oldest frame | 0.58 | 0.51 | 0.82 | 39.4 | 0.10 | 0.12 | 0.45 |
| valid time | kept | 0.58 | 0.53 | 0.80 | 39.6 | 0.15 | 0.10 | 0.45 |
| valid time | hidden | 0.56 | 0.53 | 0.70 | 35.0 | 0.01 | 0.36 | 0.41 |
| valid time, ReDate (ours) | moved before the oldest frame | 0.62 | 0.55 | 0.85 | 39.7 | 0.09 | 0.09 | 0.44 |
Appendix figures & tables47 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Meaning |
|---|---|
| , | regions and their states |
| the state of region at time | |
| the action log | |
| an observation of at , | |
| , | a history and the query time |
| the current state of |
| Result | Assumes | Tested by | Lean |
|---|---|---|---|
| Lemma 1 , group actions reveal reversions | Assumption 1 | the state tracker on receptacles and doors ( Sec. B.1 ) | |
| Theorem 1 , keep | none | deletion-based memories at 0.02 to 0.14 on reverted steps | ✓ |
| Lemma 2 , timestamp | Assumption 1 | one time for every frame, current state from 0.51 to 0.30; HY-WorldPlay with both views at one position, either state on 16 of 218 steps | ✓ |
| Theorem 2 , valid time | 1 and 2 | ReDate at 0.94 where the target pose stored the current state; reversed capture timestamps trade new-state steps for reverted ones; HY-WorldPlay from 1 to 70 current renders as two positions exchange | ✓ |
| Corollary 1 , no return in the input | 1 and 2 | ReDate at 0.94 on seen reverted steps against FAR’s 0.88 on seen new-state steps | ✓ |
| Lemma 3 , re-timing | Assumption 1 | re-timing in the true order at 0.26; the sink at 0.52 with new-state steps at 0.21 | first part |
| Region | Value | Before its first action | Action log | Group of Lemma 1 |
|---|---|---|---|---|
| Spot where an item first stood | holds the item, or empty | holds its item | empty from the first pickup of the item | none, see the text |
| Receptacle | the items the agent put on it and has not taken away | no item put by the agent | a put adds the held item and a pickup removes it | subsets of items under symmetric difference |
| Door | open or closed | the state opposite to its first action | opening and closing | the group of order two |
| Action log | Re-dated views | Precision | Views to re-date | Recall | Views marked outdated | Disagreeing |
|---|---|---|---|---|---|---|
| Object names | 12,713 | 0.62 | 10,702 | 0.56 | 43,441 | 0.88 |
| Verb and place only | 11,358 | 0.54 | 10,702 | 0.47 | 42,686 | 0.86 |
| Constant | Value | Role | Set |
|---|---|---|---|
| Reach | 1 m | the location of a manipulated region along the optical axis | for post-action retrieval, before re-dating |
| Token radius | 0.8 m | the disc whose cells are a region’s tokens | with the transport rules of Sec. D.8 |
| Holding range | 8 m | the farthest region a stored frame holds | before any run |
| Settle time | 2 s | restored frames after the restoring action, post-action frames after the action, outdated tokens before the oldest frame | for post-action retrieval, before re-dating |
| Window | 3 s | the search window of post-action retrieval, the reveal window of FAR’s analysis | for post-action retrieval |
| Recalled frames | 2 and 2 | re-dated frames, and post-action frames from the last two interactions | the second chosen between one and two on subset A, the first following it |
| Design | Adapted from | Reads | Change from the published rule |
|---|---|---|---|
| Latest 12 frames | FAR’s temporal baseline [ 26 ] | capture order | the memory bank is cut to the context, where the released baseline is a checkpoint of its own |
| FAR | FAR [ 26 ] | capture times | none, the system as released |
| View-overlap retrieval | Context as Memory [ 75 ] | poses | FAR’s geometric retriever with its recency term at zero, under the FAR generator |
| Geometric retrieval | WorldMem [ 66 ] | poses, capture times | the WorldMem baseline of FAR’s release, and its retriever under the FAR generator |
| Decay by age | Mem-World [ 82 ] | capture times | the factor ranks FAR’s candidates again, not candidates of its own retriever |
| Re-timing in the true order | WorldPlay [ 52 ] | capture order | the retrieved frames keep their order and take evenly spaced timestamps up to the newest, not a fixed distance |
| Condition | Timestamps the generator reads | Keeps |
|---|---|---|
| Kept | order and gaps | |
| One time | the median of the set, for every retrieved frame | neither |
| Permuted | a permutation of the set’s timestamps | the values |
| Reversed | the gaps, in reverse | |
| Equal gaps | evenly spaced in the true order between and | the order |
| Gaps times | , for and | the order |
| Setting | Value |
|---|---|
| Checkpoints | step 700k on AI2-THOR, step 500k on AI2-THOR-dyn |
| Rollout | stride of four frames, context of 12 frames, 20 DDIM steps |
| Memory | store of 1,000 exploration frames, retrieval chunk of 10; the memory bank’s history stays real |
| Labels | interaction labels fed as in FAR |
| Hardware | NVIDIA H100 GPUs, and an NVIDIA L20X GPU for the HY-WorldPlay probe |
| AI2-THOR-dyn | FAR’s both-ends probe, five epochs, on trajectories whose second agent stands at exactly one corridor end |
| Runs | Clips | Seed |
|---|---|---|
| Audit of the readings ( Sec. C.2 ) | 400 test clips, and 200 held-out audit clips | |
| Timestamp conditions | subsets A and B | 0 |
| Timestamp conditions | the 249 clips with a repeat-state step | 1 |
| Timestamp conditions | the clips of shards 0 to 15 of the complete index | 0 |
| Memory designs, ReDate and its parts, the stores | the 249 clips with a repeat-state step | 1 |
| The stores | subsets A and B | 0 and 1 |
| Input whose answer is known | Evaluator’s LPIPS accuracy | Two-way reading | State reading | Threshold 0.2 | Threshold 0.1 |
|---|---|---|---|---|---|
| Simulator, current state | 0.99 | 1.00 | 0.95 | 0.96 | 0.95 |
| Simulator, outdated state | 0.05 to 0.16 | 0.08 | 0.01 | 0.01 | 0.01 |
| Simulator, another clip | 0.00 | 0.37 | 0.00 | 0.00 | 0.00 |
| Simulator, tokens erased | 0.02 to 0.17 | 0.34 to 0.37 | 0.00 | 0.00 | 0.00 |
| Even mixture, a ghost | 0.98 | 0.70 | 0.43 | 0.19 | 0.02 |
| FAR, current views alone | 0.80 | 0.83 | 0.68 | 0.65 | 0.63 |
| Reverted | Seen | Unseen | New-state | |||||
| Candidate | LPIPS current↑ | LPIPS outd.↓ | pixel current↑ | pixel outd.↓ | nearer current↑ | LPIPS current↑ | LPIPS current↑ | LPIPS current↑ |
| FAR [ 26 ] | 0.32 | 0.30 | 0.42 | 0.38 | 0.52 | 0.45 | 0.12 | 0.44 |
| ReDate (ours) | 0.55 | 0.05 | 0.71 | 0.10 | 0.83 | 0.75 | 0.22 | 0.46 |
| ReDate per frame | 0.53 | 0.09 | 0.66 | 0.16 | 0.77 | 0.73 | 0.19 | 0.46 |
| State filtering [ 71 ] | 0.37 | 0.01 | 0.59 | 0.02 | 0.86 | 0.58 | 0.02 | 0.38 |
| Oracle token deletion† [ 69 ] | 0.35 | 0.11 | 0.48 | 0.14 | 0.67 | 0.53 | 0.05 | 0.42 |
| Run | Frames | Post | Acc.↑ | Outd.↓ | Low q.↓ | New↑ |
|---|---|---|---|---|---|---|
| ReDate (ours) | 2 | ✓ | 0.62 | 0.09 | 0.09 | 0.44 |
| ReDate per frame | 2 | ✓ | 0.58 | 0.15 | 0.10 | 0.46 |
| FAR + valid time + two recalled frames | 2 | 0.57 | 0.17 | 0.11 | 0.46 | |
| Re-dating from ground truth† | 2 | 0.45 | 0.24 | 0.17 | 0.44 | |
| FAR + valid time | 0 | 0.55 | 0.19 | 0.11 | 0.46 | |
| one time for every frame | 2 | ✓ | 0.21 | 0.44 | 0.26 | 0.33 |
| Recalled frames | Outdated tokens | Acc.↑ | At 0.2↑ | LPIPS acc.↑ | PSNR↑ | Outd.↓ | Low q.↓ | New↑ |
|---|---|---|---|---|---|---|---|---|
| capture time | hidden | 0.56 | 0.54 | 0.72 | 34.5 | 0.01 | 0.37 | 0.40 |
| valid time | hidden | 0.59 | 0.58 | 0.73 | 35.4 | 0.01 | 0.34 | 0.41 |
| valid time, ReDate (ours) | moved before the oldest frame | 0.64 | 0.61 | 0.87 | 39.6 | 0.06 | 0.12 | 0.44 |
| Comparison | |||||||
| Row | Steps | Current | Outd. | Ambig. | Low q. | Difference | Clips +/− |
| ReDate (ours) , development clips | 64 | 0.828 | 0.062 | 0.078 | 0.031 | +0.359 | 13/0 |
| FAR + valid time | 64 | 0.750 | 0.141 | 0.047 | 0.062 | +0.281 | 10/0 |
| + two recalled frames | 64 | 0.797 | 0.078 | 0.062 | 0.062 | +0.328 | 11/0 |
| + post-action retrieval, ReDate per frame | 64 | 0.812 | 0.078 | 0.062 | 0.047 | +0.344 | 12/0 |
| FAR + valid time, read by the retriever too | 64 | 0.750 | 0.125 | 0.062 | 0.062 | +0.281 | 10/0 |
| Reverted | Repeat-state | New-state | ||||||||
| Memory | Current↑ | Outd.↓ | Low q.↓ | Current↑ | Outd.↓ | Low q.↓ | Current↑ | Outd.↓ | Low q.↓ | Clips +/− |
| Seed 0, 249 clips | ||||||||||
| Latest 12 frames | 0.003 | 0.000 | 0.995 | 0.008 | 0.000 | 0.991 | 0.001 | 0.000 | 0.997 | 1/75 |
| FAR [ 26 ] | 0.380 | 0.284 | 0.186 | 0.543 | 0.209 | 0.160 | 0.416 | 0.101 | 0.267 | ref. |
| View-overlap retrieval [ 75 ] | 0.060 | 0.019 | 0.896 | 0.079 | 0.010 | 0.872 | 0.007 | 0.067 | 0.869 | 6/70 |
| Frame deletion by pose [ 64 ] | 0.109 | 0.254 | 0.219 | 0.477 | 0.112 | 0.181 | 0.433 | 0.079 | 0.261 | 9/58 |
| Room type | Rooms | Clips | Reverted steps | FAR | ReDate | State filtering | Hidden, valid time | Seen steps | FAR | ReDate |
|---|---|---|---|---|---|---|---|---|---|---|
| living rooms | 5 | 47 | 100 | 0.38 | 0.78 | 0.70 | 0.65 | 78 | 0.46 | 0.96 |
| bedrooms | 5 | 26 | 102 | 0.33 | 0.61 | 0.45 | 0.59 | 61 | 0.56 | 0.98 |
| bathrooms | 6 | 59 | 119 | 0.40 | 0.61 | 0.45 | 0.55 | 74 | 0.62 | 0.89 |
| kitchens | 4 | 9 | 31 | 0.16 | 0.16 | 0.16 | 0.23 | 6 | 0.33 | 0.67 |
| All | Held-out | |||||||
|---|---|---|---|---|---|---|---|---|
| Memory | memory | difference | interval | clips +/− | memory | difference | interval | clips +/− |
| FAR [ 26 ] | 0.355 | +0.264 | 0.199 to 0.334 | 54/2 | 0.330 | +0.243 | 0.164 to 0.318 | 41/2 |
| ReDate per frame | 0.580 | +0.040 | 0.013 to 0.071 | 11/1 | 0.528 | +0.045 | 0.016 to 0.084 | 9/0 |
| Token deletion, ReDate ’s log, valid time | 0.562 | +0.057 | 0.022 to 0.096 | 16/4 | 0.531 | +0.042 | 0.006 to 0.083 | 11/4 |
| Token deletion, ReDate ’s log, capture time | 0.523 | +0.097 | 0.058 to 0.141 | 27/3 | 0.472 | +0.101 | 0.056 to 0.146 | 23/2 |
| State filtering [ 71 ] | 0.497 | +0.122 | 0.071 to 0.179 | 31/4 | 0.434 | +0.139 | 0.083 to 0.205 | 27/2 |
| Reverted | Repeat-state | New-state | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Memory | Current | Outd. | Low q. | Current | Outd. | Low q. | Current | Outd. | Low q. | Clips +/− |
| FAR [ 26 ] | 0.355 | 0.267 | 0.219 | 0.575 | 0.184 | 0.162 | 0.434 | 0.103 | 0.252 | ref. |
| FAR with ReDate per frame | 0.580 | 0.151 | 0.099 | 0.769 | 0.084 | 0.071 | 0.458 | 0.083 | 0.240 | 48/4 |
| FAR with ReDate | 0.619 | 0.085 | 0.094 | 0.812 | 0.037 | 0.063 | 0.442 | 0.089 | 0.259 | 54/2 |
| FAR generator, geometric retrieval | 0.026 | 0.091 | 0.818 | 0.265 | 0.030 | 0.651 | 0.233 | 0.029 | 0.703 | 3/71 |
| with target-pose frames and post-action retrieval | 0.381 | 0.222 | 0.233 | 0.608 | 0.130 | 0.178 | 0.383 | 0.086 | 0.379 | 69/2 |
| Reverted | Repeat-state | New-state | ||||
|---|---|---|---|---|---|---|
| Memory bank | Current↑ | Outd.↓ | Low q.↓ | Current↑ | Current↑ | Clips +/− |
| FAR, real history | 0.355 | 0.267 | 0.219 | 0.575 | 0.434 | ref. |
| FAR, generated frames at render time | 0.352 | 0.287 | 0.216 | 0.529 | 0.430 | 9/11 |
| FAR, generated frames at evidence time | 0.349 | 0.264 | 0.230 | 0.516 | 0.429 | 9/13 |
| ReDate (ours) , real history | 0.619 | 0.085 | 0.094 | 0.812 | 0.442 | ref. |
| ReDate , generated frames at render time | 0.611 | 0.088 | 0.099 | 0.800 | 0.444 | 5/8 |
| Comparison | ||||||||
|---|---|---|---|---|---|---|---|---|
| Row | Steps | Current | Outd. | Ambig. | Low q. | LPIPS acc. | Difference | Clips +/− |
| FAR, all steps | 18,006 | 0.493 | 0.102 | 0.211 | 0.194 | 0.761 | ref. | ref. |
| ReDate per frame, all steps | 18,006 | 0.523 | 0.084 | 0.217 | 0.176 | 0.787 | +0.030 | 218/88 |
| ReDate (ours) , all steps | 18,006 | 0.523 | 0.075 | 0.219 | 0.184 | 0.781 | +0.030 | 233/122 |
| FAR, clips outside the first 200 | 15,607 | 0.488 | 0.105 | 0.222 | 0.185 | 0.754 | ref. | ref. |
| ReDate per frame, clips outside the first 200 | 15,607 | 0.510 | 0.087 | 0.228 | 0.175 | 0.777 | +0.022 | 167/79 |
| Benchmark | Measure | System | Reported | Our rerun |
|---|---|---|---|---|
| FAR’s changing rooms, 1,309 clips | surface accuracy | Temporal baseline [ 26 ] | 0.057 | 0.129 |
| WorldMem baseline [ 66 ] | 0.421 | 0.420 | ||
| FAR [ 26 ] | 0.802 | 0.802 | ||
| ReDate (ours) | 0.824 | |||
| container accuracy | FAR [ 26 ] | 0.757 | 0.760 | |
| ReDate (ours) | 0.817 |
| Memory | Clip set | Surface | Container | Clips +/− |
| FAR [ 26 ] | complete index | 0.802 | 0.705 | ref. |
| FAR + post-action retrieval | complete index | 0.817 | 0.705 | 259/179 |
| ReDate (ours) | complete index | 0.824 | 0.754 | 312/214 |
| FAR, released analysis | complete index | 0.760 | ||
| ReDate , released analysis | complete index | 0.824 | 0.817 | |
| FAR + post-action retrieval | subset A | 0.829 |
| Subset A | Subset B | A at 0.2 | LPIPS acc. | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Timestamps given to the FAR generator | Current | Outdated | PSNR | Current | Outdated | Current | Outdated | A | B | Corridor |
| Kept | 0.510 | 0.118 | 42.0 | 0.457 | 0.107 | 0.479 | 0.152 | 0.787 | 0.788 | 0.953 |
| Equal gaps, true order | 0.524 | 0.124 | 41.0 | 0.473 | 0.115 | 0.485 | 0.145 | |||
| Gaps times 0.1, true order | 0.317 | 0.380 | 39.6 | 0.343 | 0.323 | 0.314 | 0.480 | 0.600 | 0.621 | |
| Gaps times 4, true order | 0.225 | 0.048 | 29.4 | 0.203 | 0.021 | 0.155 | 0.043 | 0.508 | 0.452 | |
| One time | 0.297 | 0.340 | 37.5 | 0.309 | 0.283 | 0.274 | 0.438 | 0.576 | 0.574 | 0.909 |
| Reverted | Repeat-state | New-state | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Memory | Current | Outd. | Low q. | Current | Outd. | Low q. | Current | Outd. | Low q. | Clips +/− |
| FAR [ 26 ] | 0.355 | 0.267 | 0.219 | 0.575 | 0.184 | 0.162 | 0.434 | 0.103 | 0.252 | ref. |
| one time for every frame | 0.355 | 0.247 | 0.233 | 0.513 | 0.181 | 0.216 | 0.269 | 0.253 | 0.352 | 31/30 |
| timestamps reversed | 0.537 | 0.148 | 0.170 | 0.589 | 0.146 | 0.189 | 0.034 | 0.476 | 0.436 | 44/19 |
| ReDate per frame | 0.580 | 0.151 | 0.099 | 0.769 | 0.084 | 0.071 | 0.458 | 0.083 | 0.240 | 48/4 |
| ReDate (ours) | 0.619 | 0.085 | 0.094 | 0.812 | 0.037 | 0.063 | 0.442 | 0.089 | 0.259 | 54/2 |
| Capture | ReDate | ||||||
|---|---|---|---|---|---|---|---|
| Generator and memory | Time enters as | Sensitivity | Cur. | Outd. | Cur. | Outd. | Gain in log-odds |
| WorldMem baseline [ 66 ] | timestamp embedding | 0.63 | 0.40 | 0.07 | 0.43 | 0.05 | 0.56 |
| FAR generator, geometric retrieval | timestamp embedding | 1.21 | 0.38 | 0.22 | 0.60 | 0.09 | 1.37 |
| FAR [ 26 ] | timestamp embedding | 1.81 | 0.39 | 0.25 | 0.62 | 0.09 | 1.53 |
| HY-WorldPlay [ 52 ] | rotary positions | 3.45 | 0.00 | 0.28 | 0.19 | 0.09 | 5.32 |
| Reverted | Repeat-state | New-state | ||||
| Generator and memory | Current↑ | Outd.↓ | Low q.↓ | Current↑ | Current↑ | |
| FAR | ||||||
| capture timestamps [ 26 ] | 0.355 | 0.267 | 0.219 | 0.29 | 0.575 | 0.434 |
| ReDate ’s context, capture timestamps | 0.392 | 0.250 | 0.188 | 0.45 | 0.646 | 0.440 |
| ReDate (ours) | 0.619 | 0.085 | 0.094 | 1.98 | 0.812 | 0.442 |
| restored frames 0 s after the action | 0.616 | 0.088 | 0.094 | 1.95 | 0.809 | 0.442 |
| At 0.2 | State metric | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| History | Positions | current | outdated | neither | Nearer current | current | outdated | low q. | PSNR |
| Deletion | B | 0.000 | 0.266 | 0.734 | 0.225 | 0.000 | 0.193 | 0.771 | 25.9 |
| Capture time | A, then B | 0.005 | 0.326 | 0.670 | 0.193 | 0.014 | 0.248 | 0.702 | 27.0 |
| One time | A and B together | 0.014 | 0.060 | 0.927 | 0.404 | 0.051 | 0.142 | 0.578 | 28.8 |
| Valid time, ReDate (ours) | B, then A | 0.321 | 0.023 | 0.656 | 0.899 | 0.408 | 0.041 | 0.477 | 29.9 |
| Filtering | A | 0.367 | 0.000 | 0.633 | 0.959 | 0.367 | 0.000 | 0.569 | 28.5 |
| Capture order | Valid time | ||||
|---|---|---|---|---|---|
| Offset | current | outdated | current | outdated | Difference of |
| 0 (one time) | 0.014 | 0.060 | 0.014 | 0.060 | 0.0 |
| 1 | 0.028 | 0.262 | 0.289 | 0.060 | 3.7 |
| 2 | 0.009 | 0.440 | 0.390 | 0.050 | 5.7 |
| 4 (one chunk) | 0.005 | 0.326 | 0.321 | 0.023 | 6.4 |
| At 0.2 | State metric | ||||||
| History | current↑ | outdated↓ | neither | Nearer current↑ | current↑ | low q.↓ | PSNR↑ |
| Seed 1, all 352 reverted steps | |||||||
| capture time | 0.003 | 0.281 | 0.716 | 0.19 | 0.003 | 0.713 | 27.1 |
| state filtering, ReDate ’s log | 0.173 | 0.020 | 0.807 | 0.84 | 0.190 | 0.767 | 24.0 |
| token deletion, ReDate ’s log | 0.233 | 0.028 | 0.739 | 0.76 | 0.256 | 0.713 | 25.7 |
| ReDate (ours) | 0.188 | 0.091 | 0.722 | 0.63 | 0.213 | 0.625 | 28.0 |
| Memory reference | Probe reference | ||||||
|---|---|---|---|---|---|---|---|
| History | Positions | nearer G↑ | current at 0.2↑ | PSNR↑ | nearer G↑ | current at 0.2↑ | PSNR↑ |
| Deletion | B | 0.32 | 0.00 | 14.8 | 0.29 | 0.00 | 14.8 |
| Capture time | A, then B | 0.23 | 0.00 | 15.1 | 0.23 | 0.00 | 15.1 |
| One time | A and B together | 0.48 | 0.03 | 16.3 | 0.58 | 0.03 | 16.3 |
| Valid time, ReDate (ours) | B, then A | 0.61 | 0.06 | 16.2 | 0.68 | 0.06 | 16.3 |
| Filtering | A | 0.65 | 0.03 | 15.6 | 0.71 | 0.03 | 15.9 |
| Repeat-state | Reverted | New-state | ||||||||
| Store read by FAR | Current↑ | Outd.↓ | Low q.↓ | Readable↑ | At 0.2↑ | Current↑ | Low q.↓ | Current↑ | Outd.↓ | Low q.↓ |
| FAR [ 26 ] | 0.575 | 0.184 | 0.162 | 0.69 | 0.558 | 0.355 | 0.219 | 0.434 | 0.103 | 0.252 |
| + post-action retrieval | 0.659 | 0.137 | 0.116 | 0.75 | 0.637 | 0.412 | 0.176 | 0.434 | 0.087 | 0.258 |
| + decay by age | 0.602 | 0.163 | 0.153 | 0.71 | 0.590 | 0.401 | 0.193 | 0.436 | 0.093 | 0.254 |
| + post-action retrieval and decay | 0.656 | 0.135 | 0.124 | 0.75 | 0.639 | 0.426 | 0.173 | 0.437 | 0.083 | 0.255 |
| Frame deletion by pose [ 64 ] | 0.507 | 0.108 | 0.162 | 0.60 | 0.474 | 0.111 | 0.193 | 0.439 | 0.085 | 0.252 |
| Reverted | New-state | All | ||||
|---|---|---|---|---|---|---|
| Frame stored at the target pose | Seen↑ | Unseen↑ | Seen↑ | Unseen↑ | Reverted↑ | New-state↑ |
| FAR | 0.54 | 0.05 | 0.88 | 0.06 | 0.36 | 0.43 |
| ReDate (ours) | 0.94 | 0.10 | 0.90 | 0.06 | 0.62 | 0.44 |
| ReDate per frame | 0.89 | 0.07 | 0.93 | 0.07 | 0.58 | 0.46 |
| relay | 0.89 | 0.13 | 0.91 | 0.05 | 0.60 | 0.45 |
| relay, spliced | 0.86 | 0.11 | 0.91 | 0.06 | 0.58 | 0.45 |
| Comparison | FAR (s) | ReDate (s) | Peak memory |
|---|---|---|---|
| ReDate per frame | 0.443 | 0.452 | 6.1 GiB |
| ReDate (ours) | 0.436 | 0.472 | 6.1 GiB |
| Memory | A contradicted view | Signal | After the agent restores the region |
|---|---|---|---|
| PermaVid [ 69 ] | invalidated or masked | change times and masks | the view of the restored state is gone |
| WorldCraft [ 19 ] | invalidated | the agent’s edit | the view of the restored state is gone |
| Echo-Forcing [ 64 ] | decayed or removed | a later frame of the same pose | the view of the restored state is gone or weakened |
| ONEIRA [ 71 ] | withheld while it disagrees | a world-state table | a frame returns when every region of the target view agrees |
| MemWorld [ 3 ] | kept as depth only | an edit | the geometry returns without the appearance |
| StateCommit [ 4 ] | replaced by a committed symbolic state | the agent’s interactions | the symbolic state returns, bound to the predicted view |
| Pair | Actions | Undone | Median delay (s) | Undone within 30 s |
|---|---|---|---|---|
| Open, close | 5,680 | 0.58 | 4.9 | 0.89 |
| Turn on, turn off | 2,596 | 0.66 | 7.4 | 0.81 |
| Take, put | 16,785 | 0.58 | 5.4 | 0.87 |
Memory-Augmented Video Generation
Memory-augmented video generation gives video models persistent context to preserve identities, scenes, and events across long sequences.
See all 122 papers