ReRead: Wrong Predictions, Useful Representations - Correcting Frozen Robot Policies by Re-reading Their Own Representations
Organizations: Tsinghua University · Xi’an Jiaotong University
Abstract
When a pretrained robot policy fails on challenging tasks, common remedies include retraining the backbone, adding an external visual encoder, or introducing a memory module. We argue that wrong predictions do not imply wrong representations: a policy can retain the required information yet fail to read it out as the correct action. To test this view, we propose ReRead: with the backbone frozen, a corrector re-reads the current and two previous caches to predict an action residual. The corrector sees no images, is trained solely on offline demonstrations from all 50 RoboTwin 2.0 tasks, and adds only 4.17% latency. Ablations and interventions show that the correction relies on the backbone's own representations: none of four external visual encoders adds further gain, and swapping in past caches from other trajectories sharply increases action error. On the 11 hardest RoboTwin 2.0 tasks, ReRead improves the success rates of LingBot-VLA, , and Fast-WAM by 23.45, 13.09, and 8.00 percentage points, respectively. In paired evaluation on all 50 tasks, it raises Fast-WAM's average success rate from 90.73% to 92.66%. On a Flexiv dual-arm robot, it raises success on a phase-ambiguous task from 0% to 36%.
Figures & tables
| Model | Clean | Random | Average |
|---|---|---|---|
| LingBot-VLA [ 22 ] | 46.55 | 53.45 | 50.00 |
| ReRead-LingBot-VLA | 75.27 | 71.64 | 73.45 |
| [ 2 ] | 30.18 | 30.55 | 30.36 |
| ReRead- | 45.45 | 41.45 | 43.45 |
| Fast-WAM [ 23 ] | 76.73 | 70.18 | 73.45 |
| ReRead | 85.45 | 77.45 | 81.45 |
| Model | Clean | Random | Average |
|---|---|---|---|
| [ 2 ] | 42.98 | 43.84 | 43.41 |
| LingBot-VLA [ 22 ] | 86.50 | 85.34 | 85.92 |
| Motus [ 2 ] | 88.66 | 87.02 | 87.84 |
| HyVLA-0.5 [ 25 ] | 90.90 | 90.06 | 90.48 |
| GeoMem-VLA [ 26 ] | 91.10 | 91.24 | 91.17 |
| LingBot-VA [ 15 ] | 92.90 | 91.50 | 92.20 |
| Variant | Clean | Random | Average |
|---|---|---|---|
| ReRead | 85.45 | 77.45 | 81.45 |
| A. External vision and short-term dense history | |||
| DINOv3 [ 20 ] | 81.82 | 78.91 | 80.36 |
| C-RADIOv3 [ 9 ] | 83.27 | 76.36 | 79.82 |
| LingBot-Vision [ 5 ] | 80.00 | 78.91 | 79.45 |
| V-JEPA 2.1 [ 16 ] | 82.55 | 75.27 | 78.91 |
| Model | Clean | Random | Average |
|---|---|---|---|
| Simulation | |||
| Fast-WAM | 4.00 | 0.00 | 2.00 |
| ReRead | 48.00 | 52.00 | 50.00 |
| Real robot | |||
| Fast-WAM | 0.00 | 0.00 | 0.00 |
| ReRead | 36.00 | 36.00 | 36.00 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Configuration |
|---|---|
| Backbone / optimization scope | Frozen Fast-WAM / corrector only |
| Action interface | 20 denoising steps, replan after 20 executed steps, 32×14 action chunk |
| K/V source layers (0-based) | , visual prefill and last action denoising call |
| Corrector | Width 1536, 5 layers, 24 heads, FFN 6144, ReLU |
| Trainable parameters | 198,523,406 |
| Training budget | 16 GPUs, batch size 64 per GPU and 1024 globally, 20K steps |
| Configuration | Mean latency (ms) | Throughput (decisions/s) |
|---|---|---|
| Fast-WAM | 279.89 | 3.573 |
| ReRead | 291.55 | 3.430 |
| Item | Value |
|---|---|
| Resident past K/V memory | 9 MiB |
| Temporary K/V packing memory | 13.5 MiB |
| Compute per decision | 87.84 GFLOPs |
| Statistic | Value |
|---|---|
| Mean single-candidate MSE | 20.48 |
| MSE of the six-candidate mean | 19.76 |
| Sampling variance | 0.86 |
| Risk minus sampling variance | 19.62 |
| History condition | Clean/seen | Random/seen | Random/unseen |
|---|---|---|---|
| Native candidate | 26.561 | 12.867 | 13.219 |
| True history | 1.506 | 1.477 | 1.514 |
| Current only | 8.990 | 6.224 | 6.195 |
| Repeat current | 8.995 | 6.223 | 6.198 |
| Swap order | 5.618 | 5.387 | 5.347 |
| Other trajectory (same task) | 22.629 | 14.201 | 14.328 |
| History condition | Past candidate MSE | Current action MSE |
|---|---|---|
| Mask past K/V and states | — | 7.61 |
| Zero past K/V | — | 14.92 |
| Current K/V in past slots | — | 7.81 |
| Low-error history K/V | 14.39 | 2.13 |
| Medium-error history K/V | 18.72 | 2.13 |
| High-error history K/V | 24.40 | 2.73 |
| History error group | Past candidate MSE | Current only | True history | MSE reduction (%) |
|---|---|---|---|---|
| Low error | 1.18 | 9.73 | 1.81 | 81.40 |
| Medium error | 5.07 | 7.43 | 1.66 | 77.66 |
| High error | 45.37 | 5.79 | 1.04 | 82.04 |
| Both past candidates: high error | 44.62 | 4.05 | 0.83 | 79.51 |
| Fast-WAM | LingBot-VLA | |||||||||||
| Backbone only | ReRead | Backbone only | ReRead | Backbone only | ReRead | |||||||
| Task | C | R | C | R | C | R | C | R | C | R | C | R |
| open_microwave | 13 | 8 | 25 | 23 | 13 | 11 | 5 | 6 | 1 | 1 | 21 | 23 |
| hanging_mug | 10 | 19 | 16 | 10 | 0 | 0 | 5 | 1 | 5 | 6 | 10 | 9 |
| turn_switch | 19 | 18 | 16 | 15 | 6 | 5 | 13 | 11 | 10 | 16 | 16 | 17 |
| place_can_basket | 20 | 16 | 19 | 18 | 14 | 8 | 17 | 17 | 10 | 11 | 17 | 15 |
| LingBot-VLA | Motus | HyVLA-0.5 | GeoMem-VLA | LingBot-VA | Fast-WAM | Fast-WAM † | ReRead | |||||||||||
| Task | C | R | C | R | C | R | C | R | C | R | C | R | C | R | C | R | C | R |
| Adjust Bottle | 79 | 83 | 100 | 100 | 89 | 93 | 99 | 99 | 100 | 100 | 90 | 94 | 100 | 100 | 100 | 100 | 100 | 100 |
| Beat Block Hammer | 63 | 50 | 87 | 91 | 95 | 88 | 99 | 99 | 99 | 94 | 96 | 98 | 99 | 97 | 97 | 95 | 100 | 100 |
| Blocks Ranking RGB | 43 | 35 | 92 | 91 | 99 | 97 | 99 | 99 | 99 | 95 | 99 | 98 | 100 | 100 | 100 | 100 | 100 | 100 |
| Blocks Ranking Size | 8 | 14 | 66 | 73 | 75 | 63 | 93 | 94 | 87 | 86 | 94 | 96 | 94 | 98 | 91 | 91 | 96 | 94 |
| Click Alarmclock | 97 | 93 | 93 | 26 | 100 | 100 | 100 | 100 | 92 | 97 | 99 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
| History condition | Action MSE |
|---|---|
| Native candidate | 379.79 |
| True history | 149.73 |
| Current only | 250.64 |
| Cross-trajectory, same phase | 127.52 |
| Cross-trajectory, opposite phase | 521.52 |
| Same-scene, opposite phase | 519.45 |
Robot Policy Adaptation
Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.
See all 190 papers