Reflex: Fast Robot Policy Serving by Reusing and Correcting Past Actions
Organizations: University of Illinois at Urbana-Champaign · Cisco.
Abstract
Modern robot policies often perform expensive inference to generate action chunks, making serving a latency bottleneck for real-time control and a throughput bottleneck for multi-robot serving. Building on the success of caching in image, video, and audio generation, recent robot-policy advances have reported promising speedups by reusing intermediate denoising states to avoid redundant computation. Surprisingly, we find that such reuse often offers little advantage over simply running fewer denoising steps from fresh noise, without any cache. Moreover, accelerating denoising leaves the expensive policy backbone intact, which can account for 70% of GPU time. We present Reflex, a serving layer that turns past policy executions into a backbone-free path for future decisions. Rather than reusing intermediate computations, Reflex caches complete action chunks to bypass both the backbone and action head. This introduces two challenges: similar observations can demand different actions, and small action deviations (errors) in reuse can compound into execution stalls. Reflex addresses these through two-level cache correction. At the action level, it learns action-aware similarity from past policy decisions, assembles compatible chunks, and applies lightweight residual correction to limit action mismatch. At the execution level, it captures task progress and adaptively refreshes the original policy to limit accumulated errors and balance inference cost. Without modifying the underlying policy, Reflex improves inference latency by 1.4x to 11x and throughput by 2.0x to 2.2x on representative workloads and real robots, while preserving comparable task success.
Figures & tables
| Batch 1 (ms) | Batch 16 (ms) | |
|---|---|---|
| Vision encoder | 6.6 | 55.3 |
| Backbone | 17.9 | 252.4 |
| Action head, 10 steps | 26.3 | 53.0 |
| Action head, 1 step | 3.4 | 5.9 |
| Method | Policy | Reported | Re-Evaluated (ours) |
|---|---|---|---|
| VLA policies | |||
| ActionCache | 2.12 | 1.62 | |
| AdaVLA | 1.87 | 1.96 | |
| EfficientVLA | CogACT | 1.93 | 1.80 |
| VLA-Rolling | 1.30 | 1.19 | |
| rMuscle | 1.29 | 1.29 | |
| Library (episodes) | ||||
|---|---|---|---|---|
| Stage | 50 | 200 | 500 | 5,000 |
| Retrieval | 0.62 | 0.63 | 0.71 | 1.55 |
| Fusion | 0.13 | 0.13 | 0.14 | 0.15 |
| Correction | 0.36 | 0.36 | 0.38 | 0.50 |
| Stagnation check | 0.25 | 0.31 | 0.39 | 1.39 |
| Total | 1.36 | 1.43 | 1.61 | 3.58 |
| LIBERO | MT50 | |||
|---|---|---|---|---|
| Policy | Steps | Chunk | Steps | Chunk |
| GR00T N1.5 | ||||
| Diffusion Policy | ||||
| Fast-WAM | ||||
| Cosmos Policy | n/a | n/a | ||
| Setting | Successes | Hit rate (%) | Latency (ms) |
|---|---|---|---|
| Pure inference | 174/200 | n/a | 66.5 |
| Reflex | 172/200 | 79.3 | 21.8 |
| Setting | Diffusion Policy | |
|---|---|---|
| RT-Cache | 25.6/100/17.9 | 25.0/100/18.8 |
| Cache, Euclidean, nearest entry | 55.8/100/5.1 | 52.0/100/0.5 |
| + Mahalanobis distance | 62.8/100/5.2 | 60.2/100/0.5 |
| with CLIP keys | 40.8/100/0.8 | 39.6/100/0.8 |
| + Fusion of 16 entries | 77.0/100/5.2 | 76.8/100/0.5 |
| + Corrector | 84.6/100/5.2 | 78.6/100/0.5 |
| LIBERO-Plus | Unchanged | Adapted | + Stagnation | Calls (%) | |
|---|---|---|---|---|---|
| Spatial | 88.6 | 74.5 | 88.9 | 89.5 | 6.6 |
| Long | 71.3 | 51.7 | 62.1 | 65.7 | 9.5 |
| Both | 79.5 | 62.6 | 74.9 | 77.1 | 8.8 |
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Policy | Default | 1 step | 2 steps | More steps |
| LIBERO-Spatial | 98.8 (10) | 98.8 | 98.2 | ||
| GR00T N1.5 | 94.6 (8) | 93.2 | 91.8 | 3: 94.6 | |
| Diffusion Policy | 91.2 (100) | 5: 90.6, 10: 90.0 | |||
| Fast-WAM | 90.8 (10) | 90.4 | 91.8 | ||
| Cosmos Policy | 98.6 (5) | 97.4 | 97.4 | 3: 98.4 | |
| LIBERO-Long | 83.6 (10) | 82.4 | 82.8 | 4: 85.0 |
| LIBERO-Spatial | LIBERO-Long | |||||
|---|---|---|---|---|---|---|
| Policy (steps) | Default | Plain | Warm | Default | Plain | Warm |
| (2) | 98.6 | 99.6 | 92.6 | 84.4 | 84.8 | 77.8 |
| GR00T N1.5 (1) | 94.4 | 93.4 | 93.8 | 88.0 | 84.6 | 71.2 |
| GR00T N1.5 (2) | 94.4 | 93.6 | 93.8 | 88.0 | 84.2 | 81.8 |
| Diffusion Policy (10) | 79.6 | 79.2 | 79.6 | |||
| Fast-WAM (1) | 91.6 | 88.2 | 88.8 | |||
| Method | Policy, benchmark | In paper | Est. speedup |
|---|---|---|---|
| VLA policies | |||
| ActionCache | , VLABench | 2.12 | 1.48 to 1.62 |
| VLA-Rolling | , LIBERO | 1.30 | 1.19 |
| AdaVLA | , LIBERO | 1.87 | at most 1.96 |
| EfficientVLA | CogACT, SIMPLER | 1.93 | 1.80 |
| rMuscle | , LIBERO | 1.29 | 1.29 |
| Fastest prior acceleration | Plain inference | ||||
|---|---|---|---|---|---|
| Policy | Method | Est. speedup | Fewer steps | Fewer steps, longer horizon | Reflex |
| AdaVLA | 1.96 | 1.62 | 2.54 | 5.20 | |
| Fast-WAM | C 3 ache | 2.10 | 2.23 | 4.40 | 5.91 |
| Cosmos Policy | WAMachine | 2.36 | 2.51 | 2.51 | 3.38 |
| Diffusion Policy | Falcon | 12.69 | 8.62 | 17.01 | 72.23 |
| Policy | Key read from | Dimensions per camera |
|---|---|---|
| SigLIP image tokens, grid pooled to | 32,768 | |
| GR00T N1.5 | image tokens entering the language model, pooled to | 32,768 |
| Diffusion Policy | output of the ResNet-18 observation encoder | 64 |
| Fast-WAM | video tokens entering the transformer, pooled to | 49,152 ∗ |
| Cosmos Policy | VAE latent of the camera image, pooled to | 256 |
| Quantity | Fitted on | Episode left out |
| PCA bases | Keys of all library decisions | None |
| Metric | Key differences of action-matched pairs | The two entries of a pair come from different episodes |
| Corrector , | Residual of each library chunk from its cache estimate | The chunk’s own episode |
| Correction scale | Nearest-neighbor distances of library entries | The entry’s own episode |
| Bounds , | Windows of successful library episodes | The window’s own episode (Appendix C ) |
| Refresh interval | Replay of library episodes | The replayed episode (Appendix E ) |
| Library (episodes) | ||||||
|---|---|---|---|---|---|---|
| Part | 50 | 200 | 500 | 1,000 | 2,000 | 5,000 |
| Retrieval | 0.62 | 0.63 | 0.71 | 0.78 | 0.94 | 1.55 |
| PCA projection | 0.52 | 0.50 | 0.53 | 0.53 | 0.54 | 0.59 |
| Mahalanobis distance | 0.09 | 0.11 | 0.15 | 0.21 | 0.34 | 0.82 |
| Nearest entries | 0.01 | 0.02 | 0.03 | 0.04 | 0.07 | 0.15 |
| Fusion | 0.13 | 0.13 | 0.14 | 0.14 | 0.14 | 0.15 |
| Component | Constant | Default | Role |
| Retrieval and fusion | Fused entries | 16 | entries averaged into the assembled chunk |
| Scale neighbor | its distance sets the width of the fusion weights | ||
| Action-matched entries | 3 | entries per decision that form | |
| Regularization | 0.1 | keeps poorly sampled directions from dominating | |
| PCA dimensions | 64 per camera | size of the visual part of the key | |
| Corrector | Ridge penalty | 100 | shrinks the corrector weights |
| Diffusion Policy | Diffusion Policy | GR00T N1.5 | Fast-WAM | |
| Benchmark | LIBERO | MT50 | MT50 | MT50 |
| Initialization | LIBERO release | random | base model | LIBERO release |
| Trained parameters | all but text encoder | all but text encoder | action head | action expert |
| Updates | 100k | 100k | 60k | 16k |
| Batch size | 256 | 256 | 64 | 128 |
| Peak / final learning rate | / | / | / 0 | / |
| Diffusion Policy | ||||
|---|---|---|---|---|
| Library of 50 episodes | Default | Expanded | Default | Expanded |
| Random | 61.4 | 67.3 | 37.2 | 57.1 |
| Random, successful only | 64.7 | 69.9 | 52.1 | 75.3 |
| Farthest apart | 69.5 | 71.6 | 47.3 | 67.4 |
| Representative | 68.3 | 75.0 | 54.9 | 76.7 |
| Served policy | Library from | Stagnation check only | Full system |
|---|---|---|---|
| (56.3) | GR00T N1.5 (75.6) | 48.7 60.2 | 58.6 61.8 |
| (56.3) | Diffusion Policy (61.2) | 48.7 55.0 | 58.6 58.8 |
| Fast-WAM (56.5) | GR00T N1.5 (75.6) | 46.2 48.3 | 56.9 57.0 |
| Diffusion Policy (61.2) | (56.3) | 53.0 48.0 | 61.4 56.0 |
| Fast-WAM (56.5) | (56.3) | 46.2 44.7 | 56.9 54.1 |
| GR00T N1.5 (75.6) | (56.3) | 56.0 48.8 | 76.0 70.7 |
| Setting | Pure inference | Reflex |
|---|---|---|
| Model copies on the GPU | 1 | 1 |
| Precision | bfloat16 | bfloat16 |
| Compilation | TorchInductor, CUDA graphs | TorchInductor, CUDA graphs |
| Prefix tokens | 592 | 592 |
| Batch limit of the vision encoder | 16 | 32 |
| Batch limit of the backbone and action head | 16 | 16 |
| Stage | Evaluation | Serving |
|---|---|---|
| Vision encoder | 10.3 | 6.6 |
| Backbone | 27.7 | 17.9 |
| Action head, 10 steps | 29.6 | 26.3 |
| Action head, 1 step | 3.0 | 3.4 |
| Full-step call | 67.5 | 50.8 |
| Pure inference | Reflex | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Policy | Library | Default | Expanded horizon | Reduced steps | Both | Default | Expanded horizon | Reduced steps | Both | |
| LIBERO-Spatial | 50 | 96.6/67.5 | 97.8/34.5 | 98.0/40.9 | 98.4/20.9 | 96.6/76/23.7 | 96.5/68/14.6 | 97.4/71/19.1 | 96.9/72/9.6 | |
| 100 | 97.4/83/20.2 | 97.7/82/10.5 | 96.9/86/14.4 | 97.2/84/7.7 | ||||||
| 200 | 96.5/87/17.5 | 97.5/83/10.3 | 97.2/84/15.2 | 97.4/86/7.5 | ||||||
| 300 | 96.7/86/18.4 | 97.7/81/10.9 | 97.5/80/16.3 | 97.7/91/6.7 | ||||||
| 400 | 96.2/89/16.8 | 98.0/94/7.0 | 96.6/86/14.6 | 97.7/95/6.1 | ||||||
| Default execution horizon | Expanded execution horizon | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Policy | Library | Pure | Cache | +Corr. | +Stag. | +Both | Cache | +Corr. | +Stag. | +Both | |
| LIBERO-Spatial | 50 | 96.6 | 89.8/100/10.3 | 91.2/100/10.3 | 91.0/90/13.2 | 93.2/89/13.5 | 80.2/100/5.2 | 86.4/100/5.2 | 84.4/94/6.2 | 89.4/94/6.2 | |
| 100 | 93.4/100/10.3 | 94.4/100/10.3 | 96.6/94/12.2 | 97.0/94/12.0 | 92.8/100/5.2 | 95.0/100/5.3 | 95.0/96/5.8 | 96.4/97/5.7 | |||
| 200 | 92.8/100/10.3 | 96.6/100/10.3 | 94.8/94/12.1 | 96.8/95/11.9 | 93.4/100/5.2 | 96.6/100/5.3 | 94.2/97/5.7 | 96.6/98/5.6 | |||
| 300 | 94.2/100/10.3 | 96.4/100/10.3 | 95.8/94/12.0 | 96.8/95/11.8 | 95.8/100/5.3 | 98.0/100/5.3 | 96.0/98/5.6 | 97.2/98/5.6 | |||
| 400 | 93.2/100/10.3 | 96.0/100/10.3 | 95.8/94/12.0 | 96.4/95/11.8 | 96.4/100/5.3 | 98.2/100/5.3 | 97.0/98/5.7 | 97.2/98/5.6 | |||
| Default execution horizon | Expanded execution horizon | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Policy | Library | Pure | Cache | +Corr. | +Stag. | +Both | Cache | +Corr. | +Stag. | +Both | |
| MT50 | 250 | 56.3 | 34.8 | 39.1 | 35.9 | 40.0 | n/a | n/a | n/a | n/a | |
| 500 | 41.6 | 46.3 | 42.2 | 46.5 | n/a | n/a | n/a | n/a | |||
| 1,000 | 47.4 | 50.5 | 47.5 | 51.1 | n/a | n/a | n/a | n/a | |||
| 1,500 | 50.1 | 52.4 | 50.6 | 53.0 | n/a | n/a | n/a | n/a | |||
| 2,000 | 50.6 | 52.2 | 50.8 | 53.0 | n/a | n/a | n/a | n/a | |||
| One interval | ||||
| Policy | Pure | Per task | Closest | Best |
| LIBERO-Spatial | ||||
| 98.4/20.9 | 97.4/7.5 | 95.4/5.7 | 98.4/14.0 | |
| Cosmos Policy | 96.6/151.3 | 97.3/104.0 | 95.6/106.1 | 97.2/111.3 |
| LIBERO-Long | ||||
| 91.8/20.6 | 93.7/10.1 | 93.0/11.4 | 93.0/11.4 | |
| Perturbation | No adaptation | Adaptation | + Stagnation check | Policy calls (%) | |
|---|---|---|---|---|---|
| LIBERO-Spatial | |||||
| All except added objects | 88.6 | 74.5 | 88.9 | 89.5 | 6.6 |
| Lighting | 95.9 | 99.3 | 99.0 | 99.3 | 0.4 |
| Background | 98.8 | 93.4 | 89.9 | 88.8 | 4.0 |
| Camera viewpoint | 80.9 | 35.9 | 78.2 | 80.1 | 9.6 |
| Sensor noise | 88.0 | 72.6 | 94.6 | 95.2 | 5.4 |
| Adaptation | + Stagnation check | |||
| Perturbation | Learned | Euclidean | Learned | Euclidean |
| LIBERO-Spatial, all except added objects | 88.9 | 82.8 | 89.5 | 84.2 |
| LIBERO-Long, all except added objects | 62.1 | 52.2 | 65.7 | 57.5 |
| Both suites | ||||
| All except added objects | 74.9 | 66.8 | 77.1 | 70.2 |
| Lighting | 79.9 | 69.6 | 82.9 | 70.1 |
| Setting | Success (%) | Policy calls per episode |
|---|---|---|
| Pure inference | 61.2 | 38.1 |
| Reflex | 48.8 | 15.9 |
| without the stagnation check | 45.6 | 12.8 |
| with takeover, | 57.8 | 20.5 |
| with takeover, | 63.2 | 21.0 |