Homeward: No Place Like the Training View
Organizations: Institute of Automation, Chinese Academy of Sciences, Beijing, China · Zhongguancun Academy, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · ByteDance, China ·
Abstract
Vision-language-action (VLA) policies implicitly assume that third-person cameras remain fixed at their data-collection poses; once a camera is bumped, remounted, or swapped at deployment, task success degrades precipitously even though the underlying task and robot remain unchanged. We present Homeward, a training-free, plug-and-play observation adapter that canonicalizes observations from an uncalibrated deployment camera back into the training view for frozen VLA policies. Our design stems from a closed-loop diagnostic paradox in simulation: while re-rendering perturbed views into the training camera with oracle geometry recovers much of the lost performance, substituting an off-the-shelf feed-forward 3D foundation model can leave the policy worse off than applying no correction at all. This degradation concentrates on the thin, articulated robot arm, which feed-forward depth prediction tears and scatters - yet the arm's exact 3D state is already known from robot kinematics. Homeward exploits this asymmetry: it estimates what is unknown - camera parameters and static scene depth - from the live RGB frame and a single training-view reference, and renders what is known - the moving robot links - directly from the kinematic model. Evaluated across 300,000 closed-loop episodes spanning four frozen VLAs, Homeward boosts success across all 1,599 camera variants of LIBERO-Plus from 62.5%, 76.5%, 68.0%, and 3.6% to 77.6%, 95.1%, 91.2%, and 26.4% for OpenVLA-OFT, , , and OpenVLA, respectively. Without any scene fine-tuning or deployment camera calibration, Homeward extends to 6-DoF random placements, focal-length changes, and, with per-step pose estimation, continuous intra-episode camera drift, while matching or surpassing a calibrated, scene-trained view-synthesis baseline on three of four policies on LIBERO-Plus.
Figures & tables
| trained or | moved | robot | |
|---|---|---|---|
| fine-tuned | camera | region | |
| AnyCamVLA [ 14 ] | synthesis model | calibrated | synthesized |
| VistaBot [ 12 ] | VGGT, policy | intrinsics | reconstructed |
| Mirage [ 4 ] | dynamics model | calibrated | rendered |
| Homeward | none | uncalibrated | rendered |
| OFT | OpenVLA | |||||||
| G | L | G | L | G | L | G | L | |
| frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 65.2 | 44.9 | 8.1 | 1.2 |
| oracle re-rendering † | 90.4 ∗ | 76.1 ∗ | 88.7 | 77.8 ∗ | 89.7 ∗ | 74.7 ∗ | 27.5 ∗ | 15.0 ∗ |
| oracle homography † | 70.8 ∗ | 38.7 | 74.8 ∗ | 51.3 | 60.8 | 41.5 | 13.7 ∗ | 4.5 ∗ |
| scene restoration only | 72.8 ∗ | 51.3 ∗ | 76.0 ∗ | 60.4 ∗ | 82.6 ∗ | 62.5 ∗ | 22.3 ∗ | 7.2 ∗ |
| Homeward | 91.4 ∗ | 72.3 ∗ | 94.4 ∗ | 94.0 ∗ | 93.4 ∗ | 78.3 ∗ | 39.2 ∗ | 11.2 ∗ |
| OpenVLA-OFT | OpenVLA | ||||||||
| Suite | Variants | frozen | + Homeward | frozen | + Homeward | frozen | + Homeward | frozen | + Homeward |
| Spatial | 376 | 91.8 | 97.3 ∗ | 76.9 | 98.1 ∗ | 79.5 | 98.7 ∗ | 3.2 | 32.2 ∗ |
| Object | 396 | 45.7 | 50.3 | 93.2 | 93.9 | 84.6 | 95.7 ∗ | 2.0 | 23.7 ∗ |
| Goal | 408 | 79.7 | 91.4 ∗ | 84.3 | 94.4 ∗ | 65.2 | 93.4 ∗ | 8.1 | 39.2 ∗ |
| Long | 419 | 35.3 | 72.3 ∗ | 52.7 | 94.0 ∗ | 44.9 | 78.3 ∗ | 1.2 | 11.2 ∗ |
| All | 1,599 | 62.5 | 77.6 ∗ | 76.5 | 95.1 ∗ | 68.0 | 91.2 ∗ | 3.6 | 26.4 ∗ |
| OFT | OpenVLA | |||
| CamRand | ||||
| frozen | 83.8 | 93.9 | 88.2 | 30.5 |
| + Homeward | 93.6 ∗ | 97.2 ∗ | 93.0 ∗ | 62.8 ∗ |
| AnyCamVLA-style, other suites | ||||
| frozen | 62.0 | 59.3 | 43.3 | 5.9 |
| + Homeward | 70.9 ∗ | 79.7 ∗ | 62.3 ∗ | 12.6 ∗ |
| scene | moved | |||||
|---|---|---|---|---|---|---|
| training | camera | OFT | OpenVLA | |||
| frozen policy | – | – | 62.5 | 76.5 | 68.0 | 3.6 |
| AnyCamVLA ‡ | fine-tuned | true | 80.7 | 85.1 | 87.5 | 26.0 |
| AnyCamVLA § | fine-tuned | estimated | 79.5 | 82.1 | 83.1 | 14.1 |
| Homeward | none | estimated | 77.6 | 95.1 | 91.2 | 26.4 |
| OFT | OFT+ (fine-tuned) | |||
|---|---|---|---|---|
| frozen | + ours | frozen | + ours | |
| LIBERO-Plus camera | 62.5 | 77.6 ∗ | 95.7 | 91.1 ∗ |
| AnyCamVLA, Long | 33.7 | 61.7 ∗ | 68.2 | 83.3 ∗ |
| AnyCamVLA-style, other suites | 62.0 | 70.9 ∗ | 84.2 | 87.4 ∗ |
| CamRand | 83.8 | 93.6 ∗ | 91.2 | 95.4 ∗ |
| Drift (per-step pose) | 56.0 | 74.7 ∗ | 76.5 | 86.7 ∗ |
| OFT | OpenVLA | |||||||
| Goal | Long | Goal | Long | Goal | Long | Goal | Long | |
| frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 65.2 | 44.9 | 8.1 | 1.2 |
| scene restoration only | 72.8 ∗ | 51.3 ∗ | 76.0 ∗ | 60.4 ∗ | 82.6 ∗ | 62.5 ∗ | 22.3 ∗ | 7.2 ∗ |
| + whole robot incl. pedestal | 39.0 ∗ | 31.0 | 72.3 ∗ | 53.9 | 74.3 ∗ | 43.4 | 4.4 ∗ | 0.0 |
| + moving links, pose per step | 88.2 ∗ | 70.6 ∗ | 93.6 ∗ | 83.1 ∗ | 90.7 ∗ | 74.0 ∗ | 36.3 ∗ | 12.2 ∗ |
| + moving links, pose once | 90.0 ∗ | 71.4 ∗ | 93.4 ∗ | 92.6 ∗ | 92.9 ∗ | 73.5 ∗ | 35.8 ∗ | 11.0 ∗ |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| LIBERO-Long | small | medium | large | all |
|---|---|---|---|---|
| OpenVLA-OFT | ||||
| frozen | 80.6 | 9.6 | 11.0 | 33.7 |
| + Homeward | 88.4 | 62.6 | 34.0 | 61.7 ∗ |
| + AnyCamVLA ‡ | 89.6 | 79.4 | 61.6 | 76.9 ∗ |
| + AnyCamVLA § | 85.6 | 78.8 | 57.8 | 74.1 ∗ |
| + zero-shot LVSM ‡ | 40.8 | 18.2 | 2.6 | 20.5 ∗ |
| All suites | Object | |||
| Policy | Homeward | + refined erase | Homeward | + refined erase |
| LIBERO-Plus camera variants | ||||
| 95.1 | 95.4 | 93.9 | 97.0 ∗ | |
| 91.2 | 90.3 | 95.7 | 97.0 | |
| OpenVLA | 31.6 † | 27.7 ∗ | 23.7 | 12.1 ∗ |
| AnyCamVLA protocol, all orbits | ||||
| OFT | OpenVLA | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Category | frozen | + Homeward | frozen | + Homeward | frozen | + Homeward | frozen | + Homeward | |
| Background Textures | 400 | 62.2 | 80.5 ∗ | 83.2 | 92.8 ∗ | 69.0 | 84.8 ∗ | 4.8 | 31.2 ∗ |
| Language Instructions | 400 | 58.2 | 78.0 ∗ | 75.2 | 88.5 ∗ | 53.0 | 65.2 ∗ | 3.0 | 23.8 ∗ |
| Light Conditions | 400 | 69.8 | 89.0 ∗ | 88.2 | 96.2 ∗ | 75.5 | 89.8 ∗ | 3.2 | 24.8 ∗ |
| Objects Layout | 400 | 46.5 | 67.2 ∗ | 76.5 | 84.5 ∗ | 58.2 | 75.5 ∗ | 4.5 | 34.2 ∗ |
| Robot Initial States | 400 | 23.2 | 34.2 ∗ | 66.5 | 80.5 ∗ | 36.2 | 48.2 ∗ | 2.5 | 11.8 ∗ |
| OFT | ||||||
|---|---|---|---|---|---|---|
| Goal | Long | Goal | Long | Goal | Long | |
| frozen policy | 81.6 | 34.8 | 83.8 | 52.7 | 65.2 | 44.9 |
| Homeward , exact training camera | 87.3 ∗ | 71.1 ∗ | 95.6 ∗ | 92.4 ∗ | 92.9 ∗ | 73.5 ∗ |
| camera off by 5 mm, | 89.0 ∗ | 69.5 ∗ | 94.9 ∗ | 89.7 ∗ | 92.9 ∗ | 74.2 ∗ |
| camera off by 10 mm, | 88.7 ∗ | 68.0 ∗ | 93.4 ∗ | 90.9 ∗ | 89.7 ∗ | 75.9 ∗ |
| camera off by 20 mm, | 86.0 | 58.9 ∗ | 93.4 ∗ | 86.6 ∗ | 90.7 ∗ | 75.2 ∗ |
| OFT | |||
|---|---|---|---|
| frozen policy | 62.5 | 76.5 | 68.0 |
| + temporal ensembling of action chunks | 64.7 ∗ | 74.8 | 67.3 |
| + 2D registration to the reference frame | 62.4 | 74.1 ∗ | 70.7 ∗ |
| + Homeward | 77.6 ∗ | 95.1 ∗ | 91.2 ∗ |
| OFT | ||||||
| Goal | Long | Goal | Long | Goal | Long | |
| frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 65.2 | 44.9 |
| 2D registration | 78.4 | 44.6 ∗ | 84.8 | 56.1 | 70.8 ∗ | 49.6 ∗ |
| + robot layer | 71.6 ∗ | 51.3 ∗ | 89.7 ∗ | 67.5 ∗ | 87.5 ∗ | 61.8 ∗ |
| oracle homography † | 70.8 ∗ | 38.7 | 74.8 ∗ | 51.3 | 60.8 | 41.5 |
| + robot layer | 79.7 | 46.3 ∗ | 90.9 ∗ | 78.5 ∗ | 86.0 ∗ | 63.5 ∗ |
| OFT | † | OpenVLA | ||||||
|---|---|---|---|---|---|---|---|---|
| Goal | Long | Goal | Long | Goal | Long | Spatial | Object | |
| frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 64.5 | 45.6 | 3.2 | 2.0 |
| exact render | 91.4 ∗ | 72.3 ∗ | 94.4 ∗ | 94.0 ∗ | 91.7 ∗ | 73.3 ∗ | 32.2 ∗ | 23.7 ∗ |
| 91.9 ∗ | 67.1 ∗ | 92.6 ∗ | 91.4 ∗ | 94.6 ∗ | 78.5 ∗ | 31.6 ∗ | 20.7 ∗ | |
| 89.2 ∗ | 64.2 ∗ | 92.2 ∗ | 90.2 ∗ | 91.9 ∗ | 77.1 ∗ | 28.7 ∗ | 15.4 ∗ | |
| 87.5 ∗ | 57.3 ∗ | 90.7 ∗ | 88.3 ∗ | 90.7 ∗ | 81.4 ∗ | 28.7 ∗ | 13.4 ∗ | |
| OFT | OpenVLA | |||||||
|---|---|---|---|---|---|---|---|---|
| Goal | Long | Goal | Long | Goal | Long | Goal | Long | |
| Homeward , exact render | 91.4 ∗ | 72.3 ∗ | 94.4 ∗ | 94.0 ∗ | 93.4 ∗ | 78.3 ∗ | 39.2 ∗ | 11.2 ∗ |
| robot drawn with a real-robot colour error | 89.2 ∗ | 64.2 ∗ | 92.2 ∗ | 90.2 ∗ | 91.7 ∗ | 79.5 ∗ | 30.4 ∗ | 10.0 ∗ |
| + colour fit on the reference frame | 87.3 ∗ | 64.9 ∗ | 92.6 ∗ | 86.6 ∗ | 93.9 ∗ | 79.7 ∗ | 21.8 ∗ | 13.6 ∗ |
| OFT | OpenVLA | |||
| clean camera, 400 episodes | ||||
| frozen | 94.8 | 96.8 | 92.5 | 75.2 |
| + Homeward | 94.8 | 96.2 | 92.8 | 74.8 |
| without motion gate | 95.8 | 97.0 | 92.8 | 73.5 |
| non-camera variants of the four suites, 2,400 episodes | ||||
| frozen | 74.5 | 90.7 | 76.0 | 34.1 |
| small drift (400) | all drift (1,200) | |||||||
|---|---|---|---|---|---|---|---|---|
| frozen | no gate | first query | every query | frozen | no gate | first query | every query | |
| OpenVLA-OFT | 85.2 | 85.2 | 84.8 | 86.8 | 58.1 | 72.5 | 72.3 | 73.0 |
| 84.8 | 90.0 | 84.8 | 88.0 | 67.5 | 82.5 | 80.8 | 80.8 | |
| 64.8 | 66.8 | 64.2 | 69.0 | 50.3 | 64.0 | 63.2 | 63.8 | |
| OpenVLA | 30.8 | 23.8 | 30.2 | – | 11.4 | 12.4 | 14.6 | – |
| OFT | drift (1,200) | ||||||||
| Goal | Long | Goal | Long | Goal | Long | OFT | |||
| frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 65.2 | 44.9 | 58.1 | 67.5 | 50.3 |
| acting on 0.25 s-old observations | 31.9 ∗ | 10.7 ∗ | 55.1 ∗ | 34.6 ∗ | 50.2 ∗ | 13.4 ∗ | – | – | – |
| Homeward | 91.4 | 72.3 | 94.4 | 94.0 | 93.4 | 78.3 | 72.5 | 82.5 | 64.0 |
| robot holds 0.25 s per query | 92.4 | 73.5 | 90.7 ∗ | 92.1 | 92.9 | 77.3 | 69.2 ∗ | 81.2 | 64.3 |
| acting on 0.25 s-old observations | 21.3 ∗ | 24.3 ∗ | 56.9 ∗ | 37.9 ∗ | 47.5 ∗ | 26.5 ∗ | – | – | – |
| variants | rotation (deg) | position (cm) | focal (%) | position 20 cm | |
|---|---|---|---|---|---|
| Spatial | 376 | 1.2 / 3.0 | 5.2 / 9.4 | 5.2 / 14.1 | 0 |
| Object | 396 | 0.9 / 1.7 | 5.8 / 64.4 | 3.5 / 7.9 | 53 |
| Goal | 408 | 1.2 / 2.3 | 5.9 / 8.8 | 6.6 / 12.8 | 0 |
| Long | 419 | 0.9 / 1.7 | 4.5 / 7.5 | 4.7 / 13.6 | 0 |
| All | 1,599 | 1.0 / 2.3 | 5.4 / 9.1 | 4.5 / 12.5 | 53 |
| position error 20 cm (343) | position error 20 cm (53) | |||||
| frozen | Homeward | oracle geometry + robot layer † | frozen | Homeward | oracle geometry + robot layer † | |
| OpenVLA-OFT | 51.9 | 57.1 | 98.8 ∗ | 5.7 | 5.7 | 69.8 ∗ |
| 92.1 | 98.0 ∗ | 98.8 ∗ | 100.0 | 67.9 ∗ | 100.0 | |
| 91.5 | 96.5 ∗ | 96.5 ∗ | 39.6 | 90.6 ∗ | 96.2 ∗ | |
| passed (%) | OFT | OpenVLA | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| gate thresholds | L1 | all | L1 | all | L1 | all | L1 | all | L1 | all |
| no gate ( Homeward as reported) | 0 | 0 | 87.4 | 77.6 | 98.0 | 95.1 | 97.0 | 91.2 | 47.2 | 26.4 |
| , 1% (default) | 0 | 0 | 87.4 | 77.6 | 98.0 | 95.1 | 97.0 | 91.2 | 47.2 | 26.4 |
| , 4% (4 ) | 0 | 0 | 87.4 | 77.6 | 98.0 | 95.1 | 97.0 | 91.2 | 47.2 | 26.4 |
| , 10% (10 ) | 3 | 2 | 86.9 | 77.4 | 98.0 | 95.4 | 96.5 | 90.6 ∗ | 46.7 | 26.3 |
| , 20% (20 ) | 31 | 14 | 85.9 | 76.0 ∗ | 98.0 | 95.4 | 96.0 | 88.6 ∗ | 27.6 ∗ | 19.8 ∗ |
| OFT | OpenVLA | |||||||
| Goal | Long | Goal | Long | Goal | Long | Goal | Long | |
| training camera, frozen policy † | 98.0 | 91.0 | 96.0 | 91.0 | 95.0 | 84.0 | 74.0 | 56.0 |
| moved camera, frozen policy | 79.7 | 35.3 | 84.3 | 52.7 | 65.2 | 44.9 | 8.1 | 1.2 |
| oracle re-rendering ‡ | 90.4 | 76.1 | 88.7 | 77.8 | 89.7 | 74.7 | 27.5 | 15.0 |
| oracle re-rendering + robot layer ‡ | 91.4 | 83.3 | 97.1 | 91.9 | 95.8 | 80.2 | 52.7 | 25.1 |
| Homeward | 91.4 | 72.3 | 94.4 | 94.0 | 93.4 | 78.3 | 39.2 | 11.2 |
| models | time per query (ms) | peak GPU memory (GiB) | |
|---|---|---|---|
| Homeward | VGGT-1B | 164 + 10 | 6.6 |
| Homeward , per-step | VGGT-1B | 191 + 10 | 6.6 |
| AnyCamVLA, true cameras ‡ | LVSM | 32 | 1.1 |
| AnyCamVLA, estimated cameras § | LVSM, VGGT-1B | 31 | 7.2 |
| frozen | Homeward | per-step | ||||
|---|---|---|---|---|---|---|
| GPU | software | GPU | software | GPU | software | |
| Spatial | 79.5 | 79.5 | 97.6 | 97.3 | 97.9 | 97.1 |
| Object | 84.6 | 85.1 | 96.0 | 96.2 | 96.7 | 96.2 |
| Goal | 65.2 | 64.7 | 92.9 | 93.1 | 90.7 | 91.9 |
| Long | 44.9 | 48.0 | 73.5 | 78.0 ∗ | 74.0 | 76.1 |
| Policy | gain (points) | 95% CI, tasks | 95% CI, tasks within suite | tasks up/down/tied | sign test | Wilcoxon |
|---|---|---|---|---|---|---|
| OpenVLA-OFT | 26 / 5 / 9 | |||||
| 23 / 9 / 8 | ||||||
| 35 / 1 / 4 | ||||||
| OpenVLA | 38 / 1 / 1 |
| Comparison (difference in points) | diff. | 95% CI, tasks |
| Homeward minus AnyCamVLA, LIBERO-Plus camera variants | ||
| OpenVLA-OFT | / | / |
| / | / | |
| / | / | |
| OpenVLA | / | / |
| Homeward minus AnyCamVLA (true cameras), orbit protocol on Long | ||
Robustness of VLA Models
Robustness of vision-language-action (VLA) models concerns reliable robot behavior under changes to language, vision, sensors, tasks, and hardware.
See all 239 papers