GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments
Organizations: VinRobotics, Vietnam · Austrian Institute of Technology, Vienna, Austria · Hanoi University of Science and Technology, Vietnam · Center for AI Research, VinUniversity, Vietnam ·
Abstract
Vision-language-action (VLA) models have shown promising generalization for language-conditioned robot manipulation, but deploying them in unstructured environments remains challenging. A single end-to-end VLA policy must simultaneously solve long-range transport of the end effector to task-relevant regions and short-horizon, contact-rich interaction upon arrival. This formulation is inefficient and brittle: small visual shifts, distractors, clutter, occlusions, or unfavorable initial gripper poses can push the policy outside the local state distribution in which it was trained, leading to task failure. We introduce GloVLA, a hybrid framework that explicitly separates object-centric manipulation into two complementary regimes: a geometric transport controller moves the end-effector into interaction-centric handoff regions, and local VLA policies handle only the short-horizon interaction phases. GloVLA is model-agnostic and can be integrated with different VLA backbones with no additional demonstrations and no changes to the action space or success predicate. Experiments on standard LIBERO and LIBERO-Plus Object tasks together with a newly introduced LIBERO-Challenge benchmark ettings with clutter, distractors,illumination changes, visual shifts, and obstruction show that GloVLA improves task success and substantially lowers VLA inference cost compared with full end-to-Challenge, full-trajectory GR00T N1.6execution degrades to 20.9% average success while GloVLA retains 88.5%; on a physical UR10e, overall success improves from 35.6% to 90.0% while mean inference time is more than halved. Videos and additional results are available at https://glovla-project.github.io/
Figures & tables
| LIBERO Object Task | GR00T N1.6 | GR00T N1.7 | ||||||
|---|---|---|---|---|---|---|---|---|
| FullVLA | GloVLA | FullVLA | GloVLA | FullVLA | GloVLA | FullVLA | GloVLA | |
| Alphabet Soup | 94.0 12.0 | 100.0 0.0 ( 6.0 ) | 100.0 0.0 | 100.0 0.0 | 100.0 0.0 | 100.0 0.0 | 98.0 4.5 | 100.0 0.0 ( 2.0 ) |
| BBQ Sauce | 82.0 4.5 | 90.0 0.0 ( 8.0 ) | 100.0 0.0 | 100.0 0.0 | 60.0 15.8 | 90.0 8.9 ( 30.0 ) | 94.0 5.5 | 98.0 4.5 ( 4.0 ) |
| Butter | 84.0 11.4 | 100.0 0.0 ( 16 ) | 96.0 5.5 | 100.0 0.0 ( 4.0 ) | 100.0 0.0 | 100.0 0.0 | 98.0 4.5 | 100.0 0.0 ( 2.0 ) |
| Chocolate Pudding | 92.0 4.5 | 98.0 4.5 ( 6.0 ) | 90.0 8.9 | 96.0 5.5 ( 6.0 ) | 90.0 0.0 | 100.0 0.0 ( 10.0 ) | 94.0 5.5 | 98.0 4.5 ( 4.0 ) |
| Cream Cheese | 88.0 11.0 | 98.0 4.5 ( 10.0 ) | 98.0 4.5 | 100.0 0.0 ( 2.0 ) | 60.0 14.1 | 100.0 0.0 ( 40.0 ) | 98.0 4.5 | 100.0 0.0 ( 2.0 ) |
| Difficulty | Perturbation group | Scenes | FullVLA | GloVLA |
|---|---|---|---|---|
| All | All challenge scenes | 50 | 20.8 ±10.0 | 88.5 ±9.4 ( 67.6 ) |
| Easy | Single perturbations | 30 | 31.4 ±13.2 | 93.8 ±6.0 ( 62.4 ) |
| Clutter | 6 | 23.7 ±14.1 | 94.5 ±3.7 ( 70.8 ) | |
| Distraction | 6 | 13.5 ±11.9 | 91.7 ±7.1 ( 78.2 ) | |
| Obstruction | 6 | 43.2 ±11.8 | 96.2 ±3.4 ( 53.0 ) | |
| Visual shift | 6 | 36.7 ±19.7 | 90.3 ±11.0 ( 53.6 ) |
| Object task | Clutter | Distraction | Obstruction | Visual shift | Illumination | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| FullVLA | GloVLA | FullVLA | GloVLA | FullVLA | GloVLA | FullVLA | GloVLA | FullVLA | GloVLA | |
| Alphabet soup | 26.3 20.9 | 99.7 0.8 ( 73.4 ) | 12.0 14.4 | 100.0 0.0 ( 88.0 ) | 66.7 15.8 | 100.0 0.0 ( 33.3 ) | 54.3 19.6 | 100.0 0.0 ( 45.7 ) | 83.3 7.7 | 100.0 0.0 ( 16.7 ) |
| BBQ sauce | 3.3 4.8 | 94.7 3.0 ( 91.4 ) | 0.0 0.0 | 98.7 1.5 ( 98.7 ) | 10.0 5.7 | 86.7 7.0 ( 76.7 ) | 9.7 8.8 | 88.7 11.5 ( 79.0 ) | 4.3 2.7 | 90.3 9.1 ( 86.0 ) |
| Butter | 16.3 15.5 | 94.0 5.8 ( 77.7 ) | 5.0 5.6 | 57.3 22.9 ( 52.3 ) | 33.3 17.0 | 91.7 12.7 ( 58.4 ) | 37.3 21.1 | 80.3 20.9 ( 43.0 ) | 36.3 8.5 | 95.3 4.5 ( 59.0 ) |
| Chocolate pudding | 24.0 17.8 | 98.0 1.8 ( 74.0 ) | 10.7 15.1 | 94.0 7.4 ( 83.3 ) | 35.0 14.4 | 98.3 2.7 ( 63.3 ) | 37.3 22.7 | 88.0 9.9 ( 50.7 ) | 40.0 8.4 | 94.7 6.8 ( 54.7 ) |
| Cream cheese | 14.3 7.3 | 90.0 4.7 ( 75.7 ) | 20.0 9.4 | 96.0 3.3 ( 76.0 ) | 24.7 4.7 | 96.7 1.0 ( 72.0 ) | 14.3 12.5 | 86.0 19.7 ( 71.7 ) | 29.0 6.3 | 96.3 1.5 ( 67.3 ) |
| Method | Avg. | Cam. | Robot | Lang. | Light | BG | Noise | Layout |
|---|---|---|---|---|---|---|---|---|
| OpenVLA-OFT [ 17 ] | 66.5 | 38.9 | 25.4 | 99.0 | 73.7 | 97.6 | 72.3 | 71.8 |
| -FAST [ 24 ] | 72.7 | 72.0 | 27.6 | 71.5 | 71.0 | 95.2 | 93.1 | 84.5 |
| OpenVLA-OFT m [ 17 ] | 77.1 | 70.2 | 18.1 | 98.5 | 100.0 | 91.9 | 94.1 | 77.4 |
| GR00T N1.6 | 76.6 | 56.8 | 47.2 | 90.4 | 100.0 | 99.6 | 76.3 | 81.6 |
| GloVLA | 88.2 | 77.3 | 70.6 | 100.0 | 100.0 | 99.6 | 92.4 | 85.9 |
| Settings | Method | Demos | ||
|---|---|---|---|---|
| 10 | 30 | 50 | ||
| Clean Table-top | FullVLA | 86.0 15.2 | 90 8.9 | 100.0 0.0 |
| GloVLA | 100.0 0.0 ( 14.0 ) | 100.0 0.0 ( 10.0 ) | 100.0 0.0 | |
| Unstructured Scene | ||||
| Clutter | FullVLA | 8.7 6.8 | 19.7 13.6 | 26.3 20.9 |
| GloVLA | 23.0 10.9 ( 14.3 ) | 99.0 1.7 ( 79.3 ) | 99.7 0.8 ( 73.4 ) | |
| Perturbation | Setting | Trials | FullVLA | GloVLA |
|---|---|---|---|---|
| ✗ | Table-top | 20 | 90% | 95% ( 5.0 ) |
| ✓ Easy | Clutter | 10 | 30% | 90% ( 60.0 ) |
| Obstruction | 10 | 40% | 100% ( 60.0 ) | |
| Illumination | 10 | 0% | 80% ( 80.0 ) | |
| Visual shift | 10 | 40% | 100% ( 60.0 ) | |
| Distraction | 10 | 20% | 100% ( 80.0 ) |