GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments

Truong Thanh Nguyen; Huy Hoang Nguyen; Ha Anh Nguyen; Binh Khanh Dinh; Ngo Anh Vien; Duy Nguyen Ho Minh; Minh Nhat Vu; Ngan Le

cs.ROSep 5, 2026

GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments

Authors: Truong Thanh Nguyen, Huy Hoang Nguyen, Ha Anh Nguyen, Binh Khanh Dinh, Ngo Anh Vien, Duy Nguyen Ho Minh, Minh Nhat Vu, Ngan Le

Organizations: VinRobotics, Vietnam · Austrian Institute of Technology, Vienna, Austria · Hanoi University of Science and Technology, Vietnam · Center for AI Research, VinUniversity, Vietnam ·

Abstract

Vision-language-action (VLA) models have shown promising generalization for language-conditioned robot manipulation, but deploying them in unstructured environments remains challenging. A single end-to-end VLA policy must simultaneously solve long-range transport of the end effector to task-relevant regions and short-horizon, contact-rich interaction upon arrival. This formulation is inefficient and brittle: small visual shifts, distractors, clutter, occlusions, or unfavorable initial gripper poses can push the policy outside the local state distribution in which it was trained, leading to task failure. We introduce GloVLA, a hybrid framework that explicitly separates object-centric manipulation into two complementary regimes: a geometric transport controller moves the end-effector into interaction-centric handoff regions, and local VLA policies handle only the short-horizon interaction phases. GloVLA is model-agnostic and can be integrated with different VLA backbones with no additional demonstrations and no changes to the action space or success predicate. Experiments on standard LIBERO and LIBERO-Plus Object tasks together with a newly introduced LIBERO-Challenge benchmark ettings with clutter, distractors,illumination changes, visual shifts, and obstruction show that GloVLA improves task success and substantially lowers VLA inference cost compared with full end-to-Challenge, full-trajectory GR00T N1.6execution degrades to 20.9% average success while GloVLA retains 88.5%; on a physical UR10e, overall success improves from 35.6% to 90.0% while mean inference time is more than halved. Videos and additional results are available at https://glovla-project.github.io/

Figures & tables

Fig. 1: Comparison between existing VLA [ 5 , 30 , 9 , 18 , 22 , 23 , 3 , 4 , 24 , 2 ] (left) and GloVLA (right). Existing VLAs typically rely on a single policy to handle both long-range geometric transport and local contact-rich manipulation, making them brittle to unstructured environments, data inefficiency, and high inference cost. In contrast, GloVLA factorizes manipulation into two complementary phases: a geometric controller handles transport, while the VLA focuses solely on local, semantically contact-rich manipulation. This decomposition improves robustness, data efficiency, and inference efficiency while keeping the advantages of VLA.

Fig. 2: GloVLA spatial parameterization diagram and corresponding experimental parameter table.

Fig. 3: Example scenes from the 50-scene LIBERO-Challenge . (a)–(e): easy settings with a single perturbation (clutter, distraction, illumination, obstruction, visual shift); (f)–(g): medium settings with 2–3 combined perturbations; and (h)–(j) hard settings with 4–5 combined perturbations.

LIBERO Object Task	$\pi_{0}$		$\pi_{0.5}$		GR00T N1.6		GR00T N1.7
LIBERO Object Task	FullVLA	GloVLA	FullVLA	GloVLA	FullVLA	GloVLA	FullVLA	GloVLA
Alphabet Soup	94.0 $\pm$ 12.0	100.0 $\pm$ 0.0 ( $\uparrow$ 6.0 )	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	98.0 $\pm$ 4.5	100.0 $\pm$ 0.0 ( $\uparrow$ 2.0 )
BBQ Sauce	82.0 $\pm$ 4.5	90.0 $\pm$ 0.0 ( $\uparrow$ 8.0 )	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	60.0 $\pm$ 15.8	90.0 $\pm$ 8.9 ( $\uparrow$ 30.0 )	94.0 $\pm$ 5.5	98.0 $\pm$ 4.5 ( $\uparrow$ 4.0 )
Butter	84.0 $\pm$ 11.4	100.0 $\pm$ 0.0 ( $\uparrow$ 16 )	96.0 $\pm$ 5.5	100.0 $\pm$ 0.0 ( $\uparrow$ 4.0 )	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0	98.0 $\pm$ 4.5	100.0 $\pm$ 0.0 ( $\uparrow$ 2.0 )
Chocolate Pudding	92.0 $\pm$ 4.5	98.0 $\pm$ 4.5 ( $\uparrow$ 6.0 )	90.0 $\pm$ 8.9	96.0 $\pm$ 5.5 ( $\uparrow$ 6.0 )	90.0 $\pm$ 0.0	100.0 $\pm$ 0.0 ( $\uparrow$ 10.0 )	94.0 $\pm$ 5.5	98.0 $\pm$ 4.5 ( $\uparrow$ 4.0 )
Cream Cheese	88.0 $\pm$ 11.0	98.0 $\pm$ 4.5 ( $\uparrow$ 10.0 )	98.0 $\pm$ 4.5	100.0 $\pm$ 0.0 ( $\uparrow$ 2.0 )	60.0 $\pm$ 14.1	100.0 $\pm$ 0.0 ( $\uparrow$ 40.0 )	98.0 $\pm$ 4.5	100.0 $\pm$ 0.0 ( $\uparrow$ 2.0 )

TABLE I: Performance comparison between FullVLA and GloVLA across ten LIBERO Object tasks.

Difficulty	Perturbation group	Scenes	FullVLA	GloVLA
All	All challenge scenes	50	20.8 ±10.0	88.5 ±9.4 ( $\uparrow$ 67.6 )
Easy	Single perturbations	30	31.4 ±13.2	93.8 ±6.0 ( $\uparrow$ 62.4 )
	Clutter	6	23.7 ±14.1	94.5 ±3.7 ( $\uparrow$ 70.8 )
	Distraction	6	13.5 ±11.9	91.7 ±7.1 ( $\uparrow$ 78.2 )
	Obstruction	6	43.2 ±11.8	96.2 ±3.4 ( $\uparrow$ 53.0 )
	Visual shift	6	36.7 ±19.7	90.3 ±11.0 ( $\uparrow$ 53.6 )

TABLE II: Robustness on LIBERO-Challenge

Object task	Clutter		Distraction		Obstruction		Visual shift		Illumination
Object task	FullVLA	GloVLA	FullVLA	GloVLA	FullVLA	GloVLA	FullVLA	GloVLA	FullVLA	GloVLA
Alphabet soup	26.3 $\pm$ 20.9	99.7 $\pm$ 0.8 ( $\uparrow$ 73.4 )	12.0 $\pm$ 14.4	100.0 $\pm$ 0.0 ( $\uparrow$ 88.0 )	66.7 $\pm$ 15.8	100.0 $\pm$ 0.0 ( $\uparrow$ 33.3 )	54.3 $\pm$ 19.6	100.0 $\pm$ 0.0 ( $\uparrow$ 45.7 )	83.3 $\pm$ 7.7	100.0 $\pm$ 0.0 ( $\uparrow$ 16.7 )
BBQ sauce	3.3 $\pm$ 4.8	94.7 $\pm$ 3.0 ( $\uparrow$ 91.4 )	0.0 $\pm$ 0.0	98.7 $\pm$ 1.5 ( $\uparrow$ 98.7 )	10.0 $\pm$ 5.7	86.7 $\pm$ 7.0 ( $\uparrow$ 76.7 )	9.7 $\pm$ 8.8	88.7 $\pm$ 11.5 ( $\uparrow$ 79.0 )	4.3 $\pm$ 2.7	90.3 $\pm$ 9.1 ( $\uparrow$ 86.0 )
Butter	16.3 $\pm$ 15.5	94.0 $\pm$ 5.8 ( $\uparrow$ 77.7 )	5.0 $\pm$ 5.6	57.3 $\pm$ 22.9 ( $\uparrow$ 52.3 )	33.3 $\pm$ 17.0	91.7 $\pm$ 12.7 ( $\uparrow$ 58.4 )	37.3 $\pm$ 21.1	80.3 $\pm$ 20.9 ( $\uparrow$ 43.0 )	36.3 $\pm$ 8.5	95.3 $\pm$ 4.5 ( $\uparrow$ 59.0 )
Chocolate pudding	24.0 $\pm$ 17.8	98.0 $\pm$ 1.8 ( $\uparrow$ 74.0 )	10.7 $\pm$ 15.1	94.0 $\pm$ 7.4 ( $\uparrow$ 83.3 )	35.0 $\pm$ 14.4	98.3 $\pm$ 2.7 ( $\uparrow$ 63.3 )	37.3 $\pm$ 22.7	88.0 $\pm$ 9.9 ( $\uparrow$ 50.7 )	40.0 $\pm$ 8.4	94.7 $\pm$ 6.8 ( $\uparrow$ 54.7 )
Cream cheese	14.3 $\pm$ 7.3	90.0 $\pm$ 4.7 ( $\uparrow$ 75.7 )	20.0 $\pm$ 9.4	96.0 $\pm$ 3.3 ( $\uparrow$ 76.0 )	24.7 $\pm$ 4.7	96.7 $\pm$ 1.0 ( $\uparrow$ 72.0 )	14.3 $\pm$ 12.5	86.0 $\pm$ 19.7 ( $\uparrow$ 71.7 )	29.0 $\pm$ 6.3	96.3 $\pm$ 1.5 ( $\uparrow$ 67.3 )

TABLE III: Robustness assessment on LIBERO-Challenge under various tasks and permutations.

Method	Avg.	Cam.	Robot	Lang.	Light	BG	Noise	Layout
OpenVLA-OFT [ 17 ]	66.5	38.9	25.4	99.0	73.7	97.6	72.3	71.8
$\pi_{0}$ -FAST [ 24 ]	72.7	72.0	27.6	71.5	71.0	95.2	93.1	84.5
OpenVLA-OFT m [ 17 ]	77.1	70.2	18.1	98.5	100.0	91.9	94.1	77.4
GR00T N1.6	76.6	56.8	47.2	90.4	100.0	99.6	76.3	81.6
GloVLA	88.2	77.3	70.6	100.0	100.0	99.6	92.4	85.9

TABLE IV: Evaluation on the LIBERO-Plus object suite. Averages are weighted by the number of evaluation episodes per axis, following [ 11 ] .

Unstructured Scene
Settings	Method	Demos
Settings	Method	10	30	50
Clean Table-top	FullVLA	86.0 $\pm$ 15.2	90 $\pm$ 8.9	100.0 $\pm$ 0.0
Clean Table-top	GloVLA	100.0 $\pm$ 0.0 ( $\uparrow$ 14.0 )	100.0 $\pm$ 0.0 ( $\uparrow$ 10.0 )	100.0 $\pm$ 0.0
Clutter	FullVLA	8.7 $\pm$ 6.8	19.7 $\pm$ 13.6	26.3 $\pm$ 20.9
Clutter	GloVLA	23.0 $\pm$ 10.9 ( $\uparrow$ 14.3 )	99.0 $\pm$ 1.7 ( $\uparrow$ 79.3 )	99.7 $\pm$ 0.8 ( $\uparrow$ 73.4 )

TABLE V: Data efficiency on LIBERO-Challenge (task 1).

Perturbation	Setting	Trials	FullVLA	GloVLA
✗	Table-top	20	90%	95% ( $\uparrow$ 5.0 )
✓ Easy	Clutter	10	30%	90% ( $\uparrow$ 60.0 )
	Obstruction	10	40%	100% ( $\uparrow$ 60.0 )
	Illumination	10	0%	80% ( $\uparrow$ 80.0 )
	Visual shift	10	40%	100% ( $\uparrow$ 60.0 )
	Distraction	10	20%	100% ( $\uparrow$ 80.0 )

TABLE VI: Evaluation on real-robot under various settings.

Panel of Figure 10 — Fig. 4: Qualitative comparison of success cases between FullVLA and GloVLA across varying difficulty levels in real-robot setups.

Panel of Figure 11 — Fig. 5: Failure cases GloVLA . Our method fails when the object center estimation is wrong, or the offset is too large.

Panel of Figure 12 — Fig. 6: Ablation on the grasp and place extraction radius $r_{o}$ , $r_{b}$ .

Hierarchical Robot Control

See all 17 papers

GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation
Jun 7, 2026Yuan Zhang, Shiqi Zhang, Yedong Shen +11Generalization in Robotic Manipulation Cross-Embodiment Robot Learning
Anhui University · University of Science and Technology of China · iFLYTEK
Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents
Jul 9, 2026Yixian Zhang, Huanming Zhang, Feng Gao +14Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
Tsinghua University · Striding AI · Purdue University +4
LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies
Feb 25, 2026Yue Yang, Shuo Cheng, Yu Fang +4Long-Horizon Robotic Manipulation Robotic Manipulation
University of North Carolina at Chapel Hill · Georgia Institute of Technology · Carnegie Mellon University

Jun 7, 2026J/K move · Enter open · S save

GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation

Yuan Zhang, Shiqi Zhang, Yedong Shen, Shuai Dong+10

Anhui University · University of Science and Technology of China · iFLYTEK

Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Geometric Representation Learning Robotic Manipulation Vision-Language-Action Models Robot Learning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models achieve strong benchmark performance but still struggle in real-world deployment with unseen objects, background shifts, and different robot embodiments. We argue that this stems from the lack of a unified geometry-aware manipulation representation, leaving existing VLAs vulnerable to low-level trajectory supervision, misaligned 3D features, and embodiment differences. To address this, we propose GEAR-VLA, a VLA framework for learning unified geometry-aware action representations for generalizable robotic manipulation. GEAR-VLA adopts coarse-to-fine action learning, where multi-source embodied pretraining equips the VLM with embodied reasoning and discrete action understanding before latent action tokens connect action semantics to a gradient-decoupled DiT continuous action expert. It further performs semantic-aligned 3D integration by aligning a trainable 3D spatial backbone with the VLA representation while freezing the original VLM-aligned visual pathway. To share this representation across robots, GEAR-VLA uses embodiment canonicalization, where embodiment-aware states and embodiment-invariant actions confine robot differences to the low-level interface. Extensive simulation and real-world experiments demonstrate strong generalization: GEAR-VLA achieves state-of-the-art performance on LIBERO, zero-shot LIBERO-Plus, and RoboTwin 2.0, reaches 85.9% success on AgileX and 81.0% on the pretraining-unseen LDT-01 embodiment, and obtains 90.1% success on a 6,360-trial universal grasping benchmark with 212 unseen objects. Code and models will be released at https://github.com/babynabeauty/GEAR-VLA.

Abstract

Hierarchical Robot Control

Explore similar work

GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation

Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents

LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies