Gripper-aware Vision Language Action Models

Hanyi Zhang; Zihong Luo; Tianyu Li; Khang Nguyen; Basu Hela; Shreyas Kumar; Ngoc Duy Tran; Feng Dai

cs.ROAug 25, 2026

Gripper-aware Vision Language Action Models

Authors: Hanyi Zhang, Zihong Luo, Tianyu Li, Khang Nguyen, Basu Hela, Shreyas Kumar, Ngoc Duy Tran, Feng Dai,

Organizations: University of Liverpool, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE · Indian Institute of Science, India · The University of Tokyo, Japan ·

Abstract

Vision language action models (VLAs) have advanced general purpose robotic grasping and manipulation by enabling robots to interpret visual observations and natural language instructions to generate executable action sequences. However, existing VLAs often implicitly assume gripper invariance, despite grasping strategies being inherently embodiment-dependent. Different gripper types, such as parallel-jaw and suction, usually require distinct interaction strategies to achieve the same grasping objective. Moreover, current datasets for VLAs predominantly rely on parallel-jaw grippers, limiting gripper-aware learning. To address this gap, we introduce MiGA, a multi-gripper-aware dataset spanning five distinct gripper types across multiple robots with 103,000 demonstrations, explicitly capturing strategy divergence under shared task objectives. We further propose GVLA, which combines a new multi-gripper tokenizer with adapter-based policy routing. Our new gripper encoding induces structured embedding information that balances parameter sharing and strategy differentiation, while layer-wise probing confirms meaningful gripper-conditioned representations for VLAs. Intensive experiments in both simulation and real-world robots show that our GVLA outperforms the current baselines across evaluated settings. Our method also improves zero-shot generalization or few-shot adaptation to new objects or unseen tasks, and enable more efficient gripper adaptation.

Figures & tables

Figure 1: We tackle the challenge of gripper-specific grasping in robotics, e.g., a suction cup gripper lifts a thin, flat, DVD-case-like box from above, while a parallel-jaw gripper pushes it to the table edge and grasps it from the side. We first introduce MiGA, a large-scale multi-gripper-aware dataset collected by diverse gripper types. Leveraging this dataset, we then develop a Gripper-aware Vision Language Action (GVLA) model that incorporates gripper conditioning through a new multi-gripper tokenization method and a dual Mixture-of-Adapter. Our approach improves task performance over gripper-agnostic baselines and enables cross-gripper adaptation.

Panel of Figure 2 — Figure 2: Gripper embedding feature heatmap of different gripper representations. Our new multi-gripper tokenization method (d) produces well-clustered representations organized by gripper type. In contrast, other methods (a-c) fail to capture the underlying semantic structure of different gripper types. $V_{1}$ and $V_{2}$ denote the vacuum gripper: UR10 Suction Cup [ 23 ] and Cobot Pump [ 50 ] , $F_{1}$ , and $F_{2}$ denote finger-based gripper: Franka Panda Hand [ 16 ] and Detexterous Hand [ 4 ] .

Dataset	#Traj.	#Gripper Types	#Robots	Gripper-Specific Solution	Depth Image	Sub-step Decomposition	Data Domain	Collection
Open X-Embodiment [ 44 ]	1M+	1	22	✗	✗	✗	Real	Aggregation
DROID [ 26 ]	76K	1	1	✗	✗	✗	Real	Human
Bridge V2 [ 58 ]	60K	1	1	✗	✗	✗	Real	Human
RT-1 [ 6 ]	130K	1	1	✗	✗	✗	Real	Human
RH20T [ 12 ]	110K	1	7	✗	✓	✗	Real	Human
BridgeDataV2 [ 12 ]	60.1K	1	1	✗	✓	✗	Real	84% Human, 16% Scripted

Table 1: Comparison of existing datasets for VLAs. Our dataset provides a manually collected, gripper-aware data spanning multiple gripper types with gripper-specific manipulation strategies in both simulation and real-world settings.

Panel of Figure 3 — Figure 3: Illustration of gripper-specific strategy variations across four task categories.

Panel of Figure 4 — Figure 4: MiGA dataset statistics.

Figure 5: An overview of GVLA architecture. Our method extends the pretrained VLA backbone with two gripper-aware components. (Left) Multi-gripper tokenizer encodes gripper knowledge through three-level soft prompts: platform-specific, gripper type-specific, and instance-specific learnable tokens. (Right) A dual MoA routes action tokens through type-stratified expert pools using a cascaded domain and a gripper router, applying top-k selected adapters as residual connections to adapt to gripper-specific manipulation strategies.

Figure 6: Layer probing analysis.

Method	Flat	Stacked	Constrained	Semantic	Avg.(%)
AnyGrasp [ 13 ]	0.00	0.00	46.00	0.00	11.50
GraspMAS [ 41 ]	0.00	0.00	40.00	8.00	12.00
GraspVLA [ 9 ]	1.50	0.00	30.00	0.00	7.88
OpenVLA-OFT [ 27 ]	41.00	52.50	24.00	50.00	41.88
$\pi_{0}$ [ 5 ]	30.00	21.50	36.00	65.00	38.13
$\pi_{0.5}$ [ 22 ]	52.50	71.00	57.50	52.50	58.38

Table 2: Baseline comparison across task categories.

Panel of Figure 8 — Figure 8: Cross-object generalization results.

Panel of Figure 9 — Figure 9: Adaptation results.

	Configuration	PE $\downarrow$	Adapt. $\uparrow$
Gripper Repr.	w/o $P^{(g)}$	0.037	0.86
	w/o $P^{(p)}$	0.035	0.54
	w/o $P^{(u)}$	0.032	0.90
Dual MoA	w/o MoA	0.037	0.52
	w/o $\text{MoA}^{(g)}$	0.033	0.56
	w/o $\text{MoA}^{(p)}$	0.034	0.54

Table 4: Component analysis.

Panel of Figure 10 — Figure 10: Real-world robotic experiment setups and results.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,154 papers

DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation
Mar 1, 2026Xiongfeng Peng, Jiaqian Yu, Dingzhe Li +8Robotic Manipulation Vision-Language-Action Models
Advanced Research Lab, Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea · Hanyang University ERICA, South Korea
GIVE: Grounding Human Gestures in Vision-Language-Action Models
Jun 11, 2026Pengfei Liu, Gen Li, Junqiao Fan +4Language-Conditioned Robot Manipulation Vision-Language Grounding
1MARS Lab, Nanyang Technological University
GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations
May 21, 2026Wenxuan Guo, Ziyuan Li, Meng Zhang +7Multimodal Grounding Robotic Manipulation
1Tsinghua University · 2Dexmal

Mar 1, 2026J/K move · Enter open · S save

DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation

Xiongfeng Peng, Jiaqian Yu, Dingzhe Li, Yixiang Jin+7

Advanced Research Lab, Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea · Hanyang University ERICA, South Korea

Robotic Manipulation Vision-Language-Action Models Dynamic Robotic Manipulation Long-Horizon Robotic Manipulation Contact-Rich Robotic Manipulation

Open paper AskPDF arXiv

In dynamic environments such as warehouses, hospitals, and homes, robots must seamlessly transition between gross motion and precise manipulations to complete complex tasks. However, current Vision-Language-Action (VLA) frameworks, largely adapted from pre-trained Vision-Language Models (VLMs), often struggle to reconcile general task adaptability with the specialized precision required for intricate manipulation. To address this challenge, we propose DAM-VLA, a dynamic action model-based VLA framework. DAM-VLA integrates VLM reasoning with diffusion-based action models specialized for arm and gripper control. Specifically, it introduces (i) an action routing mechanism, using task-specific visual and linguistic cues to select appropriate action models (e.g., arm movement or gripper manipulation), (ii) a dynamic action model that fuses high-level VLM cognition with low-level visual features to predict actions, and (iii) a dual-scale action weighting mechanism that enables dynamic coordination between the arm-movement and gripper-manipulation models. Across extensive evaluations, DAM-VLA achieves superior success rates compared to state-of-the-art VLA methods in simulated (SIMPLER, FurnitureBench) and real-world settings, showing robust generalization from standard pick-and-place to demanding long-horizon and contact-rich tasks.

Abstract

Vision-Language-Action Models

Explore similar work

DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation

GIVE: Grounding Human Gestures in Vision-Language-Action Models

GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations