Agile-VLA: Few-Shot Industrial Pose Rectification via Implicit Affordance Anchoring

Teng Yan; Zhengyang Pei; Chengyu Shi; Yue Yu; Yikun Chen; Zilong Zhu; Zelin Fang; Kaile Guo

cs.ROMar 24, 2026

Agile-VLA: Few-Shot Industrial Pose Rectification via Implicit Affordance Anchoring

Authors: Teng Yan, Zhengyang Pei, Chengyu Shi, Yue Yu, Yikun Chen, Zilong Zhu, Zelin Fang, Kaile Guo,

Organizations: The Hong Kong University of Science and Technology (Guangzhou)

Abstract

Deploying Vision-Language-Action (VLA) models on resource-constrained edge platforms encounters a fundamental conflict between high-latency semantic inference and the high-frequency control required for dynamic manipulation. To address the challenge, this paper presents Agile-VLA, a hierarchical framework designed for industrial pose reorientation tasks on edge devices such as the NVIDIA Jetson Orin Nano. The core innovation is an Implicit Affordance Anchoring mechanism that directly maps geometric visual cues, specifically centroid and rim keypoint anchors, into structured parametric action primitives, thereby substantially reducing reliance on high-latency semantic inference during closed-loop control. By decoupling perception (10 Hz) from control (50 Hz) via an asynchronous dual-stream architecture, the system effectively mitigates the frequency mismatch inherent in edge-based robot learning. Experimental results on a standard 6-DoF manipulator demonstrate that Agile-VLA achieves robust rectification of complex, irregular workpieces using only 5-shot demonstrations through extrinsic dexterity.

Figures & tables

Fig. 1: Visualization of Agile-VLA task primitives across diverse industrial workpieces. The top row illustrates Stable Pick and Place operations guided by Stability Anchoring, where the grasp point is optimized toward the geometric centroid to minimize gravitational torque. The bottom row demonstrates Pick, Flip and Place sequences leveraging Pivot Anchoring; here, the anchor point is constrained to the object boundary to maximize the lever arm for extrinsic dexterity. The framework is validated on four irregular objects: (a) Battery, (b) Calculator, (c) Phone, and (d) Chip.

Fig. 2: System Overview. Compared to conventional large-scale VLA models that suffer from high cloud latency, Agile-VLA achieves real-time, few-shot pose reorientation directly on an edge device (NVIDIA Jetson Orin Nano). By decoupling high-level semantic reasoning from low-level control, the framework enables agile manipulation of complex industrial components using only a minimal number of demonstration samples.

Fig. 3: Asynchronous Dual-Stream Framework. The semantic perception stream (10 Hz) is responsible for extracting manipulation anchors, while the proprioceptive control stream (50 Hz) executes parameterized action primitives. A timestamp-based soft synchronization protocol ensures strict alignment between the visual and kinesthetic streams, thereby enabling high-fidelity few-shot adaptation on resource-constrained hardware.

Fig. 4: Implicit Affordance Anchoring Protocol. The system maps visual cues to specific action primitives according to the object’s topological state. Front-facing objects are anchored at their geometric centroid to trigger stable, guidance-oriented grasping, whereas back-facing objects are anchored at the rim to initiate extrinsic-dexterity-based flipping, thereby effectively leveraging environmental constraints to overcome kinematic limitations.

Category	Method	$T_{\mathrm{inf}}$ (ms) $\downarrow$	$f_{\mathrm{ctrl}}$ (Hz) $\uparrow$	VRAM (GB) $\downarrow$	TCP Jitter (mm) $\downarrow$
Sequence	RT-1	$125\pm 12$	8	3.8	3.25
Diffusion	Octo-Base	$245\pm 18$	4	3.2	4.12
LLM-VLA	OpenVLA (4-bit)	$835\pm 42$	1.2	6.8	5.82
Ours	Agile-VLA (Sync)	$\mathbf{92\pm 6}$	10.8	1.8	1.45
Ours	Agile-VLA (Async)	$\mathbf{92}$ (Vis)	50	2.1	0.65

TABLE I: System profiling on Jetson Orin Nano (8GB).

Method	Easy SR (%) $\uparrow$	Hard SR (%) $\uparrow$	Avg SR (%) $\uparrow$	Time (s) $\downarrow$	Collision (%) $\downarrow$	Jerk (m/s 3 ) $\downarrow$
Transporter	74.5	28.6	55.4	11.2	16.5	10.21
RT-1	78.2	34.0	60.1	14.5	18.2	12.84
RVT	82.5	36.5	63.8	12.8	14.2	11.56
Octo-Base	85.0	38.5	61.7	16.8	15.4	14.35
OpenVLA (4-bit)	88.6	46.5	71.0	19.5	21.8	15.62
Agile-VLA	96.5	84.6	90.5	8.4	3.2	1.24

TABLE II: Multi-dimensional performance on DID-127 dataset.

Fig. 5: Real-world multi-task reorientation performance across extreme physical conditions. We systematically benchmark Agile-VLA against state-of-the-art foundation models on six extreme industrial parts (involving heavy masses and complex topologies) and two novel out-of-distribution objects. Crucially , the performance on ’Unseen’ objects reflects the results after our 5-shot edge adaptation pipeline, demonstrating the framework’s rapid deployment capability in contrast to the massive fine-tuning required by large VLA baselines. By leveraging extrinsic pivoting, Agile-VLA successfully bypasses actuator torque overload (e.g., Heavy Battery) and kinematic singularities (e.g., Complex PCB), achieving consistently higher success rates.

Samples ( $N$ )	Pixel Error (px) $\downarrow$	Avg SR (%) $\uparrow$	Tuning Time (min) $\downarrow$
$N=1$	18.4	35	0.8
$N=5$	3.5	90	2.5
$N=10$	3.2	90	5.1

TABLE III: Impact of annotation samples (

N

Async	Anchor	Pivot	SR (%) $\uparrow$	Jitter (mm) $\downarrow$
$\times$ (Sync)	$\checkmark$	$\checkmark$	60.5	1.45
$\checkmark$	$\times$ (Center)	$\checkmark$	43.2	0.68
$\checkmark$	$\checkmark$	$\times$ (Direct)	18.5	0.82
$\checkmark$	$\checkmark$	$\checkmark$	90.5	0.65

TABLE IV: Ablation study on core architecture components.

Affordance-Based Robotic Manipulation

See all 22 papers

LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics
Mar 3, 2026Justin Williams, Kishor Datta Gupta, Roy George +1Efficient VLA Model Inference On-Device Inference
Clark Atlanta University, Atlanta, GA, USA · Siemens Corporation, Princeton, NJ, USA
EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference
Jul 14, 2026Zebin Yang, Qi Wang, Yunhe Wang +6Efficient VLA Model Inference Vision-Language-Action Models
Peking University · TLAIC · PrimeBot Research Institute +1
RhinoVLA Technical Report
Jun 5, 2026Huixi Technology, :, Chen Zhang +13Efficient VLM Inference Efficient VLA Model Inference
Huixi Technology

Mar 3, 2026J/K move · Enter open · S save

LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics

Justin Williams, Kishor Datta Gupta, Roy George, Mrinmoy Sarkar

Clark Atlanta University, Atlanta, GA, USA · Siemens Corporation, Princeton, NJ, USA

Efficient VLA Model Inference On-Device Inference VLMs for Robotics

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models provide a unified framework for perception, language conditioning, and action generation, but many existing systems remain difficult to deploy in embedded robotic settings because of their computational requirements and inference latency. In this paper, we present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware. Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference through the llama.cpp runtime. Under our deployment configuration, LiteVLA-Edge achieves a mean end-to-end latency of 150.5,ms (approximately 6.6,Hz) while operating entirely offline within a ROS~2-integrated perception--reasoning--action pipeline. Rather than introducing a new policy objective, our contribution is a practical systems path for executing compact multimodal control models locally on embedded hardware while preserving modular interfaces between perception, reasoning, and actuation. These results establish timing feasibility for reactive language-conditioned control and provide a reproducible baseline for future task-level evaluation of on-device VLAs in robotics.

Abstract

Affordance-Based Robotic Manipulation

Explore similar work

LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics

EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

RhinoVLA Technical Report