MobiGuide: Vocabulary-Guided Online Recovery for Mobile Manipulation

Jia Syuen Lim; Zhizhen Zhang; Peter Bohm; Brendan Tidd; Zi Huang; Yadan Luo

cs.ROApr 2, 2026

MobiGuide: Vocabulary-Guided Online Recovery for Mobile Manipulation

Authors: Jia Syuen Lim, Zhizhen Zhang, Peter Bohm, Brendan Tidd, Zi Huang, Yadan Luo

Organizations: UQMM Lab, The University of Queensland, Brisbane, Australia · Robotics and Autonomous Systems Group, CSIRO, Brisbane, Australia

Abstract

A central challenge in mobile manipulation is preserving multiple plausible action modes while remaining reactive during execution. For example, grasping a bottle in a cluttered scene can be achieved through multiple physically feasible base-arm motion plans. Deterministic VLA policies trained with $L_1$ regression are efficient, but they tend to average across valid modes and may produce physically invalid actions. Diffusion-based generative policies can better represent multimodal action distributions, yet their proposals may be weakly constrained when generated from pure noise, and iterative sampling can increase inference cost. Action chunking helps amortize inference, but it also introduces partially open-loop execution, allowing small mismatches to accumulate into drift. We present MobiGuide, a generative VLA model for efficient mobile manipulation, where action generation is guided by a structured trajectory vocabulary. Constructed from expert demonstrations, this vocabulary encodes plausible whole-body motion patterns and provides structured priors for feasible action generation. Given the current observation and instruction, MobiGuide locally refines these reference trajectories into context-compatible action proposals and selects the most suitable trajectory for execution. This preserves multimodal action candidates while keeping generation constrained around feasible expert-like motion modes. To mitigate chunk-induced drift, we further introduce a lightweight residual correction module that performs high-frequency, per-step adjustments during rollout. Across diverse mobile manipulation tasks, MobiGuide improves success and stability under execution disturbances while maintaining low-latency inference.

Figures & tables

Fig. 2 : Comparison between uni-modal prediction and multimodal vocabulary-guided local refinement. MobiGuide starts from a demonstration-derived vocabulary and refines locally, constraining generation to plausible actions.

Fig. 3 : During chunked execution, residual correction predicts state-dependent micro-adjustments to compensate for base drift and arm–object misalignment without repeatedly querying the full VLA policy.

Fig. 4 : Overview of the MobiGuide framework. The VLA backbone encodes multimodal context. The action head refines demonstration-derived trajectory anchors into context-conditioned proposals, and a scoring head selects the highest-scoring context-compatible proposal.

Fig. 5 : Experimental Environment. Simulation experiments are conducted in ManiSkill-HAB. Real-world experiments are evaluated on the Unitree Go2 quadruped equipped with an SO101 arm. The tasks need the robot to first navigate to the target place, then interact with the objects. The perception module includes two camera views, i.e., base view from Go2 and wrist view from SO101. The data collection process was conducted by teleoperation with the Meta Quest 3 VR device.

Methodology	Venue	Visual Input	Pick		Place		Open	Close	Average
			Apple	Bowl	Apple	Bowl	Door	Drawer
			Fridge	Counter	Table	Table	Fridge	Counter
$\pi_{0}$	–	RGB	$13.0\pm 1.6$	$15.7\pm 1.2$	$23.3\pm 1.7$	$21.3\pm 2.6$	$31.3\pm 2.6$	$70.0\pm 2.2$	29.10
ACT	RSS’23	RGB	$28.0\pm 2.2$	$28.0\pm 2.4$	$8.7\pm 3.3$	$13.0\pm 0.8$	$2.0\pm 2.2$	$85.7\pm 1.2$	27.57
RDT	ICLR’25	RGB	$12.0\pm 11.3$	$10.7\pm 6.8$	$32.0\pm 5.7$	$18.7\pm 5.0$	$82.7\pm 10.5$	$\mathbf{100.0\pm 0.0}$	42.68
DP	RSS’23	RGB	$21.3\pm 3.3$	$20.7\pm 3.3$	$28.0\pm 8.0$	$69.3\pm 3.3$	$7.3\pm 5.8$	$55.0\pm 5.7$	33.60

TABLE I : Simulation results and residual correction ablation on ManiSkill-HAB. We report Success Once (%, mean

\pm

standard deviation across repeated evaluations). Visual Input denotes the visual observation modality; all methods additionally receive the shared proprioceptive and task-state inputs described in Sec. IV -A.

H

denotes the commit length. For MobiGuide ,

\dagger

denotes variants without residual correction; unmarked variants include residual correction. Average is the unweighted mean across the six tasks. Bold indicates the highest mean in each column.

Method	Open Drawer	Lift Bag	Average
ACT	10%	0%	5%
Diffusion Policy	20%	0%	10%
SmolVLA	20%	10%	15%
MobiGuide	70%	50%	60%

TABLE II : Success rate comparison between MobiGuide and baselines on real-world tasks.

Method	Steps	SR (%) $\uparrow$	Latency (ms) $\downarrow$
w/o Anchor	50	$4.67\pm 0.58$	$938.37\pm 2.48$
w/o Anchor	25	$4.00\pm 0.00$	$501.09\pm 2.34$
w/o Anchor	10	$5.67\pm 0.58$	$244.91\pm 1.76$
w/ Anchor	10	$\mathbf{32.67\pm 0.58}$	$249.72\pm 2.89$
w/ Anchor	5	$26.00\pm 1.73$	$\mathbf{155.81\pm 2.26}$

TABLE III : Anchor-based and Gaussian-initialized generation under different denoising-step settings on Pick Bowl . Values are mean

\pm

standard deviation across three trials. Latency measures a complete batch-one action-chunk prediction, including preprocessing, VLM inference, and diffusion. All experiments use

H=5

and residual correction is disabled.

Task	0%		10%		20%
Task	$-$	$+$	$-$	$+$	$-$	$+$
Pick Apple	15.7	22.3	11.7	17.0	11.0	17.7
Pick Bowl	32.7	40.3	20.7	33.0	11.3	16.0
Place Apple	63.7	63.0	64.0	68.7	61.0	63.7
Place Bowl	73.0	79.0	73.3	75.3	65.3	67.7
Open Fridge	53.7	77.0	38.7	72.7	29.7	53.0

TABLE IV : Success Once (%) under action attenuation at

H=5

, without (

-

) and with (

+

) residual correction. Bold indicates higher success within each pair.

Fig. 6 : Qualitative visualization of MobiGuide in ManiSkill-HAB. We show task progress from left to right for Pick Bowl and Place Apple . White points denote candidate future end-effector trajectories reconstructed from anchored proposals, while colored points indicate the trajectory selected by the scoring head for execution.

Fig. 7 : Effect of commit length $H$ on Pick Bowl success. MobiGuide outperforms $L_{1}$ regression across all settings, with a smaller overall decline from $H=1$ to $H=10$ (53.0% to 36.0%, versus 42.7% to 8.7%).

Fig. 8 : Success–latency trade-off across vocabulary sizes $M$ . Success is averaged over Pick Apple and Pick Bowl; latency is the median batch-one inference time, with error bars indicating the 5th–95th percentiles.

Mobile Manipulation

Mobile manipulation studies robots that navigate through environments while using arms or other effectors to interact with objects.

See all 92 papers

Jun 12, 2026

ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation

Diffusion-based Vision-Language-Action (VLA) policies have demonstrated strong capability in modeling expressive and multimodal action distributions. However, their reliance on iterative sampling introduces substantial inference latency, which limits their applicability to reactive closed-loop robot manipulation. To address this limitation, we propose ReactVLA, a lightweight and low-latency VLA framework for real-time robotic manipulation. ReactVLA combines two complementary designs: (1) an improved Mean Flow (iMF) action generator that reduces expensive multi-step diffusion sampling to one-to-few-step action generation, and (2) Attention Residuals (AttnRes), a dynamic depth-wise feature routing mechanism that replaces uniform residual accumulation to better preserve task-relevant multimodal representations. We evaluate ReactVLA on large-scale simulation benchmarks, including LIBERO and RoboIMI, as well as real-world robotic manipulation tasks. Experimental results show that ReactVLA consistently outperforms similarly sized VLA baselines, including SmolVLA and

π_0

. On challenging precision manipulation tasks, ReactVLA achieves up to a 1.65

\times

improvement in task performance while providing more than a 4

\times

increase in inference speed compared with leading VLA models. Finally, it reduces real-world policy latency to below 38.6 ms, enabling fast reactive control on physical robot platforms. Please check out our project website at: https://game-loader.github.io/ReactVLA/.

Yanzhao Guo, Wenkai Chen, Jianwei Zhang

Shanghai Jiao Tong University · Technical Aspects of Multimodal Systems (TAMS), Department of Informatics, Universität Hamburg

Language-Conditioned Robot Manipulation Efficient VLM Inference Attention Residual Connections Vision-Language-Action Models

Ask PDF

Mar 13, 2026

AnchorVLA4D: an Anchor-Based Spatial-Temporal Vision-Language-Action Model for Robotic Manipulation

Since current Vision-Language-Action (VLA) systems suffer from limited spatial perception and the absence of memory throughout manipulation, we investigate visual anchors as a means to enhance spatial and temporal reasoning within VLA policies for robotic manipulation. Conventional VLAs generate actions by conditioning on a single current frame together with a language instruction. However, since the frame is encoded as a 2D image, it does not contain detailed spatial information, and the VLA similarly lacks any means to incorporate past context. As a result, it frequently forgets objects under occlusion and becomes spatially disoriented during the manipulation process. Thus, we propose AnchorVLA4D, a simple spatial-temporal VLA that augments the visual input with an anchor image to preserve the initial scene context throughout execution, and adds a lightweight spatial encoder that jointly processes the anchor and current frames to expose geometric relationships within an episode. Built on a Qwen2.5-VL backbone with a diffusion-based action head, AnchorVLA4D requires no additional sensing modalities (e.g., depth or point clouds) and introduces negligible inference overhead. Combining anchoring with a frozen pretrained spatial encoder yields further gains, realizing a 13.6% improvement on the Simpler WidowX benchmark and confirming the approach on real-world tasks, where it achieved an average success rate of 80%.

Juan Zhu, Zhanying Shao, Xiaoqi Li +4

PrimeBot · School of Computer Science, Peking University

Vision-Language-Action Models Language-Conditioned Robot Manipulation Memory-Augmented VLMs

Ask PDF

Sep 20, 2026

TaskAnchor: Grounding Task State in Reactive VLAs for Long-Horizon Manipulation

Reactive vision-language-action (VLA) policies suffer from task-state aliasing in long-horizon manipulation, where identical multimodal inputs call for distinct, context-dependent actions. Given that pretrained VLAs already possess rich control primitives to express diverse behaviors, we hypothesize that the execution bottleneck lies not in policy capacity, but in input ambiguity. In this paper, we propose TaskAnchor, a lightweight adapter that grounds task state by injecting execution context into the VLA's native input space. During post-training, TaskAnchor learns to represent the semantic execution stage as a milestone-supervised coordinate prepended to the language instruction, while incorporating fine-grained historical evidence via a residual update to the current visual tokens. This formulation avoids generating complex subtask instructions and leaves the backbone architecture unchanged. Across long-horizon benchmarks, TaskAnchor delivers substantial gains, achieving approximately 6 times the average success rate of the pi0.5 and X-VLA baselines on RMBench and more than doubling the task success rate of pi0.5 on RoboMemArena. Real-robot experiments further validate reliable multi-stage execution, with the same policy adapting its subsequent behaviors using earlier human interactions as in-context cues. Our project website is available at https://taskanchor.netlify.app/.

Hengyan Liu, Wenlve Zhou, Bo Yue +7

The Chinese University of Hong Kong, Shenzhen · Foshan University · DexForce +1

Memory-Augmented VLMs Long-Horizon Robotic Manipulation Adapter Tuning Vision-Language-Action Models

Ask PDF

Abstract

Mobile Manipulation

Explore similar work

ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation

AnchorVLA4D: an Anchor-Based Spatial-Temporal Vision-Language-Action Model for Robotic Manipulation

TaskAnchor: Grounding Task State in Reactive VLAs for Long-Horizon Manipulation