SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation

Kyoungin Baik; Youngwoon Lee

cs.ROOct 8, 2026

SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation

Authors: Kyoungin Baik, Youngwoon Lee

Organizations: Yonsei University · Seoul National University

Abstract

Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection. While simulation offers a scalable alternative, its potential for sim-to-real VLA learning in mobile manipulation remains largely underexplored. We introduce SimVLA, an end-to-end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation. SimVLA is first pre-trained on two complementary simulation-derived datasets: SimAction, a large-scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask-level visual-language supervision. We further post-train SimVLA on a mixture of SimAction and SimDeploy, a dataset collected from policy rollouts across diverse simulated environments. We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero-shot transfer to real-world mobile manipulation, including real home environments. SimVLA outperforms policies trained on 50 in-domain real-world demonstrations, suggesting that simulation can enable scalable sim-to-real mobile manipulation. We further demonstrate the value of multiple complementary forms of supervision for effectively leveraging simulation in VLA training.

Figures & tables

Dataset	# Task	Mobility	V-L sup.	Data generation	Zero-shot Sim-to-Real
MimicGen ( Mandlekar et al., 2023 )	18	$\bigcirc$	✗	Teleoperation & Augmentation	✗
RoboGen ( Wang et al., 2024 )	106	$\bigcirc$	✗	Motion planning & Optimization & RL	✗
SkillMimicGen ( Garrett et al., 2025 )	18	✗	✗	Teleoperation & Augmentation	✗
DexMimicGen ( Jiang et al., 2025 )	9	$\bigcirc$	✗	Teleoperation & Augmentation	$\bigcirc$
GraspVLA ( Deng et al., 2025 )	1	✗	$\bigcirc$	Motion planning	$\bigcirc$
RoboTwin 2.0 ( Chen et al., 2026 )	50	✗	✗	Motion planning	$\bigcirc$

Table 1: Comparison of simulation data generation methods. “V-L sup.” indicates visual-language supervision beyond action trajectories, such as VQA-style annotations. “Policy rollouts” denotes data collected by rolling out a learned policy in simulation.

Figure 1: Overview of SimVLA Training. During pre-training, SimVLA jointly optimizes continuous action prediction, tokenized action prediction, and SimVQA supervision. During post-training, SimVLA is trained on a mixture of SimAction and deployment trajectories from SimDeploy.

Panel of Figure 2 — Figure 2: Five long-horizon mobile manipulation tasks used for real-world evaluation.

Panel of Figure 3 — Figure 3: Real-world evaluation setup. Top: three evaluation environments: ( 3(a) ) mock kitchen, ( 3(b) ) pantry, and ( 3(c) ) real home. Bottom: mock kitchen evaluation conditions: ( 3(d) ) In-Distribution, ( 3(e) ) OOD-pos, and ( 3(f) ) OOD-vis.

Figure 4: Averaged task progress in real-world environments. Mock kitchen includes all five tasks; pantry includes put mug / bowl / bottle in sink ; real home includes put mug in sink / pour bottle to mug . SimVLA maintains robust performance in OOD environments, while the fine-tuned policy degrades substantially. Removing pre-training leads to poor performance, demonstrating its importance for robust generalization.

Figure 5: Averaged task progress across tasks in mock kitchen-ID. Removing either SimVQA or Tokenized SimAction degrades performance across all tasks. SimDeploy yields task-dependent gains in zero-shot deployment performance.

Panel of Figure 6 — Figure 6: Sampled drawer-grasping frames from real-world teleoperation and simulation. Teleoperation trajectories are visually homogeneous, while simulation exhibits greater diversity.

Pick
	MolmoB0T	SimVLA
MolmoSpaces	36.3 $\pm$ 2.8	71.0 $\pm$ 2.9
SimVLA kitchens	9.7 $\pm$ 1.7	91.2 $\pm$ 2.8
PnP
MolmoSpaces	1.0 $\pm$ 0.7	49.0 $\pm$ 3.4
SimVLA kitchens	0.5 $\pm$ 0.5	79.5 $\pm$ 3.1

Table 2: Zero-shot mobile manipulation. Success rate (%); Pick { mug, bottle, apple }, PnP { apple in bowl, mug next to bowl }. SimVLA is post-trained without MolmoSpaces SimDeploy. SimVLA remains robust under partial observability. None of the environments are included in the training dataset.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: The five kitchen layout types used in SimVLA: peninsula, L-shaped, U-shaped, island, and single-wall. Each layout is procedurally generated with randomized furniture configurations, object placements, and surface materials.

Figure 8: 40 bowl variants used in SimVLA, spanning diverse shapes, sizes, depths, and surface materials for robust sim-to-real generalization.

Figure 10: Real-world search behavior. Given only a language prompt and RGB observations, the policy executes searching motions until the sink is visible, then proceeds to complete the task.

Figure 11: BODex grasp pose generation and selection GUI.

Figure 12: Massive parallel environments (4096 instances) running simultaneously in Isaac Lab for hybrid system identification.

Objective	L2 mean (mm) $\downarrow$	L2 max (mm) $\downarrow$
None (URDF defaults)	$83.4\pm 50.2$	$120.7\pm 96.0$
Joint-space	$14.7\pm 2.3$	$46.2\pm 9.6$
EEF-space	$13.4\pm 1.7$	$40.8\pm 8.4$
Hybrid (joint + EEF)	$\mathbf{13.3\pm 1.6}$	$\mathbf{40.6\pm 8.1}$

Appendix

Table 3: System-identification ablation. EEF position discrepancy over ten held-out trajectories on both arms;

\pm

denotes standard deviation across trajectories.

ID	Task	Description
1	Move to mug	The robot navigates and positions itself near the mug.
2	Move to bowl	The robot navigates and positions itself near the bowl.
3	Move to bottle	The robot navigates and positions itself near the bottle.
4	Move to drawer	The robot navigates and positions itself in front of the drawer.
5	Put mug in sink	The robot navigates to the mug, grasps it, moves to the sink, and places it into the sink.
6	Put bowl in sink	The robot navigates to the bowl, grasps it, moves to the sink, and places it into the sink.

Appendix

Table 4: Task definitions.

Domain	Randomness
Camera: Position Noise (m)	$\sim\mathcal{N}(0,0.03^{2})$
Camera: Rotation Noise (deg)	$\sim\mathcal{U}(-5,5)$
Kitchen: Types	5
Kitchen: Tables	50
Kitchen: Chairs	31
Kitchen: Cabinet Materials	21

Appendix

Table 5: Domain Randomization Setup.

VQA Function	Question Prompt Prototype	Answer Prototype
high_level_subtask	To complete the task “{task}”, what should the robot do now?	{subtask}
next_subtask	After current subtask {subtask}, what is the next subtask?	{subtask}
object_detection	Is the {object} visible in the front camera?	yes. bbox: {y1,x1,y2,x2} / no
object_3d_information	How far is the {object} from the robot base?	dx {x} dy {y} dz {z}
view_correspondence	Given front bbox {bbox}, what are the wrist camera bboxes?	left {bbox} right {bbox}
object_reachable	Is the robot close enough to manipulate the {object}?	yes / no. move base dx {x} dy {y} dyaw {yaw}

Appendix

Table 6: SimVQA Templates

Hyperparameter	Pretrain	Post-train
Global batch size	256	256
Precision	BF16	BF16
Gradient clipping (L2)	1.0	1.0
Peak learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$
End learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$
LR scheduler	cosine	cosine

Appendix

Table 7: Key hyperparameters for SimVLA training.

Hyperparameter	Value
FAST loss weight	0.1
VQA loss weight	0.1
FAST sampling ratio	0.50
Co-train sampling ratio	0.01
Discrete state input	false
Delta joint actions	true

Appendix

Table 8: Co-training and FAST auxiliary-loss hyperparameters used during SimVLA pretraining.

Stage	Hardware	CPU	System RAM	Storage	Per-run compute	Total compute
Pre-training	4 $\times$ NVIDIA B200 (183GB)	128 CPU cores	1.5 TB	NVMe SSD	$\sim$ 12 days	$\sim$ 1152 GPU-hours
Post-training	4 $\times$ NVIDIA B200 (183GB)	128 CPU cores	1.5 TB	NVMe SSD	$\sim$ 30 hours	$\sim$ 120 GPU-hours

Appendix

Table 9: Compute resources used for SimVLA experiments. All experiments were conducted on an internal GPU cluster. Reported times are approximate wall-clock durations and may vary depending on storage and data-loading throughput.

Model	Gripper $\uparrow$	Subtask $\uparrow$	3D Pos. $\downarrow$	Grasp $\downarrow$	Cross-view IoU $\uparrow$
	L / R	Curr. / Next	(m)	(m)
SimVLA w/o SimVQA	– ∗	– ∗	– ∗	– ∗	– ∗
SimVLA w/o TokSimAction	0.86 / 0.69	0.18 / 0.39	0.50	0.16	0.04
SimVLA w/o both	0.85 / 0.68	– †	– †	– †	– †
SimVLA	1.00 / 0.89	0.69 / 0.77	0.06	0.05	0.61

Appendix

Table 11: Held-out 1,369 SimVQA evaluation. Ablation study on visual reasoning capabilities.

\uparrow

indicates higher is better and

\downarrow

indicates lower is better.

Task	Success Rate
Put mug to sink	76%
Put bowl in drawer	70%
Pour from bottle to mug	76%
Put mug and bowl to sink	38%
Put bowl to sink	62%
Put bottle to sink	64%

Appendix

Table 12: SimVLA success rates on eight long-horizon simulation tasks using Anubis robot. Each task is evaluated over 50 rollouts across five unseen kitchen setups.

Model	Task 1	Task 2	Task 3	Task 4	Task 5
$\pi_{0.5}$ fine-tuned w/ 50 demos	0.55 $\pm$ 0.43 [1pt] (8/20)	0.36 $\pm$ 0.39 [1pt] (5/20)	0.48 $\pm$ 0.38 [1pt] (6/20)	0.45 $\pm$ 0.46 [1pt] (7/20)	0.43 $\pm$ 0.47 [1pt] (7/20)
$\pi_{0.5}$ fine-tuned w/ 200 demos	0.75 $\pm$ 0.44 [1pt] (15/20)	0.66 $\pm$ 0.42 [1pt] (11/20)	0.70 $\pm$ 0.36 [1pt] (10/20)	0.65 $\pm$ 0.43 [1pt] (11/20)	0.63 $\pm$ 0.46 [1pt] (11/20)
SimVLA (w/o pre-train)	0.28 $\pm$ 0.38 [1pt] (3/20)	0.15 $\pm$ 0.16 [1pt] (0/20)	0.15 $\pm$ 0.25 [1pt] (1/20)	0.18 $\pm$ 0.34 [1pt] (2/20)	0.18 $\pm$ 0.29 [1pt] (1/20)
SimVLA (w/o SimDeploy)	0.45 $\pm$ 0.32 [1pt] (3/20)	0.38 $\pm$ 0.31 [1pt] (3/20)	0.45 $\pm$ 0.33 [1pt] (4/20)	0.33 $\pm$ 0.34 [1pt] (2/20)	0.33 $\pm$ 0.34 [1pt] (2/20)
SimVLA	0.60 $\pm$ 0.42 [1pt] (9/20)	0.50 $\pm$ 0.36 [1pt] (6/20)	0.52 $\pm$ 0.38 [1pt] (6/20)	0.55 $\pm$ 0.46 [1pt] (9/20)	0.55 $\pm$ 0.43 [1pt] (8/20)

Appendix

Table 13: Mock Kitchen — ID. The

\pi_{0.5}

model fine-tuned with 200 demonstrations achieves the best performance on all tasks. SimVLA outperforms

\pi_{0.5}

fine-tuned on 50 real-world demonstrations in task progress across all tasks, with comparable or higher binary success. Task progress

\pm

s.d. over 20 trials; binary successes in parentheses.

Model	Task 1	Task 2	Task 3	Task 4	Task 5
$\pi_{0.5}$ fine-tuned w/ 50 demos	0.40 $\pm$ 0.38 [1pt] (4/20)	0.33 $\pm$ 0.37 [1pt] (4/20)	0.40 $\pm$ 0.37 [1pt] (4/20)	0.30 $\pm$ 0.44 [1pt] (5/20)	0.25 $\pm$ 0.41 [1pt] (4/20)
$\pi_{0.5}$ fine-tuned w/ 200 demos	0.45 $\pm$ 0.39 [1pt] (5/20)	0.43 $\pm$ 0.40 [1pt] (6/20)	0.48 $\pm$ 0.35 [1pt] (4/20)	0.35 $\pm$ 0.46 [1pt] (6/20)	0.28 $\pm$ 0.44 [1pt] (5/20)
SimVLA (w/o pre-train)	0.23 $\pm$ 0.34 [1pt] (2/20)	0.17 $\pm$ 0.18 [1pt] (0/20)	0.13 $\pm$ 0.17 [1pt] (0/20)	0.10 $\pm$ 0.26 [1pt] (1/20)	0.25 $\pm$ 0.38 [1pt] (3/20)
SimVLA (w/o SimDeploy)	0.45 $\pm$ 0.36 [1pt] (4/20)	0.36 $\pm$ 0.31 [1pt] (3/20)	0.43 $\pm$ 0.38 [1pt] (5/20)	0.38 $\pm$ 0.32 [1pt] (2/20)	0.50 $\pm$ 0.32 [1pt] (4/20)
SimVLA	0.58 $\pm$ 0.41 [1pt] (8/20)	0.50 $\pm$ 0.36 [1pt] (6/20)	0.52 $\pm$ 0.37 [1pt] (6/20)	0.60 $\pm$ 0.45 [1pt] (10/20)	0.68 $\pm$ 0.41 [1pt] (11/20)

Appendix

Table 14: Mock Kitchen — OOD-Pos. SimVLA is best on task progress across all tasks, and best-or-comparable on binary success. Task progress

\pm

s.d. over 20 trials; binary successes in parentheses.

Model	Task 1	Task 2	Task 3	Task 4	Task 5
$\pi_{0.5}$ fine-tuned w/ 50 demos	0.35 $\pm$ 0.46 [1pt] (6/20)	0.37 $\pm$ 0.39 [1pt] (5/20)	0.40 $\pm$ 0.37 [1pt] (4/20)	0.50 $\pm$ 0.49 [1pt] (9/20)	0.30 $\pm$ 0.41 [1pt] (4/20)
$\pi_{0.5}$ fine-tuned w/ 200 demos	0.45 $\pm$ 0.48 [1pt] (8/20)	0.45 $\pm$ 0.42 [1pt] (7/20)	0.45 $\pm$ 0.39 [1pt] (5/20)	0.53 $\pm$ 0.47 [1pt] (9/20)	0.48 $\pm$ 0.47 [1pt] (8/20)
SimVLA (w/o pre-train)	0.20 $\pm$ 0.34 [1pt] (2/20)	0.13 $\pm$ 0.15 [1pt] (0/20)	0.13 $\pm$ 0.17 [1pt] (0/20)	0.18 $\pm$ 0.29 [1pt] (1/20)	0.08 $\pm$ 0.18 [1pt] (0/20)
SimVLA (w/o SimDeploy)	0.38 $\pm$ 0.36 [1pt] (3/20)	0.29 $\pm$ 0.28 [1pt] (2/20)	0.37 $\pm$ 0.36 [1pt] (4/20)	0.43 $\pm$ 0.34 [1pt] (3/20)	0.43 $\pm$ 0.29 [1pt] (2/20)
SimVLA	0.63 $\pm$ 0.39 [1pt] (9/20)	0.40 $\pm$ 0.34 [1pt] (4/20)	0.47 $\pm$ 0.38 [1pt] (6/20)	0.58 $\pm$ 0.41 [1pt] (8/20)	0.60 $\pm$ 0.38 [1pt] (8/20)

Appendix

Table 15: Mock Kitchen — OOD-Vis. SimVLA achieves the best performance on Tasks 1, 3, 4 and 5, whereas the

\pi_{0.5}

model fine-tuned with 200 demonstrations performs best on Task 2. Relative to OOD-Pos,

\pi_{0.5}

holds up better under OOD-Vis shifts, likely due to image augmentation during its fine-tuning. Task progress

\pm

s.d. over 20 trials; binary successes in parentheses.

Model	Task 1	Task 2	Task 3	Task 4	Task 5
SimVLA w/o SimVQA	0.43 $\pm$ 0.47 [1pt] (7/20)	0.36 $\pm$ 0.41 [1pt] (5/20)	0.37 $\pm$ 0.40 [1pt] (5/20)	0.50 $\pm$ 0.43 [1pt] (7/20)	0.45 $\pm$ 0.43 [1pt] (6/20)
SimVLA w/o TokSimAction	0.40 $\pm$ 0.38 [1pt] (4/20)	0.29 $\pm$ 0.35 [1pt] (3/20)	0.42 $\pm$ 0.37 [1pt] (5/20)	0.43 $\pm$ 0.41 [1pt] (5/20)	0.50 $\pm$ 0.43 [1pt] (7/20)
SimVLA w/o both	0.38 $\pm$ 0.36 [1pt] (3/20)	0.26 $\pm$ 0.35 [1pt] (3/20)	0.37 $\pm$ 0.36 [1pt] (4/20)	0.48 $\pm$ 0.44 [1pt] (7/20)	0.45 $\pm$ 0.43 [1pt] (6/20)
SimVLA	0.60 $\pm$ 0.42 [1pt] (9/20)	0.50 $\pm$ 0.36 [1pt] (6/20)	0.52 $\pm$ 0.38 [1pt] (6/20)	0.55 $\pm$ 0.46 [1pt] (9/20)	0.55 $\pm$ 0.43 [1pt] (8/20)

Appendix

Table 16: Pretraining Ablation , same tasks as submitted, 20 rollouts for each evaluation in mock kitchen-ID. Removing either SimVQA or TokSimAction lowers task progress on every task relative to full SimVLA; removing SimVQA degrades control mainly through less precise grasping. Task progress

\pm

s.d.; binary successes in parentheses.

	Pantry (20 trials)			Real Home (10 trials)
Model	Task 1	Task 4	Task 5	Task 1	Task 3
$\pi_{0.5}$ fine-tuned w/ 50 Mock Kitchen demos	0.10 $\pm$ 0.21 [1pt] (0/20)	0.08 $\pm$ 0.18 [1pt] (0/20)	0.18 $\pm$ 0.24 [1pt] (0/20)	0.20 $\pm$ 0.26 [1pt] (0/10)	0.17 $\pm$ 0.18 [1pt] (0/10)
$\pi_{0.5}$ fine-tuned w/ 200 Mock Kitchen demos	0.23 $\pm$ 0.30 [1pt] (1/20)	0.23 $\pm$ 0.30 [1pt] (1/20)	0.18 $\pm$ 0.29 [1pt] (1/20)	0.35 $\pm$ 0.24 [1pt] (0/10)	0.23 $\pm$ 0.16 [1pt] (0/10)
SimVLA w/o pre-train	0.10 $\pm$ 0.21 [1pt] (0/20)	0.10 $\pm$ 0.26 [1pt] (1/20)	0.08 $\pm$ 0.18 [1pt] (0/20)	0.10 $\pm$ 0.21 [1pt] (0/10)	0.10 $\pm$ 0.16 [1pt] (0/10)
SimVLA	0.60 $\pm$ 0.38 [1pt] (8/20)	0.53 $\pm$ 0.47 [1pt] (9/20)	0.58 $\pm$ 0.44 [1pt] (9/20)	0.70 $\pm$ 0.42 [1pt] (6/10)	0.47 $\pm$ 0.42 [1pt] (3/10)

Appendix

Table 17: Pantry and Real Home. Task progress

\pm

s.d.; binary successes in parentheses. Pantry is evaluated on Tasks 1, 4 and 5 with 20 trials each; Real Home on Tasks 1 and 3 with 10 trials each.

Explore similar work

MobileManiBench: Simplifying Model Verification for Mobile Manipulation
Feb 5, 2026Wenbo Wang, Fangyun Wei, QiXiu Li +5Mobile Manipulation Robotics Simulation
University of Sydney · 1Microsoft Research Asia · 3Tsinghua University
Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement
Jun 17, 2026Kinam Kim, Namiko Saito, Heecheol Kim +3Reinforcement Learning Zero-Shot Robotic Manipulation
KAIST, South Korea · Microsoft Research Asia - Tokyo, Japan · The University of Tokyo, Japan
SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation
Jun 13, 2026Wenkang Hu, Haoran Wang, Yitong Li +10Zero-Shot Robotic Manipulation Robot Skill Learning
1Shanghai Jiao Tong University · 2Horizon Robotics · 3Style3D Research

Feb 5, 2026, cs.ROJ/K move · Enter open · S save

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Wenbo Wang, Fangyun Wei, QiXiu Li, Xi Chen+4

University of Sydney · 1Microsoft Research Asia · 3Tsinghua University

Mobile Manipulation Robotics Simulation Synthetic Data Generation for Robotics Vision-Language-Action Models Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.

Abstract

Explore similar work

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement

SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation