Vision-language-action (VLA) models have made strong progress in language-conditioned robot manipulation, but improving their performance in a new workspace still often requires action-labeled data from that environment. Collecting such data by human teleoperation is costly, especially when each workspace, object arrangement, or task may require new demonstrations. We present DREAM, a framework that generates fine-tuning data for a pretrained VLA from a captured workspace and a language instruction, without requiring a task-specific human demonstration. DREAM reconstructs the workspace, automatically translates the instruction into symbolic task goals and success criteria using a large language model, and uses task-and-motion planning to generate feasible robot trajectories. The planned trajectories are augmented across randomized object configurations, verified by the generated success criteria, and rendered into image-action examples for VLA fine-tuning. Through real-robot experiments on language-conditioned manipulation tasks, we study whether DREAM can serve as a scalable data-collection system for the deployment workspace by examining whether fine-tuning on its automatically generated data improves success over direct deployment and how its data-collection cost compares with human teleoperation when adapting a VLA to a new workspace.
Figures & tables
Method
Teleop- free
Demo- free
Real-to- Sim
Language task
Long- horizon
Scaling Up [ 9 ]
✓
✓
–
✓
✓
OPTIMUS [ 12 ]
✓
✓
–
–
✓
SkillMimicGen [ 15 ]
–
–
–
–
✓
RialTo [ 6 ]
–
–
✓
–
–
RoboSplat [ 20 ]
–
–
✓
–
–
R2R2R [ 5 ]
✓
–
✓
–
–
TABLE I: Comparison of representative robot data-generation methods. ✓ denotes support, while – denotes that the capability is not provided.
Fig. 2: The DREAM pipeline. Given a short RGB video of the target workspace and a language instruction, DREAM reconstructs a metrically aligned digital twin, translates the instruction into symbolic goals and staged success criteria with a large language model, generates feasible robot trajectories with TAMP, augments them across randomized object configurations, and renders them into photorealistic image–action demonstrations. The resulting synthetic dataset is used to fine-tune a pretrained VLA policy, which is then deployed directly in the physical workspace.
Fig. 3: The two tabletop manipulation tasks. For each task, the left image compares the initial state in the reconstructed digital twin (Simulation) with the corresponding real workspace (Real), and the right image shows the target state, rendered in the reconstructed simulation, together with the language instruction. In (a) BlockIntoBowl , the robot places a blue block into a red bowl. In (b) FruitPacking , the robot places a banana and a peach onto a red plate in sequence.
Fig. 4: Sim-to-real correlation of policy performance. Each point represents a policy checkpoint trained with a particular dataset size and random seed. The horizontal axis shows real-world success rate, and the vertical axis shows success rate in the reconstructed simulation. Colors distinguish the two tasks, while marker shapes distinguish the three policy architectures (Diffusion Policy, SmolVLA, and π0.5 ). The dashed line represents equal simulated and real-world performance.
Fig. 5: Real-world success rate as a function of cumulative data acquisition time for BlockIntoBowl (top) and FruitPacking (bottom). Each panel compares SmolVLA and π0.5 trained on demonstrations obtained through human teleoperation or generated by DREAM . Human-teleoperation datasets contain 10, 50, and 100 demonstrations, whereas DREAM datasets contain 100, 500, and 1000 demonstrations.
Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.
Hanchen Cui, Sergio Arnaud, Arjun Majumdar +5
1Fundamental AI Research (FAIR), Meta · University of Minnesota Twin Cities
Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robot control. However, their performance remains fundamentally constrained by the availability of high-quality robot trajectory data. In current robot learning practice, such data are primarily collected through human teleoperation, which is labor-intensive, costly, and difficult to scale. In this paper, we propose RDGen, a sim-to-real reinforcement learning framework for generating high-quality robot demonstrations. Rather than employing reinforcement learning solely as the final control policy, RDGen leverages trained RL policies as a structured trajectory generator. The system consists of a VLM-based task parser that identifies task-relevant objects, a Grounding DINO-based object localizer, and an RL policy transferred from simulation to the real robot. Successful rollouts are then harvested as clean, high-quality demonstrations for downstream VLA training, while the simulation stage further provides a scalable source of additional trajectories at little marginal cost. Experiments on a pick-and-place task demonstrate that the transferred RL policy achieves a high task success rate. Compared with human teleoperation, RDGen produces significantly smoother trajectories and yields superior downstream VLA performance. These results indicate that RL-generated demonstrations can serve as more reliable and consistent supervisory signals for robot policy learning.
Collecting high-quality robot data remains a fundamental challenge for training robot foundation models. Task and motion planning (TAMP) offers a scalable way to generate demonstrations, but our experiments show that raw TAMP trajectories provide surprisingly little benefit when used to fine-tune pretrained vision-language-action (VLA) models, despite successfully solving the target tasks. We hypothesize that this failure arises from a behavioral distribution mismatch between planner-generated trajectories and the data used to pretrain the VLA. To address this mismatch, we introduce DATAFARM: Distribution-Aligned Task And motion planning for Fine-tuning A Robot foundation Model, an approach that incorporates the pretraining distribution directly into TAMP trajectory generation. DATAFARM aligns generated trajectories with the pretraining data in robot joint configurations, motion style, and temporal execution profiles. We evaluate DATAFARM on three tabletop manipulation tasks that TAMP can perform and a cloth-folding task beyond the capability of TAMP. DATAFARM achieves an average success rate of 56.7%, substantially outperforming raw TAMP (8.3%) while approaching human teleoperation (61.7%). On Deformable Object Manipulation, which is outside the fine-tuning distribution, the fine-tuned model retains 85% success, compared with 90% for the pretrained model. These results show that aligning planner-generated demonstrations with the pretraining distribution can make TAMP an effective source of data for VLA fine-tuning. Website and code: https://prpl-group.com/datafarm/