General-purpose robots must handle tasks, objects and environments they were never trained on, yet zero-shot generalization remains one of the least solved problems in physical AI. In digital AI, zero-shot capability grew with the volume and, above all, the diversity of training data, much of it sourced from the crowd. Robotics instead relies on paid collection by a few well-funded teams, which yields volume without diversity. We formulate zero-shot generalization in physical AI as coverage of the deployment distribution, and show that more samples from the same collectors shrink the unseen region only slowly, while each new contributor can extend it. Building on this and on zero-shot learning, we propose a shared latent space for robot actions that combines multi-modal alignment, generative augmentation, and noisy-label learning. As a scalable source of diverse data, we propose a community of developers building applications on one standard, approachable robot, Origami. We close with a testable hypothesis and an evaluation protocol relating real-robot zero-shot success to the diversity of contributed data.
Figures & tables
Figure 2: Volume makes each skill reliable; diversity determines whether a new task falls within reach. A robot trained on one team’s data cannot place an unfamiliar kettle in an unfamiliar kitchen. A robot trained on diverse community data reaches it by combining skills it already has.
Source
Cost / hour
Diversity
Embodiment
Scales with
Paid teleoperation
High
Low
Match
Budget
Egocentric human video
Very low
High
Gap
Budget
Simulation only
Low
Designed
Sim gap
Compute
Research community
Low
Medium
Match
Papers
Developer community (ours)
Low
High
Match
Users
Table 1: Sources of robot training data. Diversity is how widely a source covers situations in Pworld ; embodiment is whether the data comes from the robot that will use it. Only a developer community scales with the number of people solving their own problems on the target robot.
Figure 3: The training platform in an online environment: teaching by demonstration, designing policies, and setting challenges.
Figure 4: Origami Dual-Arm: an upper-body platform with 14 DoF that learns by record-and-replay and has a browser twin for training before deployment.
Figure 5: The vision for Origami: a 27-DoF bipedal humanoid with multi-view sensing and on-board AI compute. Specifications are design targets.
Origami Dual-Arm
Origami Premium
DoF
14 (neck 2, arms 6 each)
27 (neck 3, arms 7, legs 5 each)
Size / mass
57 cm, 2.8 kg (upper body)
∼ 110 cm, ≈ 18–25 kg
Manipulation
2 × 6-DoF arms, 1-DoF grippers
2 × 7-DoF arms, 1-DoF grippers, 600 g payload
Locomotion
Stationary
Bipedal walking
Sensing
Camera, IMU, Joint position and load
3 wide cameras (head, chest, hand), IMU,
per-actuator position, current, temperature
Table 2: Origami specifications. The Premium version includes every feature of the dual-arm version and adds the capabilities listed. Humanoid values are design targets and may change.
Figure 6: Growth of all employees in computer system designs and related services from 1990 to 2023 in the US.
Figure 7: Columns: simulation (left) and real robot (right). Rows: seen task (top) and unseen task (bottom). The target is the highlighted quadrant: an unseen task solved on the real robot using knowledge built largely in simulation.
Sequential robot manipulation requires policies to execute novel combinations of familiar instruction components. However, collecting demonstrations for all possible instruction tuples is combinatorially expensive, while sparsely covered datasets often fail under out-of-distribution recombination. This paper studies compositional generalization through the lens of instruction-space coverage. We decompose the generalization gap into three sources: marginal instruction shift, instruction-compositional shift, and context--action shift. This decomposition allows us to diagnose when sparse training coverage is sufficient, and what structure the training set must preserve for reliable action prediction. Our results show that exhaustive tuple enumeration is unnecessary: a structured subset can recover strong out-of-distribution performance when it covers action-relevant dependencies. We further find that sparse training often fails due to instruction steering rather than missing low-level skills; finetuning with only one demonstration per task improves OOD success from 0.4% to 54.7%. For semantically dependent tasks, effective coverage must capture relational structure rather than only factor diversity. These findings suggest that efficient robot data collection should prioritize dependency coverage in instruction space over exhaustive task expansion. Additional results, code, and dataset are available on the project website.
Yixiao Wang, Cheng-En Wu, Lingfeng Sun +5
University of California, Berkeley · Tsinghua University
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone. Unlike VLAs, WAMs learn physical dynamics by predicting future world states and actions, using video as a dense representation of how the world evolves. By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data without relying on repetitive demonstrations. This results in over 2x improvement in generalization to new tasks and environments compared to state-of-the-art VLAs in real robot experiments. Crucially, through model and system optimizations, we enable a 14B autoregressive video diffusion model to perform real-time closed-loop control at 7Hz. Finally, we demonstrate two forms of cross-embodiment transfer: video-only demonstrations from other robots or humans yield a relative improvement of over 42% on unseen task performance with just 10-20 minutes of data. More surprisingly, DreamZero enables few-shot embodiment adaptation, transferring to a new embodiment with only 30 minutes of play data while retaining zero-shot generalization.
Living organisms exhibit extraordinary adaptability to unseen environments through their intrinsic physical structures and lifelong feedback-driven learning. Endowing robots with comparable generalization is critical for reliable operation in the real world. While recent approaches attempt to improve generalization by scaling training data, such strategies remain impractical for robotics, where collecting real-world demonstrations at the scale of large language models is prohibitively costly and slow. Contrary to this reliance on massive datasets, we show that robots can generalize effectively under dynamics uncertainties even with limited training data by leveraging a feedback mechanism, namely PhyFilter, that corrects learning outputs with physics-filtered learning residuals. PhyFilter operates as a lightweight, model-agnostic module whose parameters can be automatically optimized through an auto-learning algorithm, eliminating manual tuning and enabling seamless integration with diverse robot policies. We validate PhyFilter across four representative robotic systems, demonstrating that it enables quadruped robots to generalize to unseen terrains, payload variations, and speed ranges; drones to flight under unseen wind disturbances; aerial manipulators to achieve centimeter-level in-air capture despite wind and mass uncertainties; and acceleration differentiators to remain robust with distribution shift. These results show that physics-filtered feedback can serve as a powerful alternative to massive data scaling.
Jindou Jia, Shixuan Han, Meng Wang +8
Nanyang Technological University · Beihang University · National University of Singapore +1