Organizations: University of Tsukuba, Tsukuba, Japan. · CNRS-AIST JRL (Joint Robotics Laboratory), IRL3218, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan.
Data-driven learning for general-purpose robotic manipulation presents a fundamental trade-off between data-collection cost and adaptability to real-world environments. Imitation learning (IL) requires costly human demonstrations, whereas simulation-based learning is vulnerable to sim-to-real gaps in dynamics and sensor observations. We propose ReDOA (Real-World Distillation via Observation Alignment), a framework that decouples control learning from perception learning to learn image-based policies in the real world. A state-based teacher policy trained in simulation is executed on a real robot, and its control knowledge is distilled into an image-based student policy using the collected trajectories. Observation Alignment resolves observation mismatches between data collection and deployment. This framework collects successful, efficiently executed trajectories paired with real-world visual observations, without requiring demonstrations from human experts. In real-robot experiments on three non-prehensile manipulation tasks, including bimanual manipulation, our method achieves better overall performance when jointly considering task success and execution efficiency than both IL based on human demonstrations and policy distillation in simulation. https://isri-aist.github.io/redoa-project-page/
Figures & tables
Fig. 1: Conceptual overview of ReDOA (Real-World Distillation via Observation Alignment).
Fig. 2: Overview of the ReDOA (Real-World Distillation via Observation Alignment) learning pipeline.
Fig. 3: Workflow for tracker artifact removal from raw demonstration images. We used planar fiducial markers as trackers for this whole study.
Fig. 4: Real-world manipulation tasks.
Fig. 5: Comparison of the training pipelines of the proposed method and the three baseline methods. The key characteristics of each method are also summarized on the right. * Real-world data are used; however, the training images undergo tracker artifact removal.
Box Rotation
Drawer Opening
Cylinder Rolling
Method
SR (%) ↑
ATR (s) ↓
TPR ↑
SR (%) ↑
ATR (s) ↓
TPR ↑
SR (%) ↑
ATR (s) ↓
TPR ↑
Distill. in Sim
14.0
7.56
-0.010
-
-
-
-
-
-
IL with Human Demo
76.0
20.26
0.037
78.0
5.49
0.119
72.0
9.84
0.056
ReDOA (Ours)
70.0
8.81
0.075
64.0
2.75
0.187
68.0
8.03
0.068
TABLE I: Performance evaluation results. Results are averaged over 5 random seeds (10 trials per seed, 50 trials in total).
Image Compositing
Deploy Env
SR (%) ↑
ATR (s) ↓
TPR ↑
Proposed
Marker-Free
70.0
8.62
0.079
w/o Image Compositing
Marker-Free
53.3
8.94
0.047
w/o Image Compositing
Marker-Based
73.3
7.95
0.086
TABLE II: Ablation study evaluating the effect of marker removal. Results are averaged over 3 random seeds (10 trials per seed, 30 trials in total).
Fig. 6: Training curves of state-based SAC and image-based SAC in simulation. Curves are smoothed for visualization. State-based SAC learns substantially faster and converges to a higher return than image-based SAC.
Visuomotor policy learning trains robots to turn camera observations, often with language and proprioception, into actions for manipulation and navigation.
When using reinforcement learning (RL) for contact-rich robotic manipulation, vision can provide task-relevant information that complements robot proprioception. However, vision-enabled policies tend to overfit to the visual conditions seen during training, limiting their robustness and transferability. We present a human-in-the-loop RL framework that employs teacher-student distillation to achieve robust performance across multiple task variants, trained entirely in the real world without requiring domain randomization or data augmentation. A vision-enabled teacher distills its knowledge into a vision-free student that relies solely on pose, twist, and wrench sensing, combining fast training with strong task generalization. On the real-world NIST assembly benchmark board, our approach achieves 95% overall success after approximately 50 minutes of training on 3 representative tasks, including robust generalization to 8 unseen task variants. Fine-tuning with distillation achieves full success on the most challenging task. We demonstrate that the resulting policies outperform baselines in both robustness and adaptability. Page: https://tuwien-asl.github.io/VE2VF/.
Victor Kowalski, Chengxi Li, Dongheui Lee
Autonomous Systems, Technische Universitaet Wien (TU Wien), Vienna, Austria · Institute of Robotics and Mechatronics (DLR), German Aerospace Center, Wessling, Germany
Collecting real-world robot data for dexterous manipulation is costly and time-consuming. While high-fidelity physics simulators enable scalable data synthesis and policy learning, constructing deployment-ready digital twins manually remains labor-intensive, and residual visual, geometric, and dynamics gaps hinder reliable sim-to-real transfer. We present DEXTERA, an automated real-to-sim-to-real framework that transforms a single RGB image into deployable policies for dexterous manipulation across four unified stages: (1) single-image scene factorization into a static Gaussian background and interactive rigid or articulated assets with VLM-inferred physical parameters; (2) metric scene global alignment, object canonicalization, and morphology-balanced robot calibration; (3) scalable simulator task primitive construction, VR teleoperation, and object-centric trajectory synthesis; and (4) a shared multimodal policy interface supporting both imitation learning and reinforcement learning. We evaluate DEXTERA across 13 task-embodiment pairs, 2 dexterous robot platforms, and 6 policy architectures. Experimental results demonstrate that DEXTERA achieves superior visual fidelity and 3D geometric reconstruction compared to generative baselines, while cross-domain trajectory replays validate strong physical interaction consistency. Furthermore, simulation-only trained policies enable viable zero-shot real-robot deployment, while simulation-real co-training substantially improves mean physical policy success from 29.2% to 61.9% across diverse policy architectures. Project website: https://dextera-project.github.io/
Jin Wu, Lianjie Yuan, Zeyan Sun +4
The University of Texas at Austin · University of Florida · BrainCo
Robots trained on real world data tend to be imprecise, slow, and brittle to perturbations. Improving these policies with reinforcement learning (RL) is an appealing alternative, but this process often requires expensive training in the real world. Performing policy improvement in simulation instead provides a far cheaper alternative, but unconstrained RL in simulation can exploit contact and dynamics mismatches, resulting in unsafe behaviors that do not transfer to hardware. Common forms of regularization can furthermore limit improvement by overconstraining to an imperfect behavior prior. In this work, we propose Support-Constrained Off-Domain REinforcement (SCORE), a real-to-sim-to-real framework that constrains RL in simulation to the support of a generative policy pretrained on real data. We instantiate this constraint through flow steering, restricting SCORE to actions the base policy can already produce, which ensures transferable behaviors while maximizing policy improvement. Improving a policy with SCORE requires minimal effort: it learns from sparse rewards, avoids distillation, and leaves the base policy untouched. Across eight real-world dexterous multi-fingered robotic manipulation tasks, SCORE improves average success rate from 37.8% to 89.9%, compared to 59.5% for the best baseline, and reaches success in 36.8% fewer steps than the base policy. Ultimately, through extensive experiments and ablations, we show that simulation can substantially improve real-world manipulation policies when policy optimization is appropriately constrained, introducing a new paradigm for real-to-sim-to-real policy improvement. Videos and code are available at https://weirdlabuw.github.io/score/.