DreamFAR: Real-Time and Long-Horizon Interaction with Diverse Objects
Organizations: Amazon FAR · Cornell University · University of Maryland, College Park · University of Pennsylvania
Abstract
World models have shown great potential in serving as a simulator for robot manipulation. However, existing models are either fit to a single environment or lack ability to simulate long-horizon contact-rich events. We present DreamFAR, a world model whose dynamics model is trained from scratch on a large robot-native teleoperation dataset, with no internet-video pretraining. We present an architecture and learning recipe designed for training on robot-native data. DreamFAR is significantly smaller than the high-performance video generation models, yet shows better understanding of contact and causality, and runs 10 FPS on a commercial GPU. Experimentally, DreamFAR significantly outperforms existing models in predicting the long-horizon future. We further demonstrate various applications where DreamFAR can be used, including policy evaluation and mixed-data co-training. Project page: https://www.yimingdou.com/DreamFAR/
Figures & tables
| Metric | No memory | Fixed stride | Contact (ours) |
|---|---|---|---|
| PSNR | 18.655 | 18.988 | 19.251 |
| LPIPS | 0.348 | 0.334 | 0.326 |
| FID | 37.156 | 36.438 | 34.811 |
| DINO | 0.749 | 0.767 | 0.777 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| View | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Top | 27.95 | 0.722 | 0.085 |
| Left wrist | 31.26 | 0.664 | 0.100 |
| Right wrist | 31.55 | 0.670 | 0.100 |
World Models for Robotics
World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.
See all 423 papers