World Models for VLA Models

VLA: Vision-Language-Action

Follow this topic Ask about this topic

Latest papers

47 papers20 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

FloMo: 3D Scene Flow as a World Action Model Intermediate for Learning from Human Video
Nov 3, 2026Jeremy Collins, Namra Patel, Ayush Agarwal +2Robot Learning from Video World Models for VLA Models
Georgia Institute of Technology
Trusting Imagined Futures: Auditing Imagined Rewards for VLA Post-Training
Nov 3, 2026Zicheng Kong, Zhenbo Xu, Haoyan Zhang +7VLMs for Robotics World Models for VLA Models
Beijing University of Posts and Telecommunications, Beijing, P.R. China · ShiFang Technology Inc., Hangzhou, P.R. China · Institute of Automation, Chinese Academy of Sciences, Beijing, P.R. China +1
HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
Oct 29, 2026Rongxing Ding, Hongyu Qu, Kehong Wu +4Vision-Language-Action Models Temporal Modeling for VLA Models
Nanjing University of Science and Technology · National University of Singapore · Shenyang Institute of Automation, Chinese Academy of Sciences +1
PRIME W0: Physical Priors from World Models for Embodied Manipulation
Oct 29, 2026Huabin Peng, Fagang Jin, Yancong Wei +4Robot Learning Robotic Manipulation
Lumos SZ Lab Team · CUHK · THU +2
Utility-WAM: Correcting Future-Induced Action Changes
Oct 29, 2026Feihong Zhang, Xi Lin, Yulong Shi +7World Action Models World Models for VLA Models
Tsinghua University · Johns Hopkins University · Northeastern University +4
ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning
Oct 27, 2026Chaehyun Kim, Chi-Pin Huang, Jisoo Kim +4Reasoning for VLA Models Robot Learning from Video
NVIDIA · KAIST
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Oct 27, 2026Hanqing Wang, Zhaotian Cai, Ruichuan An +13Vision-Language-Action Models VLA Model Training
Peking University
Vision-Language Agents Can Learn in Their Own Latent Worlds
Oct 27, 2026Haoxi Li, Sikai Bai, Jianfei Ma +5RL for VLMs World Models for VLA Models
Department of Computer Science and Engineering, HKUST, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, University of Southern California, Los Angeles, USA
What Does World Modeling Actually Improve in Robot Action Learning?
Oct 25, 2026Hanwen Cui, Yuanyuan Li, Qing Guo +4Robot Learning World Models for VLA Models
MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Gengine +1
$Ψ$ -0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model
Oct 23, 2026Khai Loong Aw, Klemen Kotar, Baihan Zhang +5VLM Reasoning World Models for VLA Models
Stanford University
See What I Need, Dream What I Want: A Navigation Foundation Model on a Self-Constructed Representation Space
Oct 22, 2026Jiawei Chen, Yinxi Wang, Zihan Yang +12Visual Robot Navigation Vision-Language Navigation
Peking University · University College London, University of London · DataCanvas
DRILL: Training World Models to Improve Policies, Not Predict Pixels
Oct 22, 2026Zhaolu Kang, Tailong Luo, Lei Wei +9World Models for VLA Models World Model Learning
Hunyuan, Tencent · Peking University
Agentic World Action Models
Oct 21, 2026Zhuoyang Liu, Dantong Niu, Botong Zhang +15Agentic AI for Robotics World Action Models
UC Berkeley · NVIDIA · University of Pennsylvania
UniWALM: Unified World Action Language Model for Embodied Robot Control
Oct 20, 2026Rahul Sajnani, Diego Romeres, Mingyuan Zhang +9Vision-Language-Action Models World Models for VLA Models
Brown University · Amazon · Northeastern University +2
Learning the 4D World through Point Trajectory Forecasting
Oct 20, 2026Byungwoo Jeon, Sunghwan Hong, Dohyeon Kim +103D Scene Prediction Multi-View Video Generation
KAIST · ETH Zürich · Seoul National University +3
De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
Oct 20, 2026Shaoyi Zheng, Zhongtao Jiang, Yue Peng +4Efficient Inference for World Action Models World Models for VLA Models
ByteDance. · New York University (NYU).
PLaCA: Learning Patch-Level Future Latent Change for Action Generation
Oct 20, 2026Zhan Yi, Yuchen Wang, Chao Li +2Temporal Modeling for VLA Models World Models for VLA Models
School of Robotics and Automation, Nanjing University, Suzhou, Jiangsu, China
Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
Oct 19, 2026Lingsen Zhang, Wei Li, Jie He +5Robotic Manipulation Vision-Language-Action Models
Harbin Institute of Technology, Shenzhen · Baidu, Inc. · Shenzhen Loop Area Institute
Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving
Oct 19, 2026Jie Wang, Guang Li, Zhijian Huang +5VLMs for Autonomous Driving Efficient VLA Models
School of Artificial Intelligence, Tianjin University · School of Artificial Intelligence, Shanxi University · Xiaomi EV
LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
Oct 16, 2026Jiajun Cao, Kai Tang, Bohua Zhang +18Long-Horizon Robotic Manipulation World Models for VLA Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics
What Matters for World Modeling in Vision-Language-Action Models?
Oct 14, 2026Qisen Ma, Yixiang Chen, Yuan Xu +10World Models for VLA Models Temporal Modeling for VLA Models
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Alibaba Group +1
ForkCast: Predictive Recovery for Vision-Language-Action Policies via Counterfactual Futures
Oct 13, 2026Chenru Wen, Huajian Zeng, Haokai Ding +5World Models for VLA Models Vision-Language-Action Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE. · Virginia Tech, USA.
World-Calibrated Proposal-to-Action Flow for Vision-Language-Action Models
Oct 1, 2026Jie He, Wei Li, Junwen Tong +3World Models for VLA Models Flow-Based VLA Models
Harbin Institute of Technology, Shenzhen · State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE, China · Sun Yat-sen University
Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation
Sep 30, 2026Chuyao Fu, Xiaowei Chi, Yuhan Rui +14World Models Action-Conditioned World Models
Southern University of Science and Technology · MUKA Robotics · Hong Kong University of Science and Technology +3
WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
Sep 28, 2026Yuheng Zha, Yilei Wang, Qiyue Gao +5World Model Learning 3D Spatial Reasoning
UC San Diego · Institute of Foundation Models, MBZUAI · Carnegie Mellon University
WorldGuide: Learning Success-Failure Boundaries in Latent World Models for Vision-Language-Action Policies
Sep 28, 2026Lin Liu, Lu Zhang, Ziying Song +6Robotic Manipulation Vision-Language-Action Models
School of Information and Communication Engineering Dalian University of Technology & Beta Infinity · School of Information and Communication, Engineering Dalian University of Technology · Nanyang Technological University +1
SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models
Sep 27, 2026Tianfu Li, Haoxuan Xu, Wenbo Chen +7World Model Learning Visuomotor Policy Learning
The Hong Kong University of Science and Technology (Guangzhou). · The Hong Kong University of Science and Technology. · Nanyang Technological University. +1
Devol-ONE: One Autoregressive Mixture of Transformers to Unify Vision-Language-Action and Latent World Modeling
Sep 26, 2026Hongyi Cai, Yi Herng Ong, Tingshiuan C. Wu +4Mixture-of-Transformers Robotic Manipulation
Devol Robots, San Francisco, CA, United States
Towards VLA-Dreamer: Refining VLA Behavior Using World Models
Sep 25, 2026Parsa Mastouri Kashani, Jan-Gerrit Habekost, Stefan WermterWorld Models for VLA Models Vision-Language-Action Models
Knowledge Technology, Dept. of Informatics, University of Hamburg, Germany
Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies
Sep 21, 2026Trung Dao, Sankalp Yamsani, Jaden Park +2World Models Robotic Manipulation
University of Wisconsin–Madison, USA. · University of Illinois Urbana-Champaign, USA.

Nov 3, 2026J/K move · Enter open · S save

FloMo: 3D Scene Flow as a World Action Model Intermediate for Learning from Human Video

Jeremy Collins, Namra Patel, Ayush Agarwal, Shitij Govil+1

Georgia Institute of Technology

Robot Learning from Video World Models for VLA Models Video Diffusion Models

Open paper AskPDF arXiv

Pretrained video generation models distill internet-scale motion priors into robot policies, but full video is a redundant signal for action prediction. We argue that dense 3D motion is the medium through which video pretraining best transfers to robot actions, and that predicting motion rather than video affords better sample-efficiency for action prediction. FloMo is a 5B video-generation model fine-tuned to predict 3D scene-flow and action tokens. Rendering scene flow as RGB and encoding it into the same latent space as video enables a pretrained video generation backbone to efficiently adapt to predict 3D scene flow. Co-trained on action-free human video and action-labeled robot teleoperation data, our model benefits from human-video pretraining and generalizes to unseen objects, motion primitives, and tasks. Replacing motion prediction with a video prediction objective degrades out-of-distribution generalization, while adding a video prediction target alongside motion lowers policy success rather than improving it. When motion, video, and action are predicted jointly, action tokens attend far more to motion than to video, suggesting that predicting video wastes model capacity. Together, these results indicate that motion effectively mediates the transfer from video pretraining to action. More information can be found at flomo-wam.github.io.