Object-Centric World Models

Follow this topic Ask about this topic

Object-centric world models represent a scene as entities—often learned slots, particles, or graph nodes—and predict how their states and relationships evolve over time, sometimes conditioned on actions. Approaches range from learned visual dynamics and causal or relational graphs to physics-inspired models and executable symbolic rules, with some systems coupling predictions to robot policies. They are evaluated on settings such as video forecasting, visual reasoning, autonomous driving, and manipulation benchmarks including CLEVRER, PushT, and LIBERO. A central challenge is making object identities and dynamics reliable under occlusion, distribution shift, and long rollouts, while ensuring accurate predictions actually improve planning and control.

Latest papers

26 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation
Oct 16, 2026Yueyang Weng, Xiaopeng Zhang, Yuxi Ou +3World Model Learning Object-Centric World Models
Harbin Institute of Technology, Shenzhen
Mallea: Building, Steering, and Memorizing Worlds with Coding Agent
Oct 15, 2026Xuanyi Liu, Ruiqi Li, Siwei MaCode World Models World Models
Grid or Nodes? Probing Object Memory in World Models Under Partial Observability
Oct 13, 2026Justin McMillen, Yasin YilmazWorld Model Learning Object-Centric World Models
University of South Florida Tampa, FL, USA
World-as-Graph: Relational World Modeling Through Latent Space Graphs
Sep 30, 2026Yaqi Yang, Shuo Huang, Yujin Huang +3Graph Structure Learning World Model Learning
School of Computing Technologies, RMIT University, Australia · Faculty of Information Technology, Monash University, Australia · KStelrix Star Dynamics Lab, KStelrix, China +1
From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations
Sep 28, 2026Bangjun Wang, Longyan Wu, Yukun Wei +8Object-Centric World Models Robotic Tool Use
The University of Hong Kong · Shanghai Innovation Institute · Xiaomi EV
Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation
Sep 2, 2026Param Thakkar, Parsika Paresh Shah, Manisha Sushant GoteWorld Model Learning World Models for Robotics
Veermata Jijabai Technological Institute · Arizona State University · ZuiGO Private Limited
AcrossVAM1.0: Particle World Modeling for Text-Assisted Robot Video Prediction
Aug 28, 2026Yafei Zhang, Nan WuVideo Prediction World Models
Across Physical AI, Beijing, China
4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting
Aug 26, 2026Yueen Ma, Zenglin Xu, Irwin KingWorld Action Models Object-Centric World Models
The Chinese University of Hong Kong · Shanghai Academy of AI for Science · Fudan University
MOSH-WM: Mask-Grounded Soft-Hamiltonian Dynamics for Object-Centric World Models
Aug 24, 2026Zhekai Wang, Haoxiang Huang, Xiang Liu +6Object-Centric World Models Video World Models
Beijing Institute of Technology · University of Science and Technology of China · Tsinghua University +1
Beyond Instance Slots: Semantically Rich World Models for Physical Interaction Planning
Aug 23, 2026Juntao Cheng, Jingkai Wang, Yijun Shen +2World Model-Based Planning Action-Conditioned World Models
Beijing Academy of Artificial Intelligence (BAAI) · Shanghai Jiao Tong University
Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models
Aug 12, 2026Shukrullo Nazirjonov, Sai Prasanna, Anna Manasyan +1World Models World Model-Based Planning
University of Tuebingen · Max Planck Institute for Intelligent Systems
RiskWorld: Object-Centric Latent World Modeling for Autonomous Driving Risk Identification
Aug 12, 2026Jingzheng Li, Yufei Ge, Qianren Mao +5Autonomous Driving World Models for Autonomous Driving
Zhongguancun Laboratory, Beijing, China · Tianjin University, Beijing, China · Hong Kong Polytechnic University, Hong Kong, China +2
Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models
Jul 6, 2026Hongyu Li, Wanjia Fu, Xiaoyan Cong +11World Models Deformable Object Manipulation
Brown University, Providence, RI · Columbia University, New York, NY · Massachusetts Institute of Technology, Cambridge, MA
Object-Centric Environment Modeling for Agentic Tasks
Jul 3, 2026Yiyang Li, Tianyi Ma, Zehong Wang +2Continual Learning for LLM Agents World Models
University of Notre Dame · †Corresponding Author
OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration for ARC-AGI-3
Jul 1, 2026David Courtis, Wenhao Li, Scott SannerWorld Model Learning World Models
Department of Computer Science, University of Toronto
FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Drift
Jun 11, 2026Yitao Jiang, Luyang Zhao, Muhao Chen +1World Model Learning World Models for Robotics
Department of Computer Science Dartmouth College, Hanover, NH · Department of Electrical and Computer Engineering Clemson University, Clemson, SC · Department of Mechanical and Aerospace Engineering University of Houston, Houston, TX
MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models
Jun 11, 2026Hanyang Yu, Haitao Lin, Jingbo Zhang +4World Model Learning Language-Conditioned Robot Manipulation
1The Hong Kong University of Science and Technology · ‡Work done during an internship at Tencent Robotics X · 2Tencent Robotics X +1
Slots, Transitions, Loops: Learning Composable World Models for ARC
Jun 10, 2026Gege Gao, Bernhard Schölkopf, Andreas GeigerWorld Model Learning Visual Reasoning
University of Tübingen · ETH Zürich
Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Tasks
Jun 7, 2026Raktim Gautam Goswami, Prashanth Krishnamurthy, Yann LeCun +1Long-Horizon Robotic Manipulation World Models for Robotics
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · AMI Labs
Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects
Jun 1, 2026Jens U. Kreber, Lukas Mack, Joerg Stueckler3D Gaussian Splatting Dynamical Systems
Intelligent Perception in Technical Systems Group, University of Augsburg, Augsburg, Germany
WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
May 24, 2026Bohai Gu, Taiyi Wu, Yueyang Yuan +9Camera-Controlled Video Generation Object-Centric World Models
The Hong Kong University of Science and Technology · AI Technology Center, Tencent Video, Tencent · Wuhan University +1
OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
May 7, 2026Yushan Liu, Peibo Sun, Shoujie Li +7World Models for Robotics Action-Conditioned World Models
Tsinghua University · Shanghai Jiao Tong University · Nanyang Technological University
HCLSM: Hierarchical Causal Latent State Machines for Object-Centric World Modeling
Mar 31, 2026Jaber Jaber, Osama JaberHierarchical World Models World Models
RightNow AI
Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling
Mar 4, 2026Tal Daniel, Carl Qi, Dan Haramati +5World Model Learning Object-Centric World Models
Carnegie Mellon University · UT Austin · Brown University +2
OOWM: Structuring Embodied Reasoning and Planning via Object-Oriented Programmatic World Modeling
Feb 25, 2026Hongyu Chen, Liang Lin, Guangrun WangLarge Language Model-Based Robot Planning World Models
Sun Yat-sen University · Guangdong Key Laboratory of Big Data Analysis and Processing · X-Era AI Lab
Soft Geometric Inductive Bias for Object Centric Dynamics
Dec 17, 2025Hampus Linander, Conor Heins, Alexander Tschantz +2World Model Learning World Models
VERSES AI · University of Sussex, Department of Informatics

Oct 16, 2026J/K move · Enter open · S save

ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation

Yueyang Weng, Xiaopeng Zhang, Yuxi Ou, Yongjin Mu+2

Harbin Institute of Technology, Shenzhen

World Model Learning Object-Centric World Models Robot Dynamics Learning Robotic Manipulation

Open paper AskPDF arXiv

World models provide robots with a predictive mechanism for reasoning about how actions change object states in manipulation tasks. However, existing image-based and dense 3D world models do not explicitly impose object-level rigid motion, leaving rigid-object geometry preservation to be learned implicitly and making multi-step rollouts susceptible to shape drift or object-level motion inconsistency. To address this, we propose ORD-WM, a 3D world model that learns object-centric rigid dynamics for robotic manipulation. ORD-WM learns a vector-neuron variational autoencoder supervised by signed distance fields, which encodes each object point cloud as a geometry-aware latent representation composed of 3D vector features, and uses an action-conditioned dynamics model to predict object-wise SE(3) motion increments. Rather than generating unconstrained next latent states, ORD-WM analytically applies predicted object-wise rigid transformations to the current latent states, separating persistent object geometry from pose changes. Experiments show that ORD-WM enables stable, geometry-preserving multi-step prediction and supports efficient real-time replanning for closed-loop model predictive control in simulated and real-world manipulation scenarios.