Interactive World Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 30% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 86

All topics

Light Interaction: Training-Free Inference Acceleration for Interactive Video World Models
May 29, 2026Jiacheng Lu, Haoyi Zhu, Sipei Yi +3Real-Time Interactive Video Generation Interactive World Models
1Zhejiang University · 2NVIDIA
World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications
May 28, 2026Arif Hassan Zidan, Yi Pan, Hanqi Jiang +23World Model Learning World Model-Based Planning
School of Computer and Cyber Sciences, Augusta University, Augusta, GA, USA · School of Computing, University of Georgia, Athens, GA, USA · Tandon School of Engineering, New York University, Brooklyn, NY, USA +9
minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models
May 28, 2026Min Zhao, Hongzhou Zhu, Bokai Yan +9Camera-Controlled Video Generation Video Diffusion Models
1ShengShu 2THU · 1ShengShu 3RUC · 1ShengShu +4
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players
May 27, 2026Fangfu Liu, Kai He, Tianchang Shen +7Interactive World Models Video World Models
NVIDIA · Tsinghua University · University of Toronto +1
LEIA: Learned Environment for Interactive Architected Materials
May 27, 2026Haiqian Yang, Yuan Cao, Markus J. BuehlerNeural Surrogate Modeling World Model Learning
Unreasonable Labs · Mountain View, CA 94043, USA
WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation
May 25, 2026Kaining Ying, Hengrui Hu, Siyu Ren +6World Model Evaluation Interactive World Models
Fudan University · Meituan Longcat Team
WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
May 24, 2026Bohai Gu, Taiyi Wu, Yueyang Yuan +9Camera-Controlled Video Generation Object-Centric World Models
The Hong Kong University of Science and Technology · AI Technology Center, Tencent Video, Tencent · Wuhan University +1
SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models
May 22, 2026Zizhao Tong, Yeying Jin, Hongfeng Lai +11Action-Conditioned Video Generation Video Diffusion Models
UCAS-Terminus AI Lab, University of Chinese Academy of Sciences · Tencent · National University of Singapore +6
Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models
May 18, 2026Shangwen Zhu, Qianyu Peng, Zhao Pu +12Action-Conditioned Video Generation Long-Horizon Video Generation
1SJTU · 8HKU · 3Matrix Team +6
WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
May 18, 2026Yu Shang, Yinzhou Tang, Yiding Ma +22World Models for Robotics World Models
Tsinghua University · Shanghai Jiao Tong University · Zhejiang University +7
Baba in Wonderland: Online Self-Supervised Dynamics Discovery for Executable World Models
May 16, 2026SeungWon Seo, DongHeun Han, SeongRae Noh +1Online System Identification World Model Learning
Korea University · Kyung Hee University
WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes
May 15, 2026Jichen Hu, Jiawei Guo, Jiazhong Cen +33D Scene Editing Interactive World Models
1Shanghai Jiao Tong University · 2Huawei Inc.
ReactiveGWM: Flexible Control and NPC Reactivity in Game World Models
May 14, 2026Zeqing Wang, Danze Chen, Zhaohu Xing +4World Models Multi-Agent Control
Tencent · National University of Singapore · The Hong Kong University of Science and Technology (Guangzhou) +3
The DAWN of World-Action Interactive Models
May 12, 2026Hongbo Lu, Liang Yao, Chenghao He +6Diffusion-Based Planning World Model-Based Planning
1COWARobot Co. Ltd · 2Shanghai Jiao Tong University · 3Hohai University
How Mobile World Model Guides GUI Agents?
May 11, 2026Weikai Xu, Kun Huang, Yunren Feng +10World Model Learning GUI Agents
1Nanyang Technological University · 2MiLM Plus, Xiaomi Inc. · 3Independent Researchers +3
DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos
May 10, 2026Can Li, Zhoujian Li, Ren Li +4World Model Learning World Models
Nankai University · Rightly Robotics, A4X · Zhejiang University +2
AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites
May 7, 2026Qinshi Zhang, Weipeng Deng, Zhihan Jiang +4World Model Learning Interactive World Models
University of California, San Diego · University of Hong Kong · Columbia University +2
iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework
May 5, 2026Jianjie Fang, Yingshan Lei, Qin Wan +8World Model Evaluation Interactive World Models
Tsinghua University, China · Northeastern University, China · South China University of Technology, China
Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond
Apr 24, 2026Meng Chu, Xuan Billy Zhang, Kevin Qinghong Lin +47World Model Learning Action-Conditioned World Models
WorldMark: A Unified Benchmark Suite for Interactive Video World Models
Apr 23, 2026Xiaojie Xu, Zhengyuan Lin, Kang He +5Action-Conditioned World Models World Model Evaluation
Alaya Studio, Shanda AI Research Tokyo · The University of Tokyo · Shanghai Innovation Institute
Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory
Apr 10, 2026Zile Wang, Zexiang Liu, Jiaxing Li +20Real-Time Interactive Video Generation Long-Horizon Video Generation
Skywork AI
LivingWorld: Interactive 4D World Generation with Environmental Dynamics
Apr 2, 2026Hyeongju Mun, In-Hwan Jin, Sohyeong Kim +13D Scene Generation Interactive World Models
Department of Electrical and Electronic Engineering, Pusan National University
SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios
Nov 20, 2025Juntao Cheng, Wanyue Zhang, Zhiwei Yu +7Long-Horizon Agent Evaluation Egocentric Video Understanding
Beijing Academy of Artificial Intelligence (BAAI) · Shanghai Jiao Tong University · School of Artificial Intelligence, University of Chinese Academy of Sciences +3
RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulation
Oct 10, 2025Chuanrui Zhang, Zhengxian Wu, Guanxing Lu +2World Models for Robotics World Models
Nanyang Technological University, Singapore · Tsinghua University, Beijing, China
Matrix-game 2.0: An open-source, real-time, and streaming interactive world model
Aug 18, 2025Xianglong He, Chunli Peng, Zexiang Liu +16Action-Conditioned Video Generation Real-Time Interactive Video Generation
Skywork AI
GameWAM: A World Action Model for Video Games
Date pendingYuncheng Guo, Zhanqiu Zhang, Yiwen Guo +1Action-Conditioned Video Generation Game-Playing Agents
Fudan University · LIGHTSPEED · Independent Researcher +1

May 29, 2026, cs.CVJ/K move · Enter open · S save

Light Interaction: Training-Free Inference Acceleration for Interactive Video World Models

Jiacheng Lu, Haoyi Zhu, Sipei Yi, Enze Xie+2

1Zhejiang University · 2NVIDIA

Real-Time Interactive Video Generation Interactive World Models Diffusion Model Inference Acceleration Block-Sparse Attention Video World Models

Open paper AskPDF arXiv

Interactive video world models generate video chunk by chunk in response to user-controlled camera movements, enabling applications such as real-time game simulation, virtual scene navigation, and embodied AI training. However, scaling to long interactive trajectories is prohibitively expensive due to growing context memory, quadratic attention complexity, and repeated denoising steps. We present Light Interaction, a training-free inference acceleration framework for interactive video world models. Our key insight is that interaction naturally enables trajectory-dependent adaptive computation: retrieved spatial memory can be discarded during novel exploration, temporal context can be adjusted according to local latent dynamics, and early-step model outputs can be reused when the camera revisits familiar regions. Based on this insight, Light Interaction combines adaptive context management, denoising cache acceleration, and hardware-software co-designed 3D block sparse attention with fused Triton kernels. Evaluated on HY-WorldPlay and Matrix-Game-3.0, Light Interaction achieves up to 2.59x speedup without model retraining while maintaining competitive visual quality.

Momentum

Related topics

Latest papers 86