Action-Conditioned Video Generation

Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 72

All topics
CardsList

  1. CtrlCache: Accelerating Interactive Video World Models with Control-Aware Caching

    Oct 6, 2026Shangye Song, Dong Gong, Hong Jia +2Action-Conditioned Video GenerationReal-Time Interactive Video Generation

    University of Auckland · UNSW Sydney

  2. SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models

    Oct 5, 2026Xiaodong Wang, Tianle Li, Chuanxin Song +4Action-Conditioned Video GenerationWorld Models for Robotics

    Peking University

  3. KineWorld: Action-Induced Transport Fields for Embodied World Modeling

    Oct 5, 2026Ziying Song, Yuchen Liu, Zhuoran Xu +6Action-Conditioned Video GenerationVideo Diffusion Models

    Nanyang Technological University · North University of China · Psibot +4

  4. FutureWorlds: Learning Robotic World Models from Alternative Futures

    Oct 1, 2026Hao Wu, Shengju Qian, Weiyan Wang +5Action-Conditioned Video GenerationWorld Model Learning

    HKUST (GZ) · CUHK · Tencent +3

  5. Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling

    Sep 30, 2026Xiangyu Zhu, Jin Xu, Yue Guo +6Action-Conditioned Video GenerationCross-Embodiment Robot Learning

    Beijing Humanoid Robot Innovation Center · Beijing Institute of Technology · Harbin Institute of Technology, Shenzhen +2

  6. BadAction: Backdoor Attacks on Interactive Video Generation via Action-Guided Triggers

    Sep 30, 2026Zhihang Wu, Zhongqi Wang, Jie Zhang +3Action-Conditioned Video GenerationBackdoor Attacks

    Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China · University of Chinese Academy of Sciences, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese of Sciences, Beijing, China

  7. WorldLine: Action-Driven Visual Simulation for Robotic Manipulation

    Sep 29, 2026Shenghe Zheng, Wenbo Li, Jiyao Zhang +4Action-Conditioned Video GenerationWorld Models for Robotics

    The Hong Kong University of Science and Technology · Joy Future Academy · Peking University +1

  8. FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning

    Sep 28, 2026Ziyao Huang, Zhengkun Rong, Shiyang Qin +5Action-Conditioned Video GenerationReal-Time Interactive Video Generation

    Bytedance Intelligent Creation

  9. ReSync: Re-Aligning the Two Clocks of Asynchronous World-Action Models

    Sep 27, 2026Xi Lin, Feihong Zhang, Yulong Shi +6Action-Conditioned Video GenerationVideo Diffusion Models

    Johns Hopkins University · Tsinghua University · Yinwang Intelligent Technology Co., Ltd. +1

  10. Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

    Sep 15, 2026Mingyang Chen, Shengdong Chen, Xiaoxiao Fu +13Action-Conditioned Video GenerationReal-Time Interactive Video Generation

    Zing Team, SeedLeap.ai

  11. Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model

    Sep 14, 2026Hoeun Lee, Jaeik Kim, Jusang Oh +4Robotic ControlAction-Conditioned Video Generation

    AIDAS Lab Seoul National University

  12. ActionSplice: In-Flight Action Editing for Interactive World Models

    Sep 8, 2026Pardis Taghavi, Tingyu Guo, Jonas Lossner +2Action-Conditioned Video GenerationEfficient Inference for World Action Models

    Texas A&M University

  13. Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation

    Sep 3, 2026Haoyu Wang, Songchun Zhang, Haoran Li +3Action-Conditioned Video GenerationMulti-View Video Generation

    1Joy Future Academy, JD · 2Tsinghua University · 3The Hong Kong University of Science and Technology

  14. H3-World: Turning Language Understanding into World Control

    Sep 1, 2026Danze Chen, Zeqing Wang, Ziyue Lin +2Action-Conditioned Video GenerationCamera-Controlled Video Generation

    Tencent · National University of Singapore · The Hong Kong Polytechnic University

  15. CAER: Causal Action Effect Reweighting for World Model Training

    Aug 31, 2026Jianjie Fang, Xvyuan Liu, Ziyou Wang +9Action-Conditioned Video GenerationWorld Model Learning

    1Tsinghua University · University of Science and Technology of China · 2Manifold AI

  16. DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation

    Aug 13, 2026DreamX Team, Rui Chen, Xiangxiang Chu +7Action-Conditioned Video GenerationRobotic Manipulation

  17. Twin Rollouts: Noise-Coupled Counterfactual Branching in Interactive Video World Models

    Aug 10, 2026Yu Ma, Hongli Shi, Xinran XuAction-Conditioned Video GenerationCounterfactual Evaluation

  18. SelfWAM: A Self-Grounded Unified World Action Model for Fast Robot Control

    Aug 1, 2026Bikang Pan, Fan Liu, Haotao Lu +2Action-Conditioned Video GenerationWorld Models for Robotics

    1ShanghaiTech University · 2InstAdapt

  19. PhiZero: A World Model Built Around Physical Language

    Jul 30, 2026Shuyao Shang, Yuqi Wang, Ruopeng Gao +4Action-Conditioned Video GenerationPhysical Consistency in Video Generation

    NLPR, Institute of Automation, Chinese Academy of Sciences (CASIA) · Independent Researcher

  20. EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE

    Jul 30, 2026Zexuan Yan, Yuzhou Wu, Yue Ma +9Action-Conditioned Video GenerationMemory-Augmented Video Generation

    1Shanghai Jiao Tong University · 2Alibaba Group · 3Tianji KernalMind Co., Ltd. +4

  21. ContactFlow: A video action conditioning that transfers across embodiments

    Jul 29, 2026Sami Azirar, Enrico Pallotta, Jan Nogga +3Action-Conditioned Video GenerationContact-Rich Robotic Manipulation

    University of Bonn, Lamarr Institute

  22. Masked Visual Actions for Unified World Modeling

    Jul 21, 2026Hadi Alzayer, Wenlong Huang, Haonan Chen +8Action-Conditioned Video GenerationRobotic Manipulation

    Stanford University · University of Maryland, College Park · Harvard University

  23. ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU

    Jul 21, 2026Fan Jiang, Zhaoxu Sun, Mengchao Wang +38Action-Conditioned Video GenerationLong-Horizon Video Generation

  24. Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation

    Jul 20, 2026Zhenjie Liu, Binyan Chen, Hao Chen +2Action-Conditioned Video GenerationText-to-Video Generation

    University of Science and Technology of China Hefei, China

  25. From Pixels to States: Rethinking Interactive World Models as Game Engines

    Jul 15, 2026Zhen Li, Zian Meng, Shuwei Shi +4Action-Conditioned Video GenerationAction-Conditioned World Models

    1Alaya Lab

  26. FlowWAM: Optical Flow as a Unified Action Representation for World Action Models

    Jul 14, 2026Yixiang Chen, Peiyan Li, Yuan Xu +13Action-Conditioned Video GenerationVideo Diffusion Models

    New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · MBZUAI +2

  27. WanToFight: Real-Time Generative Game Engine for Multi-Player Combat Interaction

    Jul 14, 2026Li Hu, Guangyuan Wang, Peng Zhang +1Action-Conditioned Video GenerationVideo Diffusion Models

    Tongyi Lab, Alibaba

  28. RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation

    Jul 7, 2026Haoyu Zhao, Xingyue Zhao, Hangyu Li +6Action-Conditioned Video GenerationWorld Models for Robotics

    DAMO Academy, Alibaba Group · Hong Kong Embodied AI Lab · CUHK +3

  29. UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation

    Jul 6, 2026Mengmeng Liu, Diankun Zhang, Jiuming Liu +7Action-Conditioned Video GenerationVideo Diffusion Models

    University of Twente · Xiaomi EV · University of Cambridge +1

  30. MemLearner: Learning to Query Context memory for Video World Models

    Jun 30, 2026Jiwen Yu, Jianxiong Gao, Jianhong Bai +7Action-Conditioned Video GenerationLong-Horizon Video Generation

    The University of Hong Kong · ‡Work done during an internship at Kling Team, Kuaishou Technology. · Fudan University +2

  31. Learning Transferable Dynamics Priors from Action to World Modeling

    Jun 28, 2026Ze Huang, Jiahui Zhang, Hairuo Liu +3Action-Conditioned Video GenerationRobot Policy Learning

    School of Data Science, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China +1

  32. ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models

    Jun 27, 2026Longyu Chen, Heng Li, Wei Yang +2Action-Conditioned Video GenerationWorld Models for Robotics

    Huawei Technologies Co., Ltd. · Huazhong University of Science and Technology

  33. Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model

    Jun 25, 2026Zizhao Yuan, Zhengtu Liang, Taowen Wang +7Action-Conditioned Video GenerationDexterous Manipulation

    The Hong Kong University of Science and Technology (Guangzhou) · Shenzhen University · Beijing University of Technology +1

  34. Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models

    Jun 24, 2026Kaiwen Zheng, Guande He, Min Zhao +7Action-Conditioned Video GenerationAutoregressive Diffusion

    Tsinghua University · UT Austin · NVIDIA

  35. IOI: Decoupling Kinematics and Physics for Interactive World Models

    Jun 22, 2026Chengyu Bai, Peidong Jia, Tiecheng Guo +11Action-Conditioned Video GenerationWorld Models for Robotics

    1Beijing Innovation Center of Humanoid Robotics · 2Peking University

  36. World Action Models: A Survey

    Jun 18, 2026Qiuhong Shen, Shihua Zhang, Yue Liao +5Action-Conditioned Video GenerationAction-Conditioned World Models

    National University of Singapore

  37. SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation

    Jun 17, 2026Wei-Cheng Tseng, Gashon Hussein, Yuzhu Dong +9Action-Conditioned Video GenerationPhysical Consistency in Video Generation

    University of Toronto · Vector Institute · NVIDIA +3

  38. Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation

    Jun 15, 2026Jie Zhang, Xiaoyue Chen, Anzhe Chen +36Action-Conditioned Video GenerationVideo Diffusion Models

    Qwen Team

  39. CausalDrive: Real-time Causal World Models for Autonomous Driving

    Jun 13, 2026Tianyi Yan, Huan Zheng, Dubing Chen +10Action-Conditioned Video GenerationAutonomous Driving

    SKL-IOTSC, CIS, University of Macau · Xiaomi EV · CASIA

  40. Diffusion Transformer World-Action Model for AV Scene Prediction

    Jun 11, 2026Ruslan Sharifullin, Benjamin Jiang, Kai Xi ChewAction-Conditioned Video GenerationDiffusion Transformer

    Stanford University

  41. Making Foresight Actionable: Repurposing Representation Alignment in World Action Models

    Jun 10, 2026Lu Qiu, Yizhuo Li, Yi Chen +3Action-Conditioned Video GenerationRepresentation Learning

    The University of Hong Kong · XPENG Robotics

  42. BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression

    Jun 8, 2026Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang +5Action-Conditioned Video GenerationCamera-Controlled Video Generation

    LynnReal AI · Shanghai Innovation Institute · Shanghai Jiao Tong University +1

  43. Echo-Memory: A Controlled Study of Memory in Action World Models

    Jun 8, 2026Wayne King, Zeyue Xue, Yuxuan Bian +13Action-Conditioned Video GenerationVideo Diffusion Models

    The University of Hong Kong · Joy Future Academy, JD · The Chinese University of Hong Kong +5

  44. DisCo: World Models with Discrete Camera Motion Control

    Jun 6, 2026Hongrui Huang, Junke Wang, Quanhao Li +2Action-Conditioned Video GenerationCamera-Controlled Video Generation

    1Fudan University

  45. OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics

    Jun 3, 2026Zhuoyuan Wu, Jun GaoAction-Conditioned Video GenerationAction-Conditioned World Models

    Peking University · University of Michigan · NVIDIA

  46. Unified Video-Action Joint Denoising for Dexterous Action and Data Generation

    Jun 2, 2026Dingrui Wang, YuAn Wang, Jinkun Liu +4Action-Conditioned Video GenerationVideo Diffusion Models

    1Technical University of Munich · 2ByteDance · 3Tsinghua University

  47. NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

    Jun 2, 2026Aarti Basant, Amlan Kar, Despoina Paschalidou +30Action-Conditioned Video GenerationVideo Diffusion Models

    NVIDIA

  48. Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenization

    Jun 1, 2026Jingyun Liang, Min Wei, Shikai Li +5Action-Conditioned Video GenerationVideo Diffusion Models

    DAMO Academy, Alibaba Group · Hupan Lab · INSAIT +1

  49. PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motion

    May 31, 2026Heyuan Gao, Bangxun Tang, Yiren Song +4Action-Conditioned Video GenerationVideo Diffusion Models

    Utopai Studios · Nanyang Technological University · University of California, Irvine +1

  50. Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends

    May 31, 2026Jiuming Liu, Chaojun Ni, Mengmeng Liu +7Action-Conditioned Video GenerationLong-Horizon Video Generation

    Department of Engineering, University of Cambridge, U.K. · Peking University, China · University of Twente, Netherlands +4

  51. τ0τ_0-WM: A Unified Video-Action World Model for Robotic Manipulation

    May 31, 2026Pengfei Zhou, Shengcong Chen, Di Chen +17Action-Conditioned Video GenerationLong-Horizon Robotic Manipulation

    1Shanghai Innovation Institute · 2AGIBOT Finch

  52. Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation

    May 29, 2026Hanlin Chen, Jiaxin Wei, Xibin Song +5Action-Conditioned Video GenerationLong-Horizon Video Generation

    School of Computing, National University of Singapore · Technische Universität München · Vertex Lab +1

  53. GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation

    May 26, 2026Boxiang Qiu, Liliang Chen, Yue Liao +12Action-Conditioned Video GenerationWorld Models for Robotics

    AgiBot · BUAA · LV-NUS Lab +1

  54. SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models

    May 22, 2026Zizhao Tong, Yeying Jin, Hongfeng Lai +11Action-Conditioned Video GenerationVideo Diffusion Models

    UCAS-Terminus AI Lab, University of Chinese Academy of Sciences · Tencent · National University of Singapore +6

  55. Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls

    May 19, 2026Abdul Mohaimen Al Radi, Kunyang Li, Yuzhang Shang +2Action-Conditioned Video GenerationAerial Robotics

    Institute of Artificial Intelligence, University of Central Florida

  56. Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models

    May 18, 2026Shangwen Zhu, Qianyu Peng, Zhao Pu +12Action-Conditioned Video GenerationLong-Horizon Video Generation

    1SJTU · 8HKU · 3Matrix Team +6

  57. Nano World Models: A Minimalist Implementation of Future Video Prediction

    May 17, 2026Siqiao Huang, Partha Kaushik, Michael Chen +5Action-Conditioned Video GenerationLong-Horizon Video Generation

    Tsinghua University · Carnegie Mellon University · University of Bristol +2

  58. DiLA: Disentangled Latent Action World Models

    May 15, 2026Tianqiu Zhang, Muyang Lyu, Yufan Zhang +2Action-Conditioned Video GenerationLatent Action Learning

    Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Peking University. · Center of Quantitative Biology, Peking University. · School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University.

  59. EgoExo-WM: Unlocking Exo Video for Ego World Models

    May 14, 2026Danny Tran, Roberto Martín-Martín, Kristen GraumanAction-Conditioned Video GenerationControllable Video Generation

    The University of Texas at Austin

  60. Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action

    May 14, 2026Yi Zhang, Yinda Chen, Che Liu +26Action-Conditioned Video GenerationUnified Multimodal Models

    Beijing Innovation Center of Humanoid Robotics (X-Humanoid) WFM System Group