Latent Action Learning

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 74

All topics

ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
Oct 6, 2026Yuan Xu, Yixiang Chen, Qisen Ma +10Latent Action Learning Visuomotor Policy Learning
CASIA · UCAS · Alibaba Group +1
Less Language, More Latents: Annotation-Efficient VLAs for Driving
Sep 23, 2026Alexey Zakharov, Kemal Oksuz, Puneet K. DokaniaLatent Action Learning Autonomous Driving
Robert Bosch GmbH, Germany · Five AI Ltd., United Kingdom
Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models
Sep 20, 2026Di Wen, Ruodi Zhang, Kailun Yang +1Latent Action Learning Latent Dynamics Modeling
Karlsruhe Institute of Technology, Karlsruhe 76131, Germany · Hunan University, Changsha 410012, China
Improving Cross-embodiment Transfer in Latent Action Models with Action-Similarity Supervision
Sep 17, 2026Maxime Alvarez, Renzo Caballero, Tatsuya Matsushima +2Latent Action Learning Robotic Manipulation
Graduate School of Engineering, The University of Tokyo
Towards High-DoF Dexterous Manipulation through VLA Post-Training
Sep 17, 2026Junlei Zhu, Shenzhe Yao, Chaogui Huang +6Latent Action Learning Robotic Manipulation
Wuji Technology · School of Information Science and Technology, ShanghaiTech University, China
UMI-Bridge: Action-Anchored Latent Alignment across Human and Robot Manipulation Data
Sep 16, 2026Haiyi Liu, Jinming Ma, Ke Rui +12Latent Action Learning Cross-Embodiment Robot Learning
Tsinghua University · Simple AI
TacBPM: A Tactile-conditioned Behavior Prior Model for Dexterous Reorientation
Sep 16, 2026Jie Yin, Wanli Xing, Zeyuan Zhao +3Latent Action Learning Dexterous Manipulation
Sharpa Robotics
GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
Sep 15, 2026Yifan Xie, Hekun Tian, Jinkun Liu +3Latent Action Learning Geometric Representation Learning
Tsinghua University · Beijing Institute of Technology · Fudan University
Learning Options for Compositional Motor Control with Adapter Banks
Sep 15, 2026Sreejan Kumar, Marcelo Mattar, Lea DunckerRL Control Latent Action Learning
Columbia University · New York University
WLA $^3$ : World Latent Action Modeling for Semantics, Dynamics, and Kinematics
Sep 14, 2026Peidong Liu, Zhiyuan Xiang, Mingyang Li +4Latent Action Learning World Models for Robotics
Joy Future Academy, JD Group
Reconstructing Is Not Acting: Action-Centric Latent Dynamics Modeling
Sep 14, 2026Dingjie Fu, Dianxing Shi, Yangyang Xu +1Latent Action Learning Latent Dynamics Modeling
Harbin Institute of Technology (Shenzhen) · Beihang University
Beyond Noise Steering: Dual-Latent Space Reinforcement Learning for Generative Robot Policy
Sep 11, 2026Pengfei Zhang, Teng Sun, Xianchao XiuLatent Action Learning Robot Policy Learning
School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving
Sep 3, 2026Ruoyu Yao, Yusen Xie, Qingzhao Liu +5Latent Action Learning End-to-End Autonomous Driving
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Central Media Technology Institute, Huawei
Latent Action as Intention Enables Efficient Future Imagination for World Action Models
Aug 25, 2026Xiang Li, Yupeng Zheng, Songen Gu +11Latent Action Learning World Models for Robotics
1CollegeAI, THU · 2AIR, THU · 4TARS Robotics +6
LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation
Aug 7, 2026Cheng Guo, Mingzhe Ni, Angelo Cangelosi +1Humanoid Loco-Manipulation Latent Action Learning
Department of Computer Science, University of Manchester, Manchester, UK · Human-Robot Interfaces and Interaction Laboratory, Italian Institute of Technology, Genoa, Italy
LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models
Aug 6, 2026Jiarui Yang, Jiale Zhange, Jiawei Li +5Latent Action Learning World Models for Robotics
Nankai University · Tsinghua University · Swiss Federal Institute of Technology in Lausanne (EPFL) +1
JoyAI-RA 0.5: Scaling Robot Manipulation Learning via Dual Action Alignment
Aug 6, 2026JoyAI-RA TeamRL for Robotics Latent Action Learning
Joy Future Academy, JD
Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actions
Aug 4, 2026Zhenyang Feng, Unnat JainLatent Action Learning Visuomotor Policy Learning
Department of Computer Science, University of California, Irvine
ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow
Jul 30, 2026Jin Cao, Zian Meng, Kaipeng ZhangLatent Action Learning Action-Conditioned World Models
1Alaya Lab · 2Shanghai Innovation Institute
DLAM: Distributional Latent Actions with Temporal Constraints
Jul 29, 2026Zuojin Tang, Feifan Luo, Haoyun Liu +10Latent Action Learning Latent Dynamics Modeling
Zhejiang University · Nanjing University · Shanghai Jiaotong University +4
ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games
Jul 29, 2026Zhenfeng Gan, ZiTong Zeng, Jiajun Cheng +3Latent Action Learning World Model-Based Planning
Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation
Jul 27, 2026Valerio Belli, Valerio Modugno, Enrico Mingo Hoffman +1RL for Robotics Latent Action Learning
UNIROMA, UCL · Sapienza University of Rome, Rome, Italy · University College London, London, UK +3
Latent-Action-Guided Video-Language Feature Learning for Surgical Instrument-Tissue Interaction Recognition
Jul 22, 2026Jiajun Cheng, Sainan Liu, Subarna Tripathi +2Latent Action Learning Surgical Video Understanding
Arizona State University, Tempe, AZ, USA · Intel Corporation, USA · University of California, Merced, CA, USA
EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation
Jul 13, 2026Yuecheng Xu, Tong Yang, Jingkai Jia +3Latent Action Learning Robot Policy Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · TeleAI, China Telecom
WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos
Jul 13, 2026Jiahao Liu, Yupeng Zheng, Zhongpu Xia +10Latent Action Learning Robot Policy Learning
CASIA · Anyverse Dynamics · UCAS +2
Towards Predictive, Aligned, and Scalable Robot Learning
Jul 13, 2026Peijun Tang, Shangjin Xie, Baifu Huang +6Latent Action Learning Cross-Modal Alignment
Astribot Team
Causally Debiased Latent Action Model for Embodied Action Conditioned World Models
Jul 10, 2026Yufan Wei, Kun Zhou, Lingjun Mao +9Latent Action Learning Causal Representation Learning
Aether AI · University of California, San Diego
FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
Jul 9, 2026Michael Murray, Daphne Chen, Simran Bagaria +7Latent Action Learning Robot Policy Learning
1Microsoft Research · University of Washington · 2Microsoft +1
Latent Memory Palace: Reasoning for Control as Autoregressive Variational Inference
Jul 9, 2026Chuning Zhu, Eva Xu, Jose Barreiros +3RL Control Latent Action Learning
University of Washington · Toyota Research Institute
LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation
Jul 7, 2026Xinye Yang, Zhiyuan Ma, Hongze Yu +5Latent Action Learning Robot Skill Learning
Fudan University · PsiBot · Zhongguancun Academy +2
CAC-VLA: Context-Gated Action Conditioning for Vision-Language-Action Models
Jul 6, 2026Yifu Xiong, Wenhao Yu, Jiaxuan Lin +5Latent Action Learning Language-Conditioned Robot Manipulation
University of Science and Technology of China (USTC) · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Geometry-Aware Motion Latents for Learning Robust Manipulation Policies
Jul 6, 2026Yunchao Zhang, Yijia Weng, Ruizhe Liu +3Latent Action Learning Geometric Representation Learning
Department of Computer Science, The University of Hong Kong, Hong Kong, China · Department of Computer Science, Stanford University, Stanford, California, USA · Department of Data Science and AI, Monash University, Melbourne, Australia
ABot-M0.5: Unified Mobility-and-Manipulation World Action Model
Jul 1, 2026Ronghan Chen, Yandan Yang, Zuojin Tang +18Latent Action Learning Long-Horizon Robotic Manipulation
AMAP CV Lab Alibaba Group
Latent Actions from Factorized Transition Effects under Agent Ambiguity
Jun 29, 2026Heejeong Nam, Chandradithya S Jonnalagadda, Harshit Aggarwal +2Latent Action Learning Latent Dynamics Modeling
Brown University
LaGO: Latent Action Guidance for Online Reinforcement Learning
Jun 23, 2026Kuan-Yen Liu, Ren-Jyun Huang, Ti-Rong WuLatent Action Learning LLM-Guided RL
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, USA · Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan.
CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation
Jun 22, 2026Sikai Li, Shuning Li, Zhenyu Wei +3RL for Robotics Humanoid Loco-Manipulation
University of North Carolina at Chapel Hill · University of California, Berkeley
Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models
Jun 19, 2026Tianyou Wang, Anson Lei, Joe Watson +1Latent Action Learning Visuomotor Policy Learning
University of Oxford
VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation
Jun 19, 2026Zhigen Zhao, Mark Leggiero, Yipu Chen +5Latent Action Learning Language-Conditioned Robot Manipulation
Institute for Robotics and Intelligent Machines (IRIM), Georgia Institute of Technology, Atlanta, GA 30332
PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning
Jun 19, 2026Youngjoon Jeong, Jihwan Yu, Minsoo Jo +2Latent Action Learning Robot Policy Learning
Seoul National University
FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning
Jun 17, 2026Takanori Yoshimoto, Yang Hu, Naruya Kondo +1Latent Action Learning
University of Tsukuba · The University of Tokyo
Motion-Focused Latent Action Enables Cross-Embodiment VLA Training from Human EgoVideos
Jun 17, 2026Runze Xu, Yiluo Zhang, Jian Wang +2Latent Action Learning VLM Adaptation
Department of Electronic Engineering, Tsinghua University · Tianfu Jiangxi Laboratory
MotionPyramid: Hierarchical Motion Representation and Residual Interfaces
Jun 15, 2026Gao Zhu, Zaishuo Xia, Yubei ChenLatent Action Learning Hierarchical Representation Learning
UC Davis
Steering Generative Reinforcement Learning into Stable Robotic Controller
Jun 15, 2026Yixuan Wang, Shutong Ding, Ke Hu +3Latent Action Learning Robotic RL
1ShanghaiTech University
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jun 14, 2026Jialei Chen, Kai Wang, Kang Chen +9Latent Action Learning Action-Conditioned World Models
Jilin University · Zhongguancun Academy · Nankai University +4
RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Jun 11, 2026Junke Wang, Qihang Zhang, Shuai Yang +5Latent Action Learning World Models for Robotics
Institute of Trustworthy Embodied AI, Fudan University · 2Robbyant, Ant Group · 3Hongkong University of Science and Technology
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
Jun 9, 2026Yi Chen, Rushuai Yang, Qiang Chen +2Latent Action Learning Markov Decision Processes
Lucy · Department of Industrial Engineering and Decision Analytics Hong Kong University of Science and Technology
LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching
Jun 9, 2026Jiexi Lyu, Xizhou Bu, Qingqiu Huang +4Flow Matching Latent Action Learning
HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
Jun 9, 2026Xiaoquan Sun, Ruijian Zhang, Chen Cao +12Latent Action Learning Long-Horizon Robotic Manipulation
3Huazhong University of Science and Technology · 1The University of Hong Kong · 2INFIFORCE +3
LARA: Latent Action Representation Alignment for Vision-Language-Action Models
Jun 5, 2026Mengya Liu, Baoxiong Jia, Jiangyong Huang +2Latent Action Learning Robotic Manipulation
State Key Laboratory of General Artificial Intelligence, BIGAI · Peking University · Delta Intelligence
CLAW: Learning Continuous Latent Action World Models via Adversarial Latent Regularization
Jun 2, 2026Tewodros Ayalew, Matthew Jeung, Samuel Wheeler +5Latent Action Learning Imitation Learning
University of Chicago · 3Argonne National Laboratory · 2Toyota Technological Institute at Chicago
PHASOR: Phase-Anchored Universal Action Representations for Humanoid Embodiments
Jun 1, 2026Kihyun Kim, Chaeyun Kim, Jongho Shin +4Latent Action Learning Representation Learning
AIM Intelligence · Seoul National University · LG Electronics +2
Enhancing Human-Likeness in Reinforcement Learning Agents via Hierarchical Macro Action Quantization
May 29, 2026M. Shaheer Luqman, Usman Nizamani, Fawad Javed Fateh +4Latent Action Learning Reinforcement Learning
Retrocausal, Inc., Redmond, WA 98052, USA
Learning High-Frequency Continuous Action Chunks in Latent Space
May 24, 2026Kunyun Wang, Yuhang Zheng, Yupeng Zheng +2Robotic Control Latent Action Learning
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · TARS Robotics · National University of Singapore, Singapore +2
MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots
May 23, 2026Yusen Feng, Xiang Wang, Heyuan Yao +7Latent Action Learning Humanoid Robot Locomotion
Peking University
Projecting Latent RL Actions: Towards Generalizable and Scalable Graph Combinatorial Optimization
May 19, 2026Franco Terranova, Guillermo Bernardez, Albert Cabellos-Aparicio +2Latent Action Learning Combinatorial Optimization
UL, LORIA, Inria · Université de Lorraine, CNRS, Inria, LORIA · UC Santa Barbara +4
Latent Action Reparameterization for Efficient Agent Inference
May 18, 2026Wenhao Huang, Qingwen Zeng, Qiyue Chen +11LLM Inference Efficiency Latent Action Learning
Université de Montréal · Mila - Quebec AI Institute · The University of Sydney +7
Imitation learning for clinical decision support in pediatric ECMO
May 15, 2026Fateme Golivand, Michael Skinner, Saurabh Mathur +5Latent Action Learning Clinical Decision Support
University of Texas at Dallas · Technische Universität Darmstadt · Swarthmore College +1
Ada-Diffuser: Latent-Aware Adaptive Diffusion for Decision-Making
May 15, 2026Fan Feng, Selena Ge, Minghao Fu +6Latent Action Learning Latent Dynamics Modeling
University of California San Diego · MBZUAI · Carnegie Mellon University +2
DiLA: Disentangled Latent Action World Models
May 15, 2026Tianqiu Zhang, Muyang Lyu, Yufan Zhang +2Action-Conditioned Video Generation Latent Action Learning
Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Peking University. · Center of Quantitative Biology, Peking University. · School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University.
Before the Body Moves: Learning Anticipatory Joint Intent for Language-Conditioned Humanoid Control
May 14, 2026Haozhe Jia, Honglei Jin, Yuan Zhang +9Robotic Control Latent Action Learning
The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. +3

Oct 6, 2026, cs.ROJ/K move · Enter open · S save

ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models

Yuan Xu, Yixiang Chen, Qisen Ma, Jiabing Yang+9

CASIA · UCAS · Alibaba Group+1

Latent Action Learning Visuomotor Policy Learning Robotic Manipulation Vision-Language-Action Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have become a central paradigm for robot policy learning, which predict actions in three forms: raw action chunks, discrete action tokens, or continuous action latents. However, existing action representations primarily model action trajectories, with limited consideration of the visual dynamics induced by these actions. We introduce ViDAL, a Visual Dynamics-grounded Action Latent Space that anchors continuous action latents in the future visual dynamics of the scene. Specifically, ViDAL learns action latent space by training an Action Variational Autoencoder (Action VAE) to reconstruct action chunks while aligning its latent with future scene dynamics. When integrated into downstream robot policies, the proposed Action VAE serves as a plug-in action interface compatible with multiple VLA architectures and enables optional future-video prediction as an additional capability. Empirically, ViDAL outperforms competitive baselines on LIBERO with 98.1% average success, improves a multi-task

π_{0.5}

policy on RoboTwin 2.0 from 54.3% to 65.5% (Clean) and from 33.2% to 43.1% (Random) success rates over 50 dual-arm tasks, and yields 20.0% and 23.4% absolute success-rate gains on real-world single-arm Franka and dual-arm ARX robot platforms.

Momentum

Related topics

Latest papers 74