Action Representation Learning

Latest papers

11 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation
Oct 2, 2026Xiangwei Jiang, Yao Mu, Lixin Duan +1Action Representation Learning Cross-Embodiment Robot Learning
University of Electronic Science and Technology of China, China. · Shanghai Jiao Tong University, China.
Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models
Sep 24, 2026Yufei Duan, Hang Yin, Alberta Longhini +2Action Representation Learning Robotic Manipulation
Department of Robotics, Perception and Learning, KTH Royal Institude of Technology, Stockholm, Sweden
Behavior-Aligned Action Tokenization for Robot Policy Learning
Sep 23, 2026Junbo Dong, Ze Chen, Zhendong Xie +5Robot Policy Learning Action Representation Learning
Southern University of Science and Technology · Dexmal · National University of Singapore +1
LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes
Aug 25, 2026Youcheng Zong, Runda Jia, Dakuo HeAction Representation Learning Policy Learning
College of Information Science and Engineering, Northeastern University, Shenyang 110004, China
Semantic Anchoring for Robotic Action Representations
Jul 15, 2026Yuan Xu, Youheng Shi, Chengyang Li +2Action Representation Learning Generalization in Robotic Manipulation
Peking University · Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University
EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation
Jul 13, 2026Yuecheng Xu, Tong Yang, Jingkai Jia +3Latent Action Learning Robot Policy Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · TeleAI, China Telecom
Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model
Jun 25, 2026Zizhao Yuan, Zhengtu Liang, Taowen Wang +7Action-Conditioned Video Generation Action Representation Learning
The Hong Kong University of Science and Technology (Guangzhou) · Shenzhen University · Beijing University of Technology +1
ActionMap: Robot Policy Learning via Voxel Action Heatmap
Jun 5, 2026Pei Yang, Hai Ci, Yanzhe Chen +3Robot Policy Learning Robot Skill Learning
1Show Lab, National University of Singapore · 2NVIDIA
PHASOR: Phase-Anchored Universal Action Representations for Humanoid Embodiments
Jun 1, 2026Kihyun Kim, Chaeyun Kim, Jongho Shin +4Action Representation Learning Latent Action Learning
AIM Intelligence · Seoul National University · LG Electronics +2
Action Images: End-to-End Policy Learning via Multiview Video Generation
Apr 7, 2026Haoyu Zhen, Zixian Gao, Qiao Sun +7Action Representation Learning World Action Models
UMass Amherst · NVIDIA · Harvard University +1
FEEL (Force-Enhanced Egocentric Learning): A Dataset for Physical Action Understanding
Mar 16, 2026Eadom Dessalene, Botao He, Michael Maynord +5Action Representation Learning Egocentric Video Understanding
University of Maryland, College Park, USA

Oct 2, 2026J/K move · Enter open · S save

DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation

Xiangwei Jiang, Yao Mu, Lixin Duan, Wen Li

University of Electronic Science and Technology of China, China. · Shanghai Jiao Tong University, China.

Action Representation Learning Cross-Embodiment Robot Learning Robotic Manipulation Robot Manipulation Benchmarks Bimanual Robotic Manipulation

Open paper AskPDF arXiv

As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical. Scalable cross-embodiment learning therefore requires a unified representation that captures shared manipulation structure while preserving morphology-specific control. Differences in hands, tasks, datasets, and control interfaces prevent existing studies from isolating the effects of representation, pretraining, and architecture. We introduce DexJoCo-X, a benchmark and toolkit for controlled comparison across seven representative dexterous hands, six single-arm and bimanual tasks, and 2,100 balanced demonstrations. DexJoCo-X provides a matched multi-hand, multi-task protocol with common scenes, success criteria, and execution interfaces, redesigned glove-to-hand mappings, and an automated pipeline that expands reviewed demonstrations across randomized scenes. Using

π_{0.5}

, Ego-Pi, and Being-H0.5, we examine whether a shared action interface is sufficient for multi-hand learning. Expanding

π_{0.5}

to an 80-dimensional bimanual output yields near-zero success. Ego-Pi preserves the pretrained action head through interleaved prediction and supports per-hand multi-task learning, but remains ineffective for seven-hand joint training. By contrast, Being-H0.5 combines cross-embodiment pretraining, a unified action space, and embodiment-aware experts, enabling one policy to control all seven hands. Within this architecture, function-aligned action slots achieve 47.7% mean success, compared with 47.0% for native coordinates and 33.1% for DexLatent. These results show that cross-embodiment representation depends on the entire learning system: action coordinates, pretraining, and architecture must jointly separate shared manipulation structure from embodiment-specific control.