Latent RL Priors for Data-Efficient Dexterous Manipulation

Yixiao Wang; Peiyi Chen; Mustafa Mukadam; Shreekant Gayaka; Masayoshi Tomizuka; Son Tran

cs.RONov 5, 2026

Latent RL Priors for Data-Efficient Dexterous Manipulation

Authors: Yixiao Wang, Peiyi Chen, Mustafa Mukadam, Shreekant Gayaka, Masayoshi Tomizuka, Son Tran

Organizations: University of California, Berkeley · Amazon

Abstract

Learning dexterous manipulation from limited visual demonstrations is challenging because policies must acquire both perceptual grounding and precise control, while reinforcement learning (RL) in simulation can acquire rich control behaviors from large-scale interaction with privileged information. We introduce Latent RL Prior, a two-stage framework that reuses this control knowledge for data-efficient visual policy learning. During privileged pretraining, we distill large-scale RL rollouts into a flow-matching action decoder conditioned on proprioception and latent task context. During visual adaptation, we freeze the decoder and learn a visual interface that maps images to its latent context using action supervision alone, without privileged labels or latent targets. The same decoder can further be shared across arms by learning coordinated visual contexts for bimanual manipulation. Across three simulated tasks, our method outperforms full-parameter fine-tuning of $π_{0.5}$ and GR00T N1.7 in the low-data regime; on door lifting and bimanual assembly, it surpasses full-data baselines using $10\times$ and $5\times$ fewer demonstrations, respectively. Further analysis shows that direct visual-to-latent conditioning improves adaptation performance and training stability over routing visual predictions through the pretrained state encoder. Robustness to unseen mass variation and greater linear accessibility of contact-force information in decoder representations further provide evidence of retained RL-acquired control knowledge. Project website: https://yixiaowang7.github.io/latent-rl-prior-page/.

Figures & tables

Figure 1: Latent RL Prior. (a) Privileged pretraining: a state tokenizer and Q-Former supply task context to an action decoder conditioned on flow time and single-arm proprioception. (b) Visual adaptation: an image encoder and Q-Former learn context tokens while the decoder remains frozen. For bimanual assembly, left- and right-arm queries interact within the same Q-Former; each output group conditions the shared decoder with its corresponding arm’s proprioception.

Figure 2: Pretraining and downstream tasks. Left: the pretraining task of target pose alignment with varied procedural objects; the inset shows example object templates. Right: the downstream tasks of target pose alignment with a held-out brush, door lifting, and bimanual assembly (left to right). The same pretrained action decoder is reused for visual adaptation across all three downstream tasks.

Figure 4: Comparison with full-parameter fine-tuning of GR00T N1.7 and $\pi_{0.5}$ across demonstration budgets. (a) Target pose alignment at strict ( $\epsilon=0.015$ m) and relaxed ( $\epsilon=0.05$ m) thresholds; (b) door lifting; (c) bimanual assembly. We report average success rate and standard deviation across three random seeds.

Figure 5: Target pose-alignment ablations with the action decoder frozen for both latent-transfer variants. Results show mean success rates and standard deviations over three random seeds, except for explicit transfer, which uses five seeds.

Figure 6: Training stability on target pose alignment at a relaxed (

\epsilon=0.05

m) threshold. Dotted curves show individual runs for latent transfer, explicit transfer, and scratch training; shaded regions show the mean

\pm

one standard deviation. Explicit transfer uses five random seeds, while latent transfer and scratch training use three seeds each.

Figure 7: Learning-rate ablations for (a) the image encoder with the action decoder frozen and (b) the action decoder with the image encoder frozen. Panels report strict (

\epsilon=0.015

m) and relaxed (

\epsilon=0.05

m) pose-alignment success. We report average success rate and standard deviation across three random seeds. Each scale multiplies the tested module’s original learning rate; zero freezes it.

Figure 8: Generalization to increased object mass. (a) Target pose alignment at a relaxed (

\epsilon=0.05

m) thresholds. (b) Door lift. The leftmost point in each plot corresponds to the reference mass used for downstream visual training, e.g., for

\mathcal{D}_{\mathrm{vis}}

Figure 9: Linear contact-force probes of the visual policies. Panels predict (a) total grasp normal force, (b) per-finger normal forces, (c) per-finger tangential forces, and (d) per-finger Cartesian force vectors from each layer of the action decoder. Colors denote visual-demonstration budgets; solid and dashed curves denote the frozen pretrained and trainable scratch action decoders. Both policies fine-tune the image encoder at

0.2\times

its base learning rate for 100k steps. Curves average three policy seeds on the same IID probe test episodes.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,756 papers

Jul 7, 2026

LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation

Real-world learning for dexterous hands remains brittle because high-dimensional hand actions amplify imitation errors and make reinforcement-learning exploration prone to contact-breaking motion. While combining imitation learning (IL) with online reinforcement learning (RL) can reduce manual supervision, unconstrained exploration in raw hand-action spaces is sample-inefficient and risky for physical hardware. We introduce a latent motion prior module (\prior{}) that maps recent hand-action histories to a compact, history-conditioned latent prior and decodes continuous latent commands into executable high-dimensional hand targets. Built on this prior, \method{} is a three-stage real-world dexterous learning framework: it pretrains \prior{} from demonstrations, trains a visuomotor policy that predicts native arm commands and latent hand-action offsets, and improves the policy with online residual RL in the same latent hand-action space. This shared, decodable interface lets residual exploration make local corrections near demonstrated, contact-consistent hand motions rather than perturbing every finger joint independently. We evaluate \method{} on four real-robot dexterous manipulation tasks against raw, linear, and discrete hand-action interfaces. Starting from small task-specific demonstration sets, \method{} achieves a 56.25% average IL success rate and raises it to 98.75% after online RL, reaching 100% final success on three tasks and 95% on the remaining task.

Xinye Yang, Zhiyuan Ma, Hongze Yu +5

Fudan University · PsiBot · Zhongguancun Academy +2

Latent Action Learning Robot Skill Learning Visuomotor Policy Learning Robotic Manipulation Robot Imitation Learning

Ask PDF

Oct 8, 2026

Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration

While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. Our key insight is to abstract the video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability. The graphs serve as generative constraints for sampling diverse reset states and provide dense rewards for each stage. Because the graphs constrain relations rather than exact poses, these reset states cover object poses and grasps beyond the video, while initializing each stage from them with dense rewards keeps exploration short and guided. Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks, Dex-One2Many exceeds baselines by 6.5% in seen configurations, while its robust generalization widens this gap to 71% in unseen scenarios.

Jusuk Lee, Sungha Kim, Yeonsoo Park +8

Seoul National University · University of Maryland, College Park · All Purpose AI +1

Robotic Manipulation Learning from Demonstration Zero-Shot Robotic Manipulation RL for Robotics Reinforcement Learning Sim-to-Real Transfer

Ask PDF

Sep 14, 2026

Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation

Real-world reinforcement learning (RL) offers a promising route to dexterous manipulation policies that can adapt directly from physical interaction, but learning is hindered by inefficient early exploration and costly failures. We propose a framework that uses sampling-based model predictive control (MPC) as scaffolding for real-world dexterous RL, providing structured prior experience and task-directed guidance during learning without human demonstrations or corrective actions. A small set of MPC trajectories is first used to populate an offline replay buffer and to pretrain the actor and critic. During online learning, MPC intermittently guides data collection while an off-policy Soft Actor-Critic learner trains from both prior MPC experience and newly collected physical interaction, with control gradually transitioning to the learned policy. On continuous in-hand rotation with a 16-DoF Allegro hand, initialized from 20 MPC trajectories collected in 12 minutes on hardware, the policy reaches 100% success after 7 minutes of online RL, with about three object drops on average during training. After 20 minutes of online learning, the policy achieves more than five times the rotation speed of the MPC controller and completes 1000 consecutive rotations without a drop. Ablations show complementary benefits from MPC-based pretraining, retained MPC experience, and online MPC guidance, while additional experiments demonstrate rapid adaptation to new object geometries and successful goal-conditioned reorientation.

Emek Barış Küçüktabak, Karankumar Patel, Zhaodong Yang +3

Honda Research Institute USA, San Jose, CA, USA · Skild AI, San Mateo, CA, USA · Georgia Institute of Technology, Atlanta, GA, USA

Dexterous Manipulation Contact-Rich Robotic Manipulation Robotic RL Offline-To-Online RL

Ask PDF

Abstract

Robotic Manipulation

Explore similar work

LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation

Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration

Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation