LiMA: Bridging Long-term Imagination to Real-time Dexterous Manipulation via Asynchronous Diffusion

Ning Chen; Yankai Fu; Junkai Zhao; Qianpu Sun; Guocai Yao; Pengwei Wang; Zhongyuan Wang; Shanghang Zhang

cs.ROSep 23, 2026

LiMA: Bridging Long-term Imagination to Real-time Dexterous Manipulation via Asynchronous Diffusion

Authors: Ning Chen, Yankai Fu, Junkai Zhao, Qianpu Sun, Guocai Yao, Pengwei Wang, Zhongyuan Wang, Shanghang Zhang

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence

Abstract

Dexterous manipulation demands long-term foresight and rapid reactive control. Vision-Language-Action (VLA) models, while proficient in high-level reasoning, often lack a fine-grained understanding of physical dynamics and spatial perception. Conversely, World-Action Models (WAMs) typically suffer from high inference latency due to iterative generation. These deficiencies result in a critical temporal misalignment where the model's intent fails to adapt to rapid physical contact changes. To overcome this fundamental bottleneck, we propose LiMA, an asynchronous dual-system generative framework that systematically decouples intent planning from reactive execution. LiMA organizes computation into a multi-scale hierarchy: a slow system handles sparse long-horizon spatiotemporal intent generation, while a fast system focuses on dense high-frequency motion refinement. To align sparse intent predictions with dense action trajectories, we introduce a Latent Schrödinger Bridge Coupling mechanism that formulates refinement as an entropy-regularized probabilistic transport process. LiMA reduces inference latency by 45.8% compared with Cosmos-Policy via asynchronous decoupling. Evaluated across six bimanual dexterous manipulation tasks spanning multiple horizons, LiMA achieves an overall success rate of 70.8% and an average subtask success rate of 78.9%, while maintaining performance in unseen scenarios. The project website is available at https://ccdcs.github.io/LiMA_repo/

Figures & tables

Figure 2: Overview of LiMA. LiMA asynchronously coordinates long-horizon foresight and real-time reactive control through a dual-system architecture. The Future Dreamer produces structured intent, while the Motion Refiner performs high-frequency latent refinement. A Latent Schrödinger Bridge Coupling mechanism maintains latent-level coherence between the two systems for manipulation.

Method	Stack cup		Roll T-shirt		Cook Rice		Make Sandwich		Make Coffee		Assemble package		Time (ms)
	SR	PSR	SR	PSR	SR	PSR	SR	PSR	SR	PSR	SR	PSR	-
GR00T N1.6	85.0%	87.5%	70.0%	71.7%	70.0%	72.0%	65.0%	72.5%	55.0%	71.7 %	50.0%	61.7%	270
VPP	80.0%	80.0%	55.0%	56.7%	55.0%	60.0%	50.0%	61.3%	50.0%	56.7%	40.0%	48.3%	225
Internvla-a1	90.0%	90.0%	75.0%	88.3%	70.0%	78.0%	65.0%	77.5%	50.0%	66.7%	50.0%	58.3%	360
cosmos-policy	80.0%	82.5%	60.0%	70.0%	60.0%	67.0%	60.0%	71.3%	55.0%	63.3%	45.0%	51.7%	600
LiMA (Ours)	90.0%	95.0%	75.0%	83.3%	80.0%	82.0%	70.0%	80.0%	60.0%	70.0%	50.0%	63.3%	325

Table 1: Comparison of LiMA and baselines on Real-World Tasks. Each experiment is evaluated with 20 trials. Inference speed is evaluated on a NVIDIA H100 GPU.

Variant	Cook Rice	Stack Cup
w/o Future Visual Intent	$61.7\pm 7.6$	$76.7\pm 2.9$
Gaussian Init. + Cross-Attn	$70.0\pm 5.0$	$78.3\pm 2.9$
Flow Matching	$73.3\pm 2.9$	$83.3\pm 5.8$
LiMA	$\mathbf{78.3\pm 2.9}$	$\mathbf{93.3\pm 5.8}$

Table 2: Bridge design ablation. Success rates (%) are reported as mean

\pm

standard deviation over three seeds, with 20 real-world trials per seed.

(a) Component Ablation			(b) Denoising Steps			(c) System Ratio ( $N_{s}:N_{f}$ )
Component	S.R.	Time	Steps	S.R.	Time	Ratio	S.R.	Time
Full (LiMA)	80%	0.325s	5	65%	0.23s	1:9	50%	0.28s
w/o Adapt.	60%	-	10	80%	0.325s	2:8	65%	0.31s
w/o Tokens	55%	-	15	85%	0.43s	3:7	80%	0.325s
w/o Fast-Slow	80%	0.45s	20	80%	0.51s	4:6	70%	0.34s

Table 3: Ablation Studies. We evaluate LiMA across three dimensions: architecture components, denoising steps, and system ratios (

N_{slow}:N_{fast}

). S.R.: Success Rate, Time: Inference Latency.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Overview of the real-world bimanual manipulation platform. The system consists of two UR5 robotic arms equipped with SharpaWave dexterous hands, a three-view RealSense D435 camera setup with one head-mounted camera and two wrist-mounted cameras, and a wearable teleoperation interface based on VIVE Trackers and MetaGlove Pro gloves for expert demonstration collection.

Figure 7: Visualization of task progress.

Task	P1	P2	P3	P4	P5
Stack Cup	Grasp cup	Stack cup	–	–	–
Roll T-shirt	Grasp corners	Fold garment	Pick and place	–	–
Cook Rice	Grasp spoon	Scoop rice	Transfer rice	Close cooker	Return spoon
Make Sandwich	Pick up tongs	Grasp lettuce	Place lettuce and return tongs	Cover with bread	–
Make Coffee	Pick up coffee bag	Pick up kettle	Hand over cup	–	–
Assemble Package	Pick up black part	Pick up transparent part	Combine parts	–	–

Appendix

Table 4: Progress stage definitions for real-world tasks. Each task is decomposed into several progress stages for computing Progress Success Rate (PSR).

Figure 8: Visualization of Dreamer & Refiner Generations.

Task	Method	P1(%)	P2(%)	P3(%)	P4(%)	P5(%)	PSR(%)
Stack Cup	LiMA	100	90.0	–	–	–	95.0
	Cosmos-Policy	85.0	80.0	–	–	–	82.5
	GR00T N1.6	90.0	85.0	–	–	–	87.5
	VPP	80.0	80.0	–	–	–	80.0
	InternVLA-A1	90.0	90.0	–	–	–	90.0
Roll T-shirt	LiMA	95.0	80.0	75.0	–	–	83.3

Appendix

Table 5: Per-stage success rates for progress evaluation. We report the success rate of each progress stage to analyze where different methods fail during task execution.

Figure 9: Visualization of Generalizations.

Explore similar work

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model
Jun 10, 2026Pankhuri Vanjani, Zhuoyue Li, Jakub Suliga +4AI Control Robotic Manipulation
Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT), Germany · NVIDIA · Robotics Institute of Germany
ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation
Jun 12, 2026Yanzhao Guo, Wenkai Chen, Jianwei ZhangLanguage-Conditioned Robot Manipulation Efficient VLM Inference
Shanghai Jiao Tong University · Technical Aspects of Multimodal Systems (TAMS), Department of Informatics, Universität Hamburg
WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
Oct 13, 2026Haomin Wang, Anqiong Ji, Ke Wang +4World Models for Robotics World Action Models
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics +1

Jun 10, 2026J/K move · Enter open · S save

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model

Pankhuri Vanjani, Zhuoyue Li, Jakub Suliga, Moritz Reuss+3

Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT), Germany · NVIDIA · Robotics Institute of Germany

AI Control Robotic Manipulation Vision-Language-Action Models VLMs for Robotics Multimodal Sensor Fusion

Open paper AskPDF arXiv

Vision-language-action (VLA) models inherit a shared synchronous clock from vision-language pretraining, processing every input at one rate. This is misaligned with physical interaction, where a high-frequency modality changes at hundreds of hertz, vision evolves more slowly, and language stays constant across an episode. A synchronous VLA oversamples slow modalities, undersamples fast ones, and caps action generation at the lowest effective frequency. We hypothesize that decoupling temporal processing per modality, letting each update and retain information at its own sensor rate, yields stronger representations and more robust control. We present DAM-VLA, which maintains per-modality latent buffers refreshed at sensor rates and read continuously by the action head, integrating new high-frequency modalities through gated cross-attention that leaves the pretrained backbone intact. Across seven contact-rich real-world manipulation tasks, DAM-VLA more than doubles the average success rate of the strongest synchronous baseline (95.2% vs.\ 40.95%) while sustaining smooth, reactive 100,Hz control. Project website: intuitive-robots.github.io/DAM-VLA/

Abstract

Explore similar work

DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model

ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation

WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction