ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning

Chaehyun Kim; Chi-Pin Huang; Jisoo Kim; Min-Hung Chen; Yu-Chiang Frank Wang; Seungryong Kim; Fu-En Yang

cs.ROOct 27, 2026

ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning

Authors: Chaehyun Kim, Chi-Pin Huang, Jisoo Kim, Min-Hung Chen, Yu-Chiang Frank Wang, Seungryong Kim, Fu-En Yang

Organizations: NVIDIA · KAIST

Abstract

Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action thought as a Chain-of-Dynamics (CoD), a sequence of discrete motion tokens that captures the intermediate dynamics connecting the current world to a future state and serves as a common action representation across embodiments. We first initialize CoD with action-labeled demonstrations, then predict the future induced by each CoD and reinforce those whose predictions better match the observed outcome. This enables videos without action annotations to provide direct supervision from observed state transitions, allowing human and robot experience to refine the same reasoning space. Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments.

Figures & tables

Figure 1: We introduce ThinkWorld , a vision-language-action framework that learns transferable Chain-of-Dynamics (CoD) reasoning from diverse human and robot experiences. CoD captures action-induced dynamics across embodiments and is reinforced through future imagination, enabling robust generalization in the real world and skill composition guided by human videos.

Figure 2: ThinkWorld architecture and training stages. In stage (a), supervised fine-tuning (SFT) tokenizes robot actions and annotated human hand motions into a unified discrete motion space to ground Chain-of-Dynamics (CoD) reasoning, while CoD-conditioned imagination predicts the future visual feature. In stage (b), reinforcement learning (RL) uses the alignment between imagined and observed futures as a scalable reward for CoD reasoning over human and robot videos, strengthening action reasoning without motion annotations and improving guidance for continuous control.

Figure 3: Real-world out-of-distribution (OOD) generalization. Success rates for color matching, cup hanging, and sorting under in-domain and OOD conditions. OOD settings introduce new colors, objects, environments, or sorting trajectories, as illustrated below the plots. ThinkWorld achieves the highest OOD success rates across all three tasks, despite substantial drops for the baselines.

Figure 4: Composing atomic robot skills from a human demonstration. The top shows atomic robot skills, while the middle shows a human demonstration that provides the complete task composition, e.g., open $\rightarrow$ put-left $\rightarrow$ close. Guided by this human demonstration, ThinkWorld composes the corresponding robot skills and executes the complete task in a single rollout at the bottom .

		LIBERO					LIBERO-PRO
Model	Params.	Spatial	Object	Goal	Long	Avg.	Object	Position	Semantic	Task	Environment	Avg.
OpenVLA ( Kim et al., 2024 )	$7.5$	84.7	88.4	79.2	53.7	76.5	93.0	0.0	97.3	0.0	68.0	51.7
$\pi_{0}$ ( Black et al., 2024a )	$3.2$	96.8	98.8	95.8	85.2	94.2	90.5	0.0	90.5	0.0	38.8	44.0
$\pi_{0.5}$ ( Intelligence et al., 2025 )	$3.4$	98.8	98.2	98.0	92.4	96.9	96.0	20.8	95.8	0.8	52.8	53.2
MolmoAct ( Lee et al., 2025 )	$8.1$	87.0	95.4	87.6	77.2	86.6	76.0	1.5	85.8	1.5	–	41.2
VLA-JEPA ( Sun et al., 2026a )	$2.8$	96.2	99.6	97.2	95.8	97.2	80.4	3.5	97.2	20.8	41.0	48.5 †

Table 1: Success rates (%) on LIBERO and LIBERO-PRO. LIBERO-PRO axis values are arithmetic means over Goal, Spatial, LIBERO-10, and Object. LIBERO-PRO baseline results are taken from the official repository unless marked with † , which denotes our reproduced result. Best and second-best results within each benchmark column are shown in bold and underlined , respectively.

Figure 7: Effect of Chain-of-Dynamics (CoD) on future prediction. We compare future DINO predictions under generated, shuffled, empty, and no-CoD conditions against the current and future ground-truth features. The generated CoD captures the future-state structure, whereas the other variants largely remain similar to the current state and fail to reflect the expected future change.

Figure 8: Attention analysis of CoD. We visualize attention from CoD tokens to the current observation. Across human and robot examples, CoD attends to action-relevant regions, including human hands, robot arms, and manipulated objects, highlighting its focus on motion-relevant cues.

Table 9

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Episodes	Frames	Hours	Hz	Camera view	Action representation
YAM	32.2K	76.0M	704.1	30	Top + two wrists	$30\times 14$ D absolute joint
DROID	74.6K	17.8M	328.9	15	Exterior + wrist	$15\times 8$ D absolute joint
RoboCOIN	112.0K	82.3M	755.2	30/50	One or three views	$30/50\times(12/14)$ D absolute EEF
Bridge	53.2K	1.9M	105.2	5	Main	$5\times 7$ D delta EEF
RT-1	87.2K	3.8M	350.6	3	Main	$3\times 7$ D delta EEF
BC-Z	39.4K	5.5M	152.0	10	Main	$10\times 7$ D delta EEF

Appendix

Table 4: Training datasets and native observation/action contracts. K and M denote thousands and millions, respectively.

Setting	VLM SFT	Action SFT	RL
Initialization	Qwen3-VL-2B	VLM SFT	Action SFT
Trainable modules	Language model + future projector	Action expert	Language model
Objective	CoD/VQA CE $+3\mathcal{L}_{\mathrm{future}}$	Masked flow matching	GRPO/GDPO + weak SFT replay
Data	Robot + human motion; auxiliary VQA	Robot trajectories	Robot + human motion
Iterations	150k	50k (5k current-view + 45k history)	2k
Learning rate	VLM $10^{-5}$ ; projector $10^{-4}$	Expert $10^{-4}\!\rightarrow\!5\times 10^{-5}$	$10^{-6}$

Appendix

Table 5: Training configuration . Batch is the effective global batch, and allocations report representative resources used for the runs.

	Color Matching		Cup Hanging		Sorting		Overall
Model	ID	OOD	ID	OOD	ID	OOD	Macro SR	Micro SR
Diffusion Policy	18/30 (60.0)	11/30 (36.7)	9/15 (60.0)	1/15 (6.7)	4/15 (26.7)	1/15 (6.7)	32.8	36.7
MolmoAct2	24/30 (80.0)	8/30 (26.7)	11/15 (73.3)	14/15 (93.3)	12/15 (80.0)	3/15 (20.0)	62.2	60.0
LaWAM	16/30 (53.3)	8/30 (26.7)	9/15 (60.0)	12/15 (80.0)	4/15 (26.7)	4/15 (26.7)	45.6	44.2
ThinkWorld (ours)	30/30 (100.0)	24/30 (80.0)	15/15 (100.0)	15/15 (100.0)	15/15 (100.0)	13/15 (86.7)	94.4	93.3

Appendix

Table 6: Full real-robot success-rate (SR) comparison. Each task is evaluated under in-distribution (ID) and out-of-distribution (OOD) settings; cells report successes/trials with SR (%) in parentheses. Task 1 comprises six ID and six OOD conditions (

5

trials each); Tasks 2 and 3 comprise three of each. Macro SR is the unweighted mean of the six per-setting rates; Micro SR pools all

120

trials.

Group	Task	SR (%)
Google-VM		49.31
Google-VA	Pick Coke can	56.00
	Drawer	33.33
	Move-near	55.83
	Put-in-drawer	2.12
	Micro avg.	46.54

Appendix

Table 7: Out-of-the-box performance on SimplerEnv.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Avg.
OpenVLA-OFT ( Kim et al., 2024 )	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6
$\pi_{0}$ -FAST ( Black et al., 2024a )	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
DreamVLA ( Zhang et al., 2026 )	65.0	40.8	63.5	85.7	82.6	84.9	74.0	69.9
ThinkWorld (Ours)	46.9	70.5	73.5	90.7	90.0	62.7	79.6	71.7

Appendix

Table 8: Success rates (%) on LIBERO-Plus across seven perturbation dimensions. Only methods with published LIBERO-Plus results are listed. Higher is better.

Policy	p50	p95	Wall	Mem.
	(ms)	(ms)	(s)	(GiB)
MolmoAct2	335.07	467.96	18.44	12.11
MolmoAct2-Think	879.37	1014.47	46.76	12.12
ThinkWorld	324.84	333.09	16.31	7.07

Appendix

Table 9: Inference efficiency. Per-call latency percentiles, total wall time, and peak GPU memory, measured at steady state over 50 consecutive eager-mode calls on a single NVIDIA H100 GPU. Lower is better for all metrics.

Figure 10: Real-robot experiment task rollout results. (a) Sample teleoperation demonstrations used for training. (b) In-domain rollouts. (c) Out-of-domain rollouts. Each panel shows qualitative rollout quality for every method, with frames sampled uniformly over an episode and time advancing left to right; the corresponding success rates are reported in Tab. 6 .

Figure 11: Real-robot experiment task rollout results. (a) Sample teleoperation demonstrations used for training. (b) In-domain rollouts. (c) Out-of-domain rollouts. Each panel shows qualitative rollout quality for every method, with frames sampled uniformly over an episode and time advancing left to right; the corresponding success rates are reported in Tab. 6 .

Figure 12: Real-robot experiment task rollout results. (a) Sample teleoperation demonstrations used for training. (b) In-domain rollouts. (c) Out-of-domain rollouts. Each panel shows qualitative rollout quality for every method, with frames sampled uniformly over an episode and time advancing left to right; the corresponding success rates are reported in Tab. 6 .

Figure 13: Real-Robot skill composition sample trajectory and rollout results. (a) Sample atomic robot trajectories for opening the oven (OPEN), placing the left or right object inside (PUT-L, PUT-R), and closing the oven (CLOSE). (b) Human demonstrations that combine these skills into open $\rightarrow$ close and open $\rightarrow$ put-L $\rightarrow$ close sequences. (c) Successful robot rollouts for the corresponding natural-language instructions. Frames in each row progress from left to right.

Figure 14: Real-Robot skill composition sample trajectory and rollout results. (a) Sample atomic robot trajectories for opening the upper or lower drawer (OPEN), placing the left or right object inside (PUT-L, PUT-R), and closing the drawer (CLOSE). (b) Human demonstrations that combine these skills into open $\rightarrow$ put-L $\rightarrow$ close and open $\rightarrow$ put-R $\rightarrow$ close sequences for the upper drawer, and an open $\rightarrow$ put-L $\rightarrow$ close sequence for the lower drawer. (c) Successful robot rollouts for the corresponding natural-language instructions. Frames in each row progress from left to right.

Figure 15: Qualitative future-prediction results. Each row is one episode. The first two columns show the current and ground-truth future RGB observations, and the next two show PCA projections of the ground-truth DINO features for those frames. The remaining five columns show the predicted future DINO features under different CoD conditions: with CoD, shuffled CoD, empty CoD, a CoD taken from a different (donor) episode, and without CoD. All PCA panels share one color mapping.

Figure 16: Qualitative future-prediction results. Each row is one episode. The first two columns show the current and ground-truth future RGB observations, and the next two show PCA projections of the ground-truth DINO features for those frames. The remaining five columns show the predicted future DINO features under different CoD conditions: with CoD, shuffled CoD, empty CoD, a CoD taken from a different (donor) episode, and without CoD. All PCA panels share one color mapping.

Figure 17: Additional attention visualizations. Attention maps from CoD and future-prediction tokens to the input observation. CoD attention concentrates on dynamic, task-relevant regions, while future-prediction attention is more broadly distributed across the scene.

Figure 18: One-second-ahead future predictions during closed-loop SimplerEnv rollouts. For each task, the upper row shows observations actually reached by the policy, while the lower row shows PCA-projected future representations predicted at the preceding replanning horizon. Each column pairs a prediction with the observation at its target time, so their scene structure should correspond. The close qualitative alignment in layout and task progress shows that the predictions track the states encountered during closed-loop execution. The initial frame has no preceding prediction.

Reasoning for VLA Models

Reasoning for vision-language-action (VLA) models connects visual and language understanding to grounded planning and action.

See all 111 papers

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
Jun 16, 2026Tianyi Lu, Hui Zhang, Zijie Diao +8Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
Fudan University · Shanghai Innovation Institue · China Unicom +1
VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning
Mar 15, 2026Chaoyang Wang, Wenrui Bao, Sicheng Gao +5Vision-Language-Action Models Reasoning for VLA Models
University of Central Florida · University of Würzburg · University of Southern California +1
World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models
Jul 30, 2026Xiangcheng Zhang, Yilun DuWorld Models for Robotics World Model-Based Planning
Harvard University

Jun 16, 2026J/K move · Enter open · S save

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation

Tianyi Lu, Hui Zhang, Zijie Diao, Junke Wang+7

Fudan University · Shanghai Innovation Institue · China Unicom+1

Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation VLM Reasoning Vision-Language-Action Models Visually Grounded Reasoning Reasoning for VLA Models

Open paper AskPDF arXiv

Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To address this, existing approaches adopt Chain-of-Thought (CoT) reasoning to enable subgoal decomposition and spatial anticipation. However, those methods lack a unified architecture for effective cross-modal reasoning and fail to explicitly include inverse reasoning ability based on the target state. We argue that manipulation planning naturally decomposes into prediction, anticipating the next visual state, and inverse dynamics, inferring the actions to reach it. Bridging both requires a unified autoregressive architecture that interleaves textual and visual reasoning in a single generation process. We propose ThinkingVLA, a generative model that realizes this decomposition within a unified Mixture-of-Transformers architecture. ThinkingVLA consists of a forward CoT that identifies the immediate subgoal and guides the visual forecasting; the predicted image then serves as the target state, grounding an inverse CoT that reasons about spatial relationships and action intent based on the predicted image; and the final action is generated conditioned on this full reasoning context. Extensive experiments on simulation and real-world benchmarks demonstrate that ThinkingVLA consistently outperforms state-of-the-art baselines, with particularly large gains on long-horizon manipulation tasks.

Abstract

Reasoning for VLA Models

Explore similar work

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation

VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning

World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models