Learning Transferable World-Action Models from Task-Paired Human Videos

Jeongeun Park; Taekyung Kim; Dongyoon Han; Sangdoo Yun

cs.CVOct 9, 2026

Learning Transferable World-Action Models from Task-Paired Human Videos

Authors: Jeongeun Park, Taekyung Kim, Dongyoon Han, Sangdoo Yun

Organizations: NAVER AI Lab · Georgia Institute of Technology

Abstract

World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-robot demonstrations. Analyses before robot post-training show improved motion alignment across embodiments and better action and video prediction on unseen robots, with representations closer to those learned from robot pairs. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across robot embodiments. WATCH outperforms within-video pre-training and co-training baselines, supporting cross-video prediction as a scalable approach to learning transferable world-action representations.

Figures & tables

Figure 1: WATCH overview. (a) Human videos of the same task are paired across collection sources and scenes. (b) Conditioned on the reference video and actions, the model predicts the target’s actions and future frames from its initial frame. The reference’s appearance does not transfer, so what it can pass on is the interaction. (c) After robot post-training, WATCH substantially improves success rates over the base model in both in-context and context-free settings.

Panel of Figure 2 — Figure 2: Where a world-action model looks. Attention to the current frame, grouped by model in the order the text reads them. (a) After robot post-training of the baseline video backbone (Cosmos-Predict2.5), attention focuses on the hand and object in a successful rollout and on the forearm in a failed one ( PnPSinkToCabinet ). (b) The original video model attends broadly to edges and background on both robot frames and human demonstrations, which offer a scalable source of interaction and motion cues for robot learning. (c) On the same frames, our cross-video pre-training shifts attention toward the acting hand and manipulated object.

Figure 3: Overview of WATCH . The reference video and its action trajectory, from a different demonstration of the same task, enter as clean conditioning tokens together with the query’s current frame and proprioceptive state; the query’s action chunk and future frames are the noised tokens the model denoises. Every token is tagged with its slot type and physical time, so the same sequence layout carries over to post-training with a human demonstration as the reference or with the reference slots removed. The loss is the joint velocity-matching objective of § 2.1 , applied to the generated tokens.

Panel of Figure 4 — Figure 4: Sensitivity to the reference demonstration. (a) Action MSE (lower is better) over $137$ held-out episodes using a matched reference, a similar-motion reference from a different collection source, or a different-task reference from the same source. Brackets: Holm-corrected $p$ -values, episode-paired Wilcoxon test. Our model is less sensitive to a change of source than of task, suggesting that our training encourages a focus on task-relevant actions. (b) Action-token attention for the same query (do dishes) with references from the same collection source (top) and a different one (bottom). Even with a reference from a different source, attention concentrates on interactions central to task execution.

Panel of Figure 5 — Figure 5: Representation alignment and motion transfer across embodiments. Action-slot features from layers $10$ – $27$ , with retrieval ranks and regression scores averaged over these layers, on $359$ held-out EgoVerse windows (human) and $396$ RoboMIND windows from three robots (robot). (a, b) Motion retrieval: share of queries whose matched window from a different collection source (a) or robot (b) falls within a normalized rank; a higher curve is better, the diagonal is chance. (c) Held-out embodiment regression: a regressor fit on two embodiments predicts nine future-motion quantities on the third; bars are the mean Spearman correlation over quantities and folds, error bars the standard error over layers.

Panel of Figure 6 — Figure 6: Attention visualizations under different pre-training supervision. All models use cross-video pairing and differ in which reference modalities they receive and predict: actions only (Act only), future frames only (Video only), or both (Act+Video, ours). Joint supervision concentrates attention on the acting hand and the manipulated object on both the action and the video slot.

	Action		Generated video
	MSE $\downarrow$	acc@ $0.1$ $\uparrow$	FVD $\downarrow$	FID $\downarrow$
Original model	–	–	290.7	12.48
Within-video	0.046	55.6	1485.1	53.53
WATCH (Ours)	0.039	68.5	154.9	8.33

Table 1: Evaluation on held-out robot data. End-effector action and video prediction on RoboMIND, unseen by every model; original model is the video model before any action training. Other sets are in § D.2 .

Figure 7: Alignment with a robot-trained model. Match@

1

for human-trained WATCH (green) and the original video model (orange), measured against the robot-trained model on

359

EgoVerse windows (robot windows: Fig. 19 ). Numbers at right: mean over all layers.

Method	L1	L2	L3	Avg.
ICL baselines
Vid2Robot [ 9 ]	90.8	51.0	0.8	47.6
HAND † [ 21 ]	29.0	21.0	1.0	17.0
MimicDroid † [ 10 ]	59.0	44.0	26.0	43.0
VLA baselines
$\pi_{0.5}$ [ 22 ]	79.2	38.3	8.3	41.9

Table 2: In-context post-training on MimicDroid. Success rate (%); Avg. over L1–L3.

\dagger

: reported by prior work under its own evaluation data.

Method	MimicDroid			TableTop-Sim			RoboCasa
Method	Easy	Hard	Avg.	Easy	Hard	Avg.	Kitchen-H50
VLA baselines
$\pi_{0.5}$ [ 22 ]	78.3	29.8	54.1	70.4	34.0	52.2	54.5
GR00T-N1.7 [ 23 ]	79.2	45.8	62.5	66.8	50.8	58.8	46.4
MolmoAct2 [ 29 ]	76.7	28.3	52.5	66.4	39.6	53.0	54.3
World-action models

Table 3: Context-free post-training. Success rate (%) without a reference demonstration; Avg. over the two levels of each benchmark.

Pre-training	In-context								Context-free
	MimicDroid				MimicDroid, $2\times$				MimicDroid			TableTop-Sim			Robo- Casa
	L1	L2	L3	Avg.	L1	L2	L3	Avg.	Easy	Hard	Avg.	Easy	Hard	Avg.	Robo- Casa
None [ 3 ]	75.0	29.4	18.3	40.9	73.3	18.8	0.0	30.7	71.7	26.9	49.3	69.6	49.6	59.6	56.0
Within-video [ 13 , 16 ]	85.8	44.4	23.3	51.2	75.0	7.5	8.3	30.3	88.3	40.8	64.6	71.6	51.6	61.6	58.3
Co-training [ 15 , 7 ]	87.5	58.3	34.2	60.0	88.3	25.2	5.0	39.5	90.0	42.3	66.1	69.2	52.8	61.0	61.7
Same-source top-1	84.2	58.3	33.3	58.6	85.8	26.7	23.3	45.3	90.0	36.7	63.3	69.6	51.2	60.4	56.8

Table 4: Pre-training ablations. Success rate (%) on a fixed Cosmos Policy backbone; rows vary the pre-training objective, data mixture, and reference set (defined in the text): in-context MimicDroid with references at recorded and at

2\times

speed, and context-free MimicDroid, TableTop-Sim, and RoboCasa Kitchen-H50. None is the backbone post-trained directly [ 3 ] .

Panel of Figure 8 — Figure 8: Pre-training supervision. Success rate (%) with post-training fixed at video + action and pre-training supervising nothing ( None ), the video only, the actions only, or both, on (a) MimicDroid in-context, averaged over the four tasks per level with Avg. over L1–L3, and (b) TableTop-Sim context-free.

Panel of Figure 9 — Figure 9: Scaling the pre-training data. MimicDroid success rate (%) against the portion of the paired human video used for pre-training on a fixed Cosmos Policy backbone, from no pre-training ( $0$ ) to all EgoVerse pairs ( $100\%$ ). (a) In-context evaluation. (b) Context-free evaluation. Dashed: within-video pre-training on the $100\%$ EgoVerse training data.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

	Retrieved	Same-class null	Random null
speed-profile correlation $\uparrow$	0.23	$-0.01$	$-0.01$
trajectory-shape cosine $\uparrow$	0.36	0.00	0.00
$\|\log\|$ path-length ratio $\downarrow$	0.69	0.93	0.92
speed off by more than $3\times$ $\downarrow$	22.9%	28.7%	39.4%

Appendix

Table 5: Within a task class, the retrieved reference agrees with the query’s motion. Agreement between a query and its top-ranked cross-source reference on motion-agreement metrics that the pairing score does not directly optimize (the descriptor sees positions, rotations and action increments, but is scored by warped descriptor distance, not by these metrics), against a random partner of the same task class and a random partner. Means over

12{,}000

queries.

Panel of Figure 10 — Figure 10: Query–reference pairs. Three queries from the training pairs, each with its reference from the same collection source and one from a different source, as five frames spanning the $3$ -second window. The source of every clip is named under its row label. (a) Unload dishwasher, a Scale query with a MECKA reference; (b) do dishes, a MECKA query with a Scale reference; (c) bagging groceries, a Scale query with an Aria reference.

Figure 11: A query of the robot analysis set with its nearest matches. Bread in basket on RoboMIND: a Franka query window and the nearest same-task window, by dynamic-time-warping distance on the wrist trajectory, from the same robot and from each of the two other robots, as five frames over the $3$ -second window. Cross-robot pairs like the last two rows are the queries of the robot retrieval test (Fig. 5 b).

Figure 12: The post-trained original video model, more frames. Attention to the current frame of the two PnPSinkToCabinet rollouts of Fig. 2 a, one frame later, from the three cameras. The success (top) attends to the hand at the object, the failure (bottom) to the forearm.

Panel of Figure 13 — Figure 13: Generated video on the held-out sets. One clip per set, (a) Humanoid Everyday, (b) EgoDex, (c) RoboMIND, as eight frames over the $3$ -second generated window: the ground truth, the original video model, within-video pre-training and WATCH .

		Action (end effector)		Generated video
Held-out set	Model	MSE $\downarrow$	acc@ $0.1$ $\uparrow$	FVD $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$
Humanoid Everyday	Original model	–	–	325.7	31.81	0.422
	Within-video	0.077	51.5	610.4	31.22	0.366
	Co-training	0.045	66.2	105.1	12.70	0.267
	WATCH (Ours)	0.023	71.4	137.6	10.01	0.215
EgoDex	Original model	–	–	200.5	8.18	0.243

Appendix

Table 6: Prediction on three held-out sets. Table 1 extended to Humanoid Everyday [ 30 ] , a humanoid teleoperation dataset, and EgoDex, a second human set,

2{,}048

clips per set, with the co-training variant of § C.3 added. Action metrics are on the end-effector subset of the action: MSE in normalization space and accuracy at tolerance

0.1

. The original model generates video only. Best in bold, second underlined.

Panel of Figure 14 — Figure 14: Swapped reference, two queries. Action-token attention of the video backbone (upper block of each query) and ours (lower block) with a reference from the same collection source and from a different one. (a) Bagging groceries: an Aria query, the different-source reference a Scale demonstration. (b) Do dishes, the query of Fig. 4 b: a Scale query, the different-source reference a MECKA demonstration.

Figure 15: Attention on robot and human frames, three models. Attention to the current frame under the original video model (Cosmos-Predict2.5), the same model pre-trained within-video, and ours. Top four rows: side-right camera of GR1 MimicDroid rollouts of the task named on the left. Bottom three rows: three EgoVerse queries, on the query’s current frame.

Figure 16: Task layout across collection sources. t-SNE of layer-

0

action-slot features for

95

human windows of three tasks, each recorded by two of the three sources. Color is the task, in a full and a pale shade for its two sources, and shape is the source. Under the original video model each source forms its own island within a task; under ours the two sources of a task share an island.

Panel of Figure 17 — Figure 17: Linear probes of motion attributes. Layers $10$ – $27$ , as in Fig. 5 ; bars are linear-probe balanced accuracy above each attribute’s chance level, in percentage points, error bars the standard error across layers. (a, b) Three-way direction probes and the four-way grasp probe on human (a) and robot (b) windows. (c, d) Four-way probes: each direction axis is split by sign and, within each sign, at the median magnitude, so the label carries the direction and a coarse size; grasp is the four-state label as before (chance $1/4$ throughout).

Figure 18: Feature space of the human-trained and the robot-trained model on robot windows. Action-slot features of

300

RoboMIND windows, per robot, at layers

12

and

27

: each model is reduced to

10

principal components, the other two are rotated onto the human-video arm’s components (two-fold cross-fit), and the three are embedded jointly with t-SNE, so every window appears three times.

Figure 19: Agreement with the robot-trained model, layer by layer. Procrustes match@

1

between the action-slot features of the robot-data arm and those of the human-video arm (green) or of the original video model (orange) at every DiT block, on (a)

300

RoboMIND windows and (b)

359

held-out EgoVerse windows. Each model’s features are reduced to

50

principal components and scale-normalized; one model is rotated onto the other by an orthogonal Procrustes map fitted on half the windows and applied to the other half, the halves swapping roles so that every window is placed by a map it did not contribute to. The score is the fraction of windows whose nearest neighbor among all

N

reference points is the window itself, so chance is

1/N

0.0033

in (a) and

0.0028

in (b). Shaded: layers

10

–

27

, the band of the summaries, with its mean at the right.

Panel of Figure 20 — Figure 20: Attention of the human-trained and the robot-trained model. Action-token attention under a same-embodiment reference, five points of its span then the query frame. (a) A Franka query, bread in basket, from the left camera for both arms. (b) A MECKA query, sort bins, over the first half of the reference, where the sorting happens.

Figure 21: Downstream transfer under a collection budget. The human-data sweep of Figure 9 (green, blue) and the robot-pairs arm (red), under the same MimicDroid post-training. An arm’s budget is its amount of paired video times the cost of collecting a unit of that video, in units of human video; the ratio between the two is not measured here, so the band sweeps

4\times

16\times

and the line draws

8\times

. Each robot line ends at the largest robot setting we ran. Both arms start at

0

from the same model without this pre-training stage. (a) In-context, L1 to L3; (b) context-free, L1 and L2. Each panel has its own y range.

Method	L1					L2					L3
Method	CLD	Faucet	S2Cab	S2Plate	Avg.	CLD	CRD	S2Cab	S2Plate	Avg.	CLD	Faucet	S2MW	S2Plate	Avg.	Avg.
ICL baselines
Vid2Robot [ 9 ]	100.0	100.0	63.3	100.0	90.8	53.3	0.0	63.3	87.5	51.0	0.0	0.0	0.0	3.3	0.8	47.6
VLA baselines
$\pi_{0.5}$ [ 22 ]	100.0	100.0	80.0	36.7	79.2	43.3	0.0	70.0	40.0	38.3	33.3	0.0	0.0	0.0	8.3	41.9
GR00T-N1.7 [ 23 ]	100.0	100.0	33.3	80.0	78.3	100.0	0.0	66.7	70.0	59.2	33.3	6.7	0.0	0.0	10.0	49.2

Appendix

Table 7: In-context MimicDroid, per task. Success rate (%) of every model of Table 2 and Table 4 on the four tasks of each level; Avg. is the mean over the tasks of a level, and the last column the mean over L1–L3. Best in bold, second best underlined. Task abbreviations: CLD/CRD = Close Left/Right CabinetDoor, Faucet = TurnOnFaucet, S2Cab = PnPSinkToCabinet, S2Plate = PnPSinkToRightCounterPlate, S2MW = PnPSinkToMicrowaveTop.

Method	L1					L2
Method	CLD	Faucet	S2Cab	S2Plate	Avg.	CLD	CRD	S2Cab	S2Plate	Avg.	Avg.
VLA baselines
$\pi_{0.5}$ [ 22 ]	100.0	100.0	73.3	40.0	78.3	10.0	0.0	56.7	52.5	29.8	54.1
GR00T-N1.7 [ 23 ]	100.0	100.0	43.3	73.3	79.2	76.7	0.0	56.7	50.0	45.8	62.5
MolmoAct2 [ 29 ]	100.0	100.0	26.7	80.0	76.7	70.0	0.0	3.3	40.0	28.3	52.5
Cosmos-Predict2.5-2B

Appendix

Table 8: Context-free MimicDroid, per task. Success rate (%) of every model of Table 3 and Table 4 on the four L1 tasks and the four L2 tasks; Avg. is the mean over the tasks of a level, and the last column the mean over the two levels. Best in bold, second best underlined. Task abbreviations: CLD/CRD = Close Left/Right CabinetDoor, Faucet = TurnOnFaucet, S2Cab = PnPSinkToCabinet, S2Plate = PnPSinkToRightCounterPlate, S2MW = PnPSinkToMicrowaveTop.

Method	Easy (seen)						Hard (unseen)
Method	handover box	lift box	shoes table	dish drainer	box into pot	Avg.	handover box	lift box	shoes table	dish drainer	box into pot	Avg.	Avg.
VLA baselines
$\pi_{0.5}$ [ 22 ]	66.0	42.0	80.0	90.0	74.0	70.4	48.0	0.0	44.0	56.0	22.0	34.0	52.2
GR00T-N1.7 [ 23 ]	62.0	30.0	84.0	84.0	74.0	66.8	56.0	14.0	82.0	76.0	26.0	50.8	58.8
MolmoAct2 [ 29 ] )	70.0	32.0	70.0	92.0	68.0	66.4	20.0	22.0	42.0	86.0	28.0	39.6	53.0
Cosmos-Predict2.5-2B

Appendix

Table 9: TableTop-Sim, per task. Success rate (%) of every model of Table 3 and Table 4 on the five tasks of the easy (seen) and the hard (unseen) split under context-free post-training; Avg. is the mean over the tasks of a split, and the last column the mean over the two splits. Best in bold, second best underlined.

Panel of Figure 22 — Figure 22: Supervision ablation, all configurations. Success rate (%). Hue: pre-training target (none, action, video + action); pale: action-only post-training, full: video + action. (a) MimicDroid, in-context, the reference given as actions only under action-only post-training; (b) TableTop-Sim, context-free.

Panel of Figure 23 — Figure 23: In-context rollouts on MimicDroid. Per task, Cosmos Policy post-trained in context without pre-training (top) and WATCH (bottom) at the same five time steps over the WATCH episode; the failing episode runs on to the time limit. The inset is the retrieved reference frame the policy is conditioned on. Each frame uses whichever of the front, left and right cameras shows the motion best; in (b) the two rows start from different initial states.

Panel of Figure 24 — Figure 24: Reading a $2\times$ -speed demonstration, chunk by chunk. The same CloseLeftCabinetDoor episode (L3, camera behind the robot) under (a) Cosmos Policy without pre-training, which fails, and (b) WATCH , which succeeds. Rows are $1.6$ -second action chunks from $1.6$ to $9.6$ s: four reference frames at $0.4$ -s steps, replayed at $2\times$ ; generated , the frame predicted for the chunk; robot , the frame reached at its end. The demonstrated hand closes the door within a chunk; without pre-training the generated frames never show it closing and the arm stalls, WATCH spreads the motion over its own chunks and finishes.

Panel of Figure 25 — Figure 25: A two-stage task under a $2\times$ -speed demonstration. The same PnPSinkToCabinet episode (L2) under (a) Cosmos Policy without pre-training, which fails, and (b) WATCH , which succeeds; rows as in Fig. 24 : chunks $1$ – $3$ ( $0$ to $4.8$ s) from the left camera, where the grasp is visible, then chunks $5$ and $7$ from the camera behind the robot, which shows the transport. Both grasp the object; WATCH places it in the cabinet, whereas without pre-training the arm rises toward the cabinet and stays there until the episode times out.

Panel of Figure 26 — Figure 26: Context-free rollouts on TableTop-Sim. Per task, the same episode under Cosmos Policy without pre-training (top) and WATCH (bottom) at the same time steps over the WATCH episode; the failing episode runs on to the time limit. Without pre-training, the plate is lifted and dropped (a), the small box reached but not lifted (b), the arms hover around the box (c), the box is dropped before the pass (d), and the shoes are carried to the table but not released (e).

World Model Learning

World model learning builds predictive models of how environments change under actions to support planning, simulation, and decision-making.

See all 418 papers

Jun 28, 2026

Learning Transferable Dynamics Priors from Action to World Modeling

We study action-conditioned world modeling as a scalable way to learn transferable dynamics priors for robot learning. By pretraining a model to predict how actions drive visual scene evolution, the resulting world model captures reusable interaction dynamics beyond appearance-level video generation. Concretely, we pretrain a multi-view interactive base diffusion world model, A2World, on large-scale robot manipulation data with real action annotations. We validate the learned dynamics priors from two complementary perspectives. First, we adapt A2World into a task- or scene-specialized real-world simulator, A2World-sim, whose long-horizon rollouts support simulator-based policy evaluation and scalable what-if analysis by replacing real-robot rollouts with world model rollouts. Second, starting from the same pretrained weights, we adapt A2World into a video-action joint prediction model, A2World-policy, that predicts actions under visual and instruction conditioning. Experiments across simulation benchmarks and real-robot settings demonstrate that action-conditioned world model pretraining yields transferable dynamics priors that benefit both simulator-centric and policy-centric robot learning.

Ze Huang, Jiahui Zhang, Hairuo Liu +3

School of Data Science, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China +1

Action-Conditioned Video Generation Robot Policy Learning World Model Learning World Models for Robotics Action-Conditioned World Models Diffusion Models

Ask PDF

Sep 4, 2026

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

World-action models (WAM) predict future states to guide robot actions, enabling learning from both action-free video and action-labeled interaction. Most inherit pretrained video generators, leaving WAM pretraining and scaling underexplored. We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all initialized from scratch on manipulation data. It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM). CoAE retains action- and instruction-relevant information under aggressive compression, while SVP produces a complete future state in one differentiable pass, so visual planning and inverse dynamics can be pretrained separately on complementary data. The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action. We evaluate pretrained checkpoints directly, without per-task fine-tuning, on 100 tasks across 20 manipulation skill groups with held-out scenes, backgrounds, lighting, and object instances. Scaling co-training data from 300 to 30,000 hours raises success from 17.1% to 44.1% on G1-OP and from 13.4% to 31.1% on G2-90D; despite comprising less than 2% of the co-training data, G2-90D improves by 17.7 points, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage strongly correlates with zero-shot out-of-distribution (OOD) success (Pearson r=0.80; Spearman rho=0.85). Under the same protocol, the model grounds object, color, shape, and position references in at least 90% of trials and follows explicit instructions even when they conflict with an already-committed behavior or a conventional scene association.

Renhang Liu, Wenzhi Zhao, Zhuo Yang +42

World Models for Robotics Robot Foundation Models Zero-Shot Robotic Manipulation Language-Conditioned Robot Manipulation

Ask PDF

Oct 27, 2026

ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning

Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action thought as a Chain-of-Dynamics (CoD), a sequence of discrete motion tokens that captures the intermediate dynamics connecting the current world to a future state and serves as a common action representation across embodiments. We first initialize CoD with action-labeled demonstrations, then predict the future induced by each CoD and reinforce those whose predictions better match the observed outcome. This enables videos without action annotations to provide direct supervision from observed state transitions, allowing human and robot experience to refine the same reasoning space. Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments.

Chaehyun Kim, Chi-Pin Huang, Jisoo Kim +4

NVIDIA · KAIST

Reasoning for VLA Models Robot Learning from Video World Models for VLA Models Cross-Embodiment Robot Learning RL for VLMs

Ask PDF

Abstract

World Model Learning

Explore similar work

Learning Transferable Dynamics Priors from Action to World Modeling

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning