Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
Authors: Kuan Zhang, Yukun Chen, Zhihao Yang, Yue Su, Xiangnan Wu, Zirong Chen, Run Luo, Jinkun Hou, Tao Tan, Yinhe Zheng, Yiming Li
Organizations: miHoYo Honkai AI R&D Team · College AI, Tsinghua University · University of Chinese Academy of Sciences · MMLab, The University of Hong Kong · National University of Singapore · Peking University
Scaling data, diversity and model capacity has improved embodied learning in physical environments, where diverse experiences remain grounded in shared physical laws. Whether these gains extend to virtual worlds with different visual styles, dynamics, controls, and objectives remains unclear. We investigate this question through a controlled study of continued pre-training (CPT) across three axes: data volume, diversity, and model size. Our 120 training configurations span four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five mixtures ranging from Minecraft-only data to diverse multi-world corpora. We assess performance and generalization using three-level validation losses and gameplay benchmarks from in-corpus worlds to unseen worlds. Our results show: (1) more data improves in-corpus performance, diversity improves transfer to related worlds with limited gains in distant worlds, and model size improves data efficiency but generalizes worse to distant worlds under single-world training; (2) we observe an early interface phase with broad cross-world benefits and a later world phase with gains narrowed to in-corpus and related worlds, and fit an empirical scaling law relating validation loss to data volume, diversity and model size; and (3) our models are competitive with Minecraft specialists and frontier models in 3D worlds, and perform best in real-time settings. However, the limits of imitation and persistent gaps in distant worlds suggest that building generalist embodied agents requires richer training stages beyond continued pre-training.
Figures & tables
Figure 1: One physical world versus diverse virtual worlds. Left: Agents learn from varied physical scenes sharing common dynamics, where they can scale and generalize with broader experience ( Wang et al., 2026a ) . Right: Agents play diverse games with varying dynamics. We want to investigate the question that whether continued pre-training scales and generalizes in virtual worlds.
Figure 2: Adaptation to the keyboard–mouse interface during CPT. Smoothed training losses for key, mouse, and format tokens, and eight-direction mouse prediction accuracy for three model capacities trained on mixtures S1 and S4.
Figure 3: Loss scaling across data volume, model size and data diversity. Rows report the smoothed training loss and three-level validation loss. Columns 1–4 denote the 0.8B-27B backbones; colors denote the training mixtures S1–S5; markers denote the six checkpoints from 50 to 1,600 hours. The base model loss is annotated in each validation panel. All plots in a given row shares a common y-axis scale. Column 5 plots the loss predicted by the fitted scaling law (Section 4.4 ) against the observed loss.
Figure 4: Benchmark performance across data budget, model size and data diversity. Rows report MCU, ViZDoom, OmniGameArena, and GameWorld performance. Columns denote the six checkpoints from 50 to 1,600 hours. Colors denote S1-S5 and the base-model reference, while each curve spans the 0.8B-27B backbones. All plots in a given row shares a common y-axis scale.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Mixture
Worlds
Minecraft share
Largest share
D=expH
Inverse Simpson
S1
1
100%
100%
1.0
1.0
S2
80
51.4%
51.4%
14.5
3.7
S3
80
27.1%
27.1%
35.2
11.9
S4
80
2.8%
4.2%
60.1
52.6
S5
79
0%
4.3%
58.7
51.0
Appendix
Table 3: Mixture diversity of S1–S5. Shares are computed over gameplay segments after cross-source aliases are merged into canonical world labels. D is the effective number of worlds (Hill number of order one); the inverse Simpson index emphasizes dominant worlds.
Mixture
Samples
Samples w/ rationale
Rationales / sample
Reasoning turns
Tokens / rationale
Reasoning tokens / sample
S1
1.50M
74.0%
1.08
5.40%
65.0
70.2
S2
1.50M
75.8%
1.14
5.69%
64.5
73.4
S3
1.50M
76.7%
1.17
5.83%
65.3
76.1
S4
1.50M
77.6%
1.19
5.96%
65.4
78.0
S5
1.50M
77.6%
1.19
5.97%
64.6
77.1
Appendix
Table 6: Sparse reasoning density in the S1–S5 training mixtures. Counts are from a full scan of the training parquets. A rationale is a non-empty <think> span attached to an assistant turn. Token lengths are measured with the Qwen3.5 tokenizer on 13k–19k sampled rationales per mixture.
Figure 5: A training window with sparse reasoning. Five consecutive 200 ms turns from an Apex Legends recording in S4. Each turn begins with the current frame. Only ot and ot+4 carry a GPT-5.6-Luna rationale; the other three turns emit an empty <think> span and act directly. Every action packet contains a mouse displacement (Δx,Δy) and the keys held during six 33 ms sub-steps; key icons show the distinct keys in each packet. Image tokens receive no loss, while thought and action tokens are trained autoregressively. Rationales are abridged excerpts of the stored annotations.
Figure 6: Frontier progress on the four benchmarks against release date. Circles: GPT series; squares: Qwen series; stars: GameScaling S3 models at 1,600 h, all under time-paused interaction.
Figure 7: Loss scaling against model size. Rows: training loss at the checkpoint, and held-out validation loss on Minecraft, on the game-balanced in-corpus set, and on the out-of-distribution set; columns: 50, 100, 200, 400, 800, and 1,600 gameplay hours; colors: mixtures S1–S5.
Figure 8: Benchmark scaling against gameplay hours under time-paused interaction. Rows: MCU, ViZDoom, OGA, and GameWorld (all in %); columns: 0.8B, 2B, 9B, and 27B backbones; points span 50 to 1,600 gameplay hours; colors: mixtures S1–S5; dashed gray: the untrained Qwen3.5 backbone.
Figure 9: Test-time scaling on MCU. Cumulative success rate of GameScaling-0.8B-S1 and GameScaling-9B-S1 (1,600 h, solid) and their untrained Qwen3.5 backbones (dashed) on the 30 MCU tasks as a function of the maximum steps allowed, on a logarithmic axis. The standard protocol uses 600 steps.
Model
Params
Latency (s/step)
Output tokens/step
Gemini-3.8-Flash
∼ T
7.75
444
GPT-5.6-Sol
∼ T
5.57
123
GPT-5.6-Luna
∼ T
5.84
144
Claude-Opus-5.0
∼ T
8.59
183
GLM-5.3-Flash
320B
23.66
1,027
DeepSeek-V4-Flash-Vision
305B
3.97
659
Appendix
Table 9: Per-step inference latency and output length of the Table 2 models, averaged over MCU, ViZDoom, OGA and GameWorld. Latency is the round trip of the model request, excluding screenshot capture and client-side image encoding; frontier models are called through the provider API, and GameScaling (S3, 1,600 h) runs under real-time interaction on the deployment of Table 8 . Output length is the completion_tokens reported by the server, covering hidden reasoning, visible <think> content, and the action.
Loss
Law
Conditions
n
R2
RMSE
Minecraft
Chinchilla, D=T (total hours)
S1–S3
72
0.925
0.0024
Minecraft
Chinchilla, D=rT (Minecraft hours)
S1–S3
72
0.959
0.0018
Minecraft
Two-phase, D=rT and T
S1–S3
72
0.989
0.0009
Minecraft
Eq. ( 3 ), ρ=r
S1–S5
120
0.982
0.0020
In-corpus
Chinchilla, D=T
S2–S5
96
0.991
0.0006
In-corpus
Chinchilla, D=(1−r)T (pool hours)
S2–S5
96
0.955
0.0014
Appendix
Table 11: Restricted scaling-law baselines against the three-factor law. Each baseline drops the share ρ and keeps only N and one data variable D (the two-factor Chinchilla law); “two-phase” adds a second data term in total hours. n is the number of fitted conditions.
Figure 10: Three-factor laws with prediction bands. Rows are the training loss, the Minecraft validation loss and the in-corpus validation loss; columns are the mixtures S1–S5; colors are model sizes. The horizontal axis is gameplay hours, from 100 h for the training loss and from 50 h for the validation losses. Points are measurements, lines are the single fitted law of each row evaluated at that panel’s (N,r) , and shaded bands are the 5–95% range of the law over B=300 stratified case-bootstrap replicates. The last column is predicted against observed loss; per-panel RMSE is printed.
Study
Experiments covered
Tokens
FLOPs
D-CPT Law [ Que et al., 2024 ]
0.5–4B; 6 domains; 9 mixtures (main grid)
4.2×1012†
5.3×1022†
CMR Scaling Law [ Gu et al., 2024 ]
0.46–3.1B; 29 Finance and Academic Papers runs
5.8×1011†
4.7×1021†
When Data Is Scarce [ Wu et al., 2026 ]
Up to 1.92B dense-equivalent; 500 fitting runs
∼3.3×1012‡
∼1.2×1022‡
GameScaling
0.8–27B; 120 conditions; 6 budgets, 5 mixtures
3.5×1011
2.2×1022
Appendix
Table 15: Training scale of language-model studies and GameScaling. Budgets cover the experiments listed, excluding original backbone pre-training for CPT. Related-work budgets are estimates; GameScaling uses the reported totals in Section 3 .
Table 16: Scope of GUI, robotics, and virtual-world scaling studies. GameScaling cost covers the full grid; other costs are per run. † : time estimate at unchanged recipe and throughput. Data use the original units; GameScaling hours include warmup, stable training, and decay.
Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck. Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment, yet their scalability is limited by high collection cost, acquisition difficulty, and low behavioral and environmental diversity. These limitations have sparked interest in egocentric human video as a scalable, substantially lower-cost, and more diverse alternative for embodied model pretraining. However, its effectiveness compared to teleoperated real-robot data remains underexplored. To address this question, we conduct a systematic study comparing egocentric human video and teleoperated real-robot trajectories as pretraining data sources for embodied foundation models, under fixed post-training and validation protocols. Surprisingly, we find that egocentric data, when processed through a carefully designed filtering and labeling pipeline, is not merely a viable substitute for model pretraining but can lead to superior performance. With the same amount of pretraining data, models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction, as well as 52.5% and 90% higher success rates on in-distribution and out-of-distribution real-robot task execution, respectively. This finding verifies a scalable paradigm for embodied foundation models: pretrain on egocentric human video to learn diverse world representations, then adapt with a small amount of labeled real-robot data for action-space alignment. We hope this study encourages broader exploration of egocentric data and offers guidance for data quality assessment before costly robot data collection.
Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general intelligence. Yet most existing work remains confined to specific tasks in a single scene and transfers poorly to new environments, leaving general embodied agents for open-world settings under-explored. We refer to such agents as Embodied World Agents (EWAs), whose defining property is the ability to understand diverse visual environments and execute general actions in open-world conditions. To advance EWA research, we study open-world embodied navigation-planning and release EmbodiedWorld-200K, a large-scale dataset of over 200K samples spanning diverse scenes. Each example is equipped with multi-level annotations, ranging from low-level camera-motion trajectories to high-level semantic instructions. The dataset also provides unified evaluation splits and task-specific metrics, supporting offline pretraining and fair evaluation of EWAs. Beyond offline supervision, a defining characteristic of an EWA is its interaction with the environment; training solely on offline data is therefore insufficient, especially for generalizing to scenes not covered in the training distribution. Building dedicated real interactive environments, however, is often prohibitively expensive. To address this, we propose a novel training scheme that employs a video world model as a low-cost environment proxy and performs interactive reinforcement-learning (RL) post-training of the EWA within it. Empirically, pretraining on EmbodiedWorld-200K substantially improves the open-world embodied planning ability of the EWA, and further RL post-training inside a world model yields clear gains on out-of-distribution scenes. The dataset, baseline checkpoints, and training and evaluation code are available at https://xiaokunfeng.github.io/EmbodiedWorld-200K/.
Web agents require massive trajectories to generalize, yet real-world training is constrained by network latency, rate limits, and safety risks. We introduce WebWorld series, the first open-web simulator trained at scale. While existing simulators are restricted to closed environments with thousands of trajectories, WebWorld leverages a scalable data pipeline to train on 1M+ open-web interactions, supporting reasoning, multi-format data, and long-horizon simulations of 30+ steps. For intrinsic evaluation, we introduce WebWorld-Bench with dual metrics spanning nine dimensions, where WebWorld achieves simulation performance comparable to Gemini-3-Pro. For extrinsic evaluation, Qwen3-14B trained on WebWorld-synthesized trajectories improves by +9.2% on WebArena, reaching performance comparable to GPT-4o. WebWorld enables effective inference-time search, outperforming GPT-5 as a world model. Beyond web simulation, WebWorld exhibits cross-domain generalization to code, GUI, and game environments, providing a replicable recipe for world model construction.