Language Models as AI Research World Models

Zijun Wang; Zewen Liu; Minhua Lin; Zhaotian Weng; Zhan Shi; Bing He; Yisi Sang; Dakuo Wang

cs.CLOct 8, 2026

Language Models as AI Research World Models

Authors: Zijun Wang, Zewen Liu, Minhua Lin, Zhaotian Weng, Zhan Shi, Bing He, Yisi Sang, Dakuo Wang, +5 more

Organizations: Amazon · UC Santa Cruz · Emory University · Pennsylvania State University · UC Santa Barbara

Abstract

AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet their ability to propose experiments outpaces their capacity to execute them in real environments, making outcome prediction a key capability for sustained self-improvement under limited experimental budgets. We investigate language models as Research World Models (RWMs), which predict the outcomes of candidate interventions across research environments. Our evaluation draws on over 2,600 experimental records from nine research environments spanning pretraining, post-training, and inference, representing more than 171,000 H100 GPU-hours of experimentation. Research knowledge acquired from real experimental experience improves RWM predictions of unseen interventions within the same environment (Spearman +0.27), and can be reused across environments. For example, using only pretraining experience from OLMo3, Marin, and Nanochat, an RWM reduces selection regret in the Qwen3 environment by 78% compared with zero-experience setting. These benefits extend to multi-round Autoresearch under a fixed selection budget: RWMs with in-env and cross-env research knowledge increase the best gain achieved by 15.8% and 11.6%, respectively. Ablations across 13 language models used as RWMs show that adding research knowledge can improve intervention ranking more than changing models or increasing reasoning effort alone. These findings support language models as RWMs and motivate accumulating experimental data for future RWM training.

Figures & tables

Figure 1: Large language models as Research World Models. Given a research environment $E_{\tau}$ and candidate interventions $a$ , an RWM predicts gains $\hat{g}$ before execution, so that only the most promising interventions consume experimental budget. Executed experiments become records $(E_{\tau_{i}},a_{i},g_{i})$ that form history $H$ for subsequent predictions. Records may come from the same environment or from other environments, allowing the same RWM to serve a target environment that has no records of its own.

Research Env.	Model / System	Eval metric	Records	H100 GPU-hours	In-env	Cross-env
Pretraining
OLMo3-100M	OLMo3-100M	mean BPB $\downarrow$	480	8,572	$\checkmark$	$\checkmark$
Marin	Marin-153M	mean BPB $\downarrow$	132	12,101	—	$\checkmark$
Nanochat	Nanochat-149M	mean BPB $\downarrow$	132	2,584	—	$\checkmark$
Qwen3	Qwen3-153M	mean BPB $\downarrow$	132	15,447	—	$\checkmark$
OLMo3-190M	OLMo3-190M	mean BPB $\downarrow$	132	5,582	—	$\checkmark$

Table 1: Research environments and experimental scale. In-env and Cross-env indicate prediction-evaluation coverage.

			MAE $\downarrow$				Spearman $\uparrow$
Environment (unit)	History	Test	$W_{0}$	kNN	ridge	$W_{\mathrm{in-env}}$	$W_{0}$	kNN	ridge	$W_{\mathrm{in-env}}$
OLMo3-100M (BPB)	320	160	2.93e-2	2.58e-2	3.13e-2	2.03e-2	0.62	0.78	0.67	0.90
Diffusion (BPB)	536	125	1.95e-2	1.58e-2	1.99e-2	8.77e-3	0.46	0.49	0.50	0.78
Math distillation (pp)	180	100	1.58	1.15	1.11	0.91	0.43	0.61	0.62	0.70
Code RL (pp)	34	115	5.98	5.45	5.02	4.17	0.48	0.29	0.48	0.62
Inference optimization (tokens/s)	457	98	13.19	13.18	12.73	1.34	0.54	0.62	0.72	0.88

Table 2: Within-environment prediction performance. Each environment uses a single fixed split. History denotes context records supplied to

W_{\mathrm{in-env}}

; Test denotes the number of held-out candidates.

	Spearman $\uparrow$				MAE (BPB) $\downarrow$
Target Environment	$W_{0}$	kNN	ridge	$W_{\mathrm{cross-env}}$	$W_{0}$	kNN	ridge	$W_{\mathrm{cross-env}}$
OLMo3-100M	0.69	0.56	0.26	0.83	1.11e-2	1.18e-2	1.65e-2	9.28e-3
OLMo3-190M	0.72	0.43	0.26	0.78	7.94e-3	1.08e-2	1.19e-2	8.56e-3
Qwen3	0.74	0.71	0.40	0.82	9.20e-3	1.13e-2	1.69e-2	7.80e-3
Marin	0.42	0.28	0.43	0.50	9.35e-3	1.13e-2	1.03e-2	8.18e-3
Nanochat	0.57	$-0.17$	$-0.12$	0.71	2.79e-2	3.61e-2	3.57e-2	2.70e-2

Table 3: Cross-environment prediction performance. Macro is the equal-weight mean over target environments.

	Regret@3 $\downarrow$					NDCG@3 $\uparrow$
Environment (unit)	Random	$W_{0}$	kNN	ridge	$W_{\mathrm{in-env}}$	Random	$W_{0}$	kNN	ridge	$W_{\mathrm{in-env}}$
OLMo3-100M (BPB)	1.32e-2	2.80e-3	1.81e-3	2.14e-3	7.00e-4	0.28	0.70	0.78	0.72	0.84
Diffusion (BPB)	1.09e-2	3.14e-3	1.79e-3	5.47e-3	4.70e-4	0.16	0.51	0.62	0.32	0.87
Math distillation (pp)	0.87	0.48	0.41	0.45	0.29	0.20	0.33	0.38	0.34	0.47
Code RL (pp)	4.16	2.20	2.12	1.54	0.83	0.42	0.59	0.64	0.73	0.81
Inference optimization (tokens/s)	4.31	3.60	1.62	2.09	2.53	0.69	0.86	0.93	0.92	0.93

Table 4: Mean within-environment experiment-selection performance. Metrics are averaged over 200 shared random partitions of each environment’s fixed test panel, selecting three candidates from each pool of 16.

	Regret@3 (BPB) $\downarrow$					NDCG@3 $\uparrow$
Target	Random	$W_{0}$	kNN	ridge	$W_{\mathrm{cross-env}}$	Random	$W_{0}$	kNN	ridge	$W_{\mathrm{cross-env}}$
OLMo3-100M	1.31e-2	4.27e-3	4.74e-3	4.84e-3	2.24e-3	0.26	0.65	0.66	0.66	0.84
OLMo3-190M	8.42e-3	1.64e-3	1.60e-3	1.96e-3	5.50e-4	0.24	0.68	0.66	0.64	0.81
Qwen3	1.51e-2	9.40e-4	3.50e-4	1.72e-3	2.10e-4	0.27	0.77	0.85	0.73	0.91
Marin	1.07e-2	7.40e-4	1.10e-3	8.40e-4	7.90e-4	0.18	0.74	0.61	0.67	0.70
Nanochat	8.27e-3	4.94e-3	1.53e-2	1.51e-2	2.22e-3	0.14	0.32	0.12	0.06	0.43

Table 5: Mean cross-environment experiment-selection performance. Metrics are averaged over 200 shared random partitions of each target’s fixed test panel, selecting three candidates from each pool of 16. Macro is the equal-weight mean over target environments.

Figure 2: Multi-round Autoresearch on OLMo3-100M. Means over five Autoresearch evaluations, each with eight rounds. (a) Best-so-far gain (BPB). (b) nAULC (%), computed by normalizing each evaluation’s curve area by that of its matched Arrival Oracle and then averaging across evaluations.

Figure 3: RWM rankings on OLMo3-100M across (a) backbones and (b) Claude Opus 5 reasoning-effort settings.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Block	Contents
Environment	Model and data configuration, resource budget, reference recipe and measurements, evaluation suite, and gain definition.
Intervention	Materialized code diff or configuration override and intervention direction.
Outcome	Raw evaluation measurements and reference-relative gain when defined by the scoring protocol.
Metadata	Execution status and random seed.

Appendix

Table 6: Fields retained in experimental records.

Target	vs. $W_{0}$	vs. kNN	vs. ridge
OLMo3-100M	$+0.14$ $[+0.02,\,+0.28]$	$+0.27$ $[+0.04,\,+0.58]$	$+0.57$ $[+0.23,\,+0.93]$
OLMo3-190M	$+0.06$ $[-0.07,\,+0.18]$	$+0.35$ $[+0.13,\,+0.61]$	$+0.52$ $[+0.22,\,+0.83]$
Qwen3	$+0.08$ $[\text{+2.00e-3},\,+0.17]$	$+0.11$ $[\text{+7.00e-3},\,+0.23]$	$+0.43$ $[+0.20,\,+0.66]$
Marin	$+0.08$ $[-0.05,\,+0.24]$	$+0.22$ $[+0.07,\,+0.40]$	$+0.07$ $[-0.25,\,+0.37]$
Nanochat	$+0.15$ $[+0.04,\,+0.27]$	$+0.89$ $[+0.62,\,+1.13]$	$+0.84$ $[+0.57,\,+1.07]$

Appendix

Table 7: Cross-environment Spearman differences:

W_{\mathrm{cross-env}}

minus each comparator. Brackets contain pointwise 95% paired bootstrap confidence intervals.

Explore similar work

Aug 12, 2026cs.LG

Scaling Automatic Research Agents via World Models

Automating empirical research is a long-standing direction of AI. Recent automatic research (AutoResearch) agents bring this goal within reach, as modern LLMs show the capability to independently implement solutions and learn from the execution outcomes. Behind these gains, post-training (especially RL) plays a central role. In this paper, we identify a fundamental tension when scaling RL for these agents: the two components of every AutoResearch trajectory (agent generation and environment execution) scale in very different manners, since all generation shares compute through batching, while each execution occupies its exclusive sandbox and real machine time. As a result, the environment execution dominates the training cost and becomes the bottleneck as trajectories grow. To resolve this tension, we propose World Model RL (WMRL), which replaces environment execution with a world model to remove this bottleneck. Additionally, the world model can be imperfect, as its rewards are corrupted by bias and noise. Therefore, we further equip WMRL with two mitigations, Online Debiasing and Inverse-Variance Denoising, which offset the bias and suppress the noise respectively. Theoretically, we prove that both mitigations of WMRL strictly improve the convergence guarantee. Empirically, WMRL accelerates training by 3-4x on various tasks at different agent scales, while exceeding the performance of standard RL baselines. Moreover, our post-trained 4B and 9B agents outperform much larger open-weight agents of 48B and 120B on held-out benchmarks. Beyond AutoResearch, WMRL also transfers to post-training embodied VLA policies, which demonstrates the generalizability of our method.

Xiyuan Yang, Sheikh Sarwar, Jingru Cheng +8

University of Illinois Urbana-Champaign · Amazon

World Model Learning RL Post-Training RL for LLM Agents Autonomous Research Agents Model-Based RL

Ask PDF

Jun 23, 2026cs.CL

Qwen-AgentWorld: Language World Models for General Agents

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

Yuxin Zuo, Zikai Xiao, Li Sheng +30

Qwen Team

LLM World Models Agentic RL World Model Learning World Model Evaluation

Ask PDF

Jul 20, 2026cs.AI

AutoWorldModel-Bench: A State-Centric Benchmark for Automated World-Model Research

World modeling is an unsettled field: architectures, training objectives, and state representations interact in complex ways, and no single recipe dominates across environments. This makes it an ideal testbed for AI coding agents acting as autonomous researchers--a setting in which the improvement direction is not specified in advance, unlike the engineering-to-spec tasks that dominate current agent benchmarks. We introduce AutoWorldModel-Bench, a closed-loop benchmark in which frontier coding agents autonomously improve a provided world-model starter under a fixed compute budget. The benchmark spans eight game environments under a unified structured-state representation--ground-truth entity state extracted from each game and consumed through a shared tensor format--which isolates dynamics modeling from perception and enables minutes-per-run iteration. Across 64 sessions, Codex-5.4 and Claude Opus 4.6 improve their starter on 63; in 91% of sessions the winning edit is a non-trivial research-style modification--a new objective, representation, rollout procedure, or architectural change--rather than a hyperparameter tweak. Our benchmark offers a setting in which frontier coding agents can be evaluated on open-ended research rather than engineering-to-spec problems.

Marjan Moodi, Xuankang Zhu, Fernando De Mesentier Silva +2

Electronic Arts · Simon Fraser University

World Model Learning AI Agent Benchmarks World Model Evaluation Autonomous Research Agents

Ask PDF