From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation

Quanyu Long; Xiao Chen; Jianda Chen; Haozhen Zhang; Qisheng Hu; Jianzhu Bao; Wenya Wang

cs.CLOct 5, 2026

From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation

Authors: Quanyu Long, Xiao Chen, Jianda Chen, Haozhen Zhang, Qisheng Hu, Jianzhu Bao, Wenya Wang

Organizations: Nanyang Technological University · The Hong Kong Polytechnic University

Abstract

Realistic environment replicas are increasingly valuable for training and evaluating LLM agents, yet the original systems may be inaccessible or impractical to reproduce. We explore agentic language world modeling: rather than rebuilding an executable environment, a world model agent serves as the environment for a task agent and supports faithful and stateful simulation. We instantiate this paradigm with Trace2Env, a learning-free framework for settings where the original system is unavailable but historical interaction traces remain accessible. Trace2Env reconstructs these traces into a reusable environment worldbook containing environment schemas, grounded evidence, and induced behavioral knowledge. At runtime, the world model agent actively consults the worldbook together with persistent episodic state to infer each action's observation and lasting state effects. Across nine environments, Trace2Env improves both next-observation fidelity and long-horizon interaction consistency over conventional prompt-based LWMs. In multi-turn interaction, task agent actions generated against Trace2Env remain valid more often when replayed in the real environment, indicating that its simulated dynamics better preserve the consequences of earlier actions across successive turns. These results establish agentic language world modeling as an alternative direction for building realistic environment replicas without reconstructing the original executable system.

Figures & tables

Figure 1 : From interaction traces to an agent-operated simulated environment. Trace2Env reconstructs traces into a reusable environment worldbook containing schemas, grounded evidence, and induced behavioral knowledge. During simulation, a task agent supplies actions, while a world model agent actively consults the worldbook together with the current episode state and episodic memory to infer their consequences and predict the resulting observations.

Figure 2 : Trace2Env framework. Trace2Env reconstructs historical traces into an environment worldbook and uses a world model agent to operate it as a stateful simulated environment.

Method	Environment Score $\uparrow$							Avg. $\uparrow$
Method	Terminal	SWE	Android	Web	Food	Shopping	Benefits	Avg. $\uparrow$
World-Model Backbone: gpt-5.6-sol
Direct Prompting	55.76	65.87	62.80	54.23	81.67	77.32	88.89	69.51
Trace RAG Prompting	58.42	66.50	66.50	57.12	85.57	86.07	96.11	73.76
Worldbook Prompting	59.07	67.82	65.53	57.70	86.74	85.89	97.68	74.35
Harness only	60.06	69.89	62.62	55.40	82.07	77.62	87.89	70.79

Table 1 : Next-observation prediction across environments and world model backbones. Scores are the official AgentWorldBench five-dimension judge score (Format, Factuality, Consistency, Realism, and Quality), averaged on a 0–100 scale.

World Model	Real	WM	W2R	CR
ALFWorld
Direct Prompting	93%	97%	3%	0.032
Trace2Env	93%	88%	85%	0.914
SciWorld
Direct Prompting	85%	92.5%	45%	0.529
Trace2Env	85%	87.5%	60%	0.706

Table 2 : Task success rate in the real environment ( Real ), the simulated world model ( WM ), and real-environment replay of world-model-induced actions ( W2R ); higher CR indicates better transfer.

Variant	Total	Effect
Direct Prompting	55.76	–
Schema only	62.05	–
+ Abstraction (rules, contracts, notes)	60.69	$-1.12\pm 0.90$
+ Evidence (evidence turns, demonstrations)	64.18	$+2.45\pm 0.89$
Full worldbook	64.89
Abstraction given Evidence		$+0.69\pm 0.63$

Table 3 : Worldbook component ablations on Terminal. Effects are paired score differences relative to the corresponding variant without the added component;

\pm

denotes standard error.

Figure 3 : Trace scaling on Terminal. Next-observation prediction score and one-time worldbook construction cost as the number of construction traces increases.

Figure 4 : The task agent generates an unsupported put action. Direct Prompting hallucinates a successful placement, causing a false state transition and a simulated success that fails under real-environment replay. Trace2Env not only predicts the correct no-op, but also uses its worldbook and persistent state tracking to preserve the inventory state and support subsequent interaction, enabling recovery with the valid move command.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Env.	Rows/traj.	Action $\rightarrow$ observation	Construction traces and Worldbook
Terminal	354 / 76	tmux keystrokes $\rightarrow$ terminal screen	20 own Terminus-2 trajectories on Terminal-Bench 2.0 (334 turns). Worldbook: 15 actions; 29 state fields; 3/12 executable-rule candidates; 18 notes; 334 evidence turns.
SWE	472 / 99	Coding-agent tools $\rightarrow$ tool result	Benchmark’s own sessions, 5-fold cross-fit per scaffold (2,585 visible transitions). Worldbook: 7–15 actions; 51–137 state fields; 7–17 executable rules; 17–30 notes; 935–1,147 evidence turns.
Android	200 / 92	UI actions $\rightarrow$ screen-element list	Benchmark’s own trajectories, 5-fold cross-fit per sub-source (789 visible transitions). Worldbook: 6–10 actions; 34–319 state fields; 7–23 executable rules; 10–23 notes; 36–334 evidence turns.
Web	200 / 118	Playwright MCP calls $\rightarrow$ page snapshot	50 own WebArena trajectories on the benchmark’s tasks. Worldbook: 9 actions; 227 state fields; 18/28 executable-rule candidates; 23 notes; 501 evidence turns.
Food	150 / 15	Dietary-profile tools $\rightarrow$ profile or status response	16 reserved successful EnvScaler trajectories (335 turns). Worldbook: 17 actions; 16 state fields; 6/21 executable-rule candidates; 17 notes; 335 evidence turns.
Shopping	84 / 15	Cart and inventory tools $\rightarrow$ item list or status response	3 reserved successful EnvScaler trajectories (55 turns). Worldbook: 10 actions; 4 state fields; 8/10 executable-rule candidates; 13 notes; 55 evidence turns.

Appendix

Table 4 : Environments, evaluation data, and construction data. Worldbook statistics for cross-fitted environments are ranges over the per-fold worldbooks.

Env.	Method	Format	Factuality	Consistency	Realism	Quality	Total
Terminal	Direct Prompting	66.2	41.7	65.2	52.8	52.9	55.76
	Trace RAG Prompting	69.3	44.4	66.7	54.9	56.7	58.42
	Worldbook Prompting	69.9	45.3	67.8	56.1	56.3	59.07
	Harness only	76.0	44.8	65.2	58.8	55.4	60.06
	Trace2Env	80.7	50.5	71.0	64.3	60.6	65.42
SWE	Direct Prompting	70.0	57.8	71.5	67.6	62.5	65.87

Appendix

Table 5 : Five-dimension judge scores, GPT-5.6-Sol. best total per environment in bold.

Env.	Method	Format	Factuality	Consistency	Realism	Quality	Total
Terminal	Direct Prompting	77.1	44.9	66.7	58.6	56.0	60.65
	Trace RAG Prompting	78.8	47.1	67.7	59.5	56.3	61.90
	Worldbook Prompting	80.7	49.6	71.2	63.9	60.8	65.24
	Harness only	78.9	46.2	67.1	60.5	57.6	62.06
	Trace2Env	80.1	48.9	68.9	61.9	60.9	64.12
SWE	Direct Prompting	68.0	55.2	68.6	66.0	59.7	63.50

Appendix

Table 6 : Five-dimension judge scores, DeepSeek-V4.1-Flash. Layout as in Table 5 .

Env.	Backbone	T2E $-$ DP	T2E $-$ H	H $-$ DP
Terminal	GPT-5.6-Sol	$+9.66{\pm}1.17$ [7.18, 12.15]	$+5.37{\pm}1.06$ [3.17, 7.66]	$+4.29{\pm}1.12$ [1.93, 6.75]
	DeepSeek-V4.1-Flash	$+3.41{\pm}1.17$ [1.13, 5.74]	$+2.06{\pm}1.22$ [ $-0.33$ , 4.53]	$+1.49{\pm}1.22$ [ $-0.88$ , 3.80]
SWE	GPT-5.6-Sol	$+4.75{\pm}0.84$ [2.98, 6.54]	$+0.70{\pm}0.76$ [ $-0.76$ , 2.17]	$+4.04{\pm}0.73$ [2.57, 5.63]
	DeepSeek-V4.1-Flash	$+6.75{\pm}0.96$ [5.00, 8.60]	$+4.11{\pm}0.79$ [2.53, 5.75]	$+2.79{\pm}0.79$ [1.24, 4.37]
Android	GPT-5.6-Sol	$+2.30{\pm}1.28$ [0.02, 4.63]	$+2.48{\pm}1.28$ [0.05, 5.00]	$-0.18{\pm}0.66$ [ $-1.52$ , 1.18]
	DeepSeek-V4.1-Flash	$+8.25{\pm}1.56$ [5.42, 11.14]	$+5.40{\pm}1.35$ [2.51, 8.36]	$+2.85{\pm}1.18$ [0.69, 5.10]

Appendix

Table 7 : Paired row differences on the four AgentWorldBench environments. Values are mean

\pm

row-level SE, with trajectory-clustered 95% CIs in brackets. WB = Worldbook Prompting, RAG = Trace RAG Prompting, H = Harness only, DP = Direct Prompting, and T2E = Trace2Env.

Env.	Stratum	Rows	GPT-5.6-Sol	DeepSeek-V4.1-Flash
Terminal	task not among construction tasks	312	$+3.69\pm 0.95$ [ $+1.71$ , $+5.63$ ]	$+0.61\pm 1.19$ [ $-1.78$ , $+2.96$ ]
Terminal	task among construction tasks	42	$+17.86\pm 5.10$	$+12.86\pm 4.86$
Web	not linked to a construction task	151	$+4.21\pm 1.50$ [ $+1.30$ , $+7.38$ ]	$+5.00\pm 1.35$ [ $+2.36$ , $+7.86$ ]
Web	linked (same task or template)	49	$+3.98\pm 2.15$	$+6.22\pm 2.77$
Android †	app covered by another fold	73	$+7.33\pm 2.31$ [ $+2.96$ , $+12.14$ ]	$+15.00\pm 2.63$ [ $+10.13$ , $+20.36$ ]
Android †	app in no other fold	127	$-0.59$	$+4.37$

Appendix

Table 8 : Worldbook gain by construction overlap (mean

\pm

SE, clustered 95% CI). Rows marked

\dagger

report Trace2Env

-

Direct Prompting; the others report Trace2Env

-

Harness only.

System	Worldbook	State	Loop	Knowledge shown	Total	Effect (vs. reference)
Direct Prompting	none	no	no	none	55.76	–
Trace RAG Prompting	none	no	no	top-5 raw turns in the prompt	58.42	$+2.66\pm 1.09$ (vs. Direct Prompting)
Worldbook Prompting	full	no	no	worldbook block in the prompt	59.07	$+3.31\pm 1.06$ (vs. Direct Prompting)
Harness only	none	no	yes	none	60.14	$+4.38\pm 1.08$ (vs. Direct Prompting)
Schema only	schemas	yes	yes	none	62.05	$+1.91\pm 0.91$ (vs. Harness only)
+ Abstraction	schemas + abstraction	yes	yes	rules, contracts, notes	60.69	$-1.12\pm 0.90$ (vs. Schema only)

Appendix

Table 9 : Complete Terminal attribution and ablation results (GPT-5.6-Sol, 354 records). Effects are paired row differences

\pm

SE against the reference in the last column.

System	Score $\uparrow$	Calls $\downarrow$	Cost/row ( $)$ \downarrow$	Latency (s) $\downarrow$
Direct Prompting	55.76	1.0	$\approx$ 0.04	26.0
Trace RAG Prompting	58.42	1.0	0.051	24.7
Worldbook Prompting	59.07	1.0	0.051	34.7
Harness only	60.14	1.4	0.036	17.3
Trace2Env without the adaptive loop	63.09	2.4	0.054	20.4
Trace2Env without state tracking	63.91	3.3	0.144	27.9

Appendix

Table 10 : Terminal efficiency–accuracy trade-off (GPT-5.6-Sol). Calls are model calls per predicted observation, cost is the list-price estimate from the recorded call logs, latency is wall-clock seconds per record.

Env.	tokens: DP / RAG / WB	Harness only	Trace2Env	Tool calls / record
Terminal	12k / 15k / 19k	$0.04, 19 s /$ 0.01, 26 s	$0.21, 41 s /$ 0.03, 48 s	1.3 / 4.1
SWE	33k / 36k / 39k	$0.05, 11 s /$ 0.18, 14 s	$0.20, 25 s /$ 0.43, 25 s	1.2 / 3.2
Android	20k / 24k / 30k	$0.07, 21 s /$ 0.22, 33 s	$0.31, 46 s /$ 0.59, 49 s	1.9 / 4.6
Web	25k / 29k / 35k	$0.09, 24 s /$ 0.26, 27 s	$0.39, 18 s /$ 0.68, 45 s	2.3 / 5.1

Appendix

Table 11 : Inference cost per record across environments for the reported runs (list prices; GPT-5.6-Sol / DeepSeek-V4.1-Flash). Single-call systems cost one prompt of the stated size.

Artifact	Content and use
Schemas
Action schema	Action types with their argument names and types; a supplied action is canonicalized against it.
State schema	Declared state variables with type and mutability; every state effect is checked against it.
Grounded evidence
Evidence records	One per aligned transition: the action, its outcome, extracted facts and candidate effects with uncertainty, and the recorded observation verbatim.
Demonstrations	Up to three transitions per action type with their observed before/after facts.

Appendix

Table 12 : Worldbook artifacts , grouped by the four components of Section 4.1 .

Setting	Value
Reconstruction
Confidence cap for ambiguous transitions	0.5
Minimum confidence to compile a rule	0.55
Demonstrations per action type	3
Simulation
Trust policy	Reference: model-proposed effects accepted after schema and state-constraint checks

Appendix

Table 13 : Hyperparameters and settings of the reported runs. Rows marked benchmark apply to next-observation evaluation only.

Case	Required information	Direct Prompting	Trace2Env
1. Heredoc echo	Terminal continuation-prompt behavior not yet observed in the episode	Begins partway through the script and omits much of the heredoc interaction	Retrieves heredoc turns from other tasks together with the induced continuation-prompt note
2. pip install pandas	Package state accumulated from earlier installation output	Predicts pandas alone and misses its remaining dependencies	Uses tracked world.package_versions together with retrieved installation evidence
3. ALFWorld placement	An unsupported put is a no-op, and the box remains in inventory	Accepts the placement and continues from a false state	Preserves the failed transition and state, then provides inventory and help behavior that enables recovery

Appendix

Table 14 : Overview of the case studies. Each case highlights the information required for a faithful prediction, what Direct Prompting misses, and the information Trace2Env uses.

Field	Content
official_input	the benchmark’s turn messages for this record, verbatim and complete: every earlier turn (action and real observation) and the current turn’s action; each earlier turn carries the memory id the agent may cite
action	the current action, normalized (type and typed arguments)
context	caller-supplied context (the current input as the benchmark presents it)
state_summary	a bounded summary of the tracked session state
state_fields	the declared state fields and their types
retrieved	worldbook items selected for the action, each with its Applicability Gate label: the action’s specification, eligible rules with whether they apply now, the applicable rule if any, invariants, notes, renderer contracts with examples, demonstrations

Appendix

Table 15 : world model agent brief. Structured information provided to the agent at each prediction step.

Tool	Description
list_actions	List the environment’s known actions with their aliases and argument names.
inspect_action	Retrieve an action’s contract, the rules eligible for it now, its observation contracts, notes, and demonstrations.
search_knowledge	Full-text search over reconstructed knowledge: rules, notes, demonstrations, evidence turns, and actions.
read_evidence	Read one raw turn of the episodes the worldbook was built from: its action and an exact span of its real observation, with the neighbouring turns’ actions.
read_state	Read the current session state at a dotted path, or the whole state.
state_schema	Describe declared state fields (types, mutability, descriptions), optionally under a path prefix.

Appendix

Table 16 : world model agent tools. The runtime interface for inspecting reconstructed knowledge, episode state, and memory, and for submitting a transition.

Field	Description
effects	typed state changes on declared paths ( set , delete , increment , decrement , append , merge , remove , schedule )
outcome	success , failure , partial , or unknown , as the environment would classify it
observation	the exact observation text the environment returns
rule_ids	rules whose effects are applied verbatim
citations	identifiers of the retrieved artifacts relied on (rules, notes, evidence turns, memory entries)
uncertainty	what could not be established from the workspace

Appendix

Table 17 : Transition proposal. Typed output of the world model agent before runtime validation and state commit.

Explore similar work

Jun 23, 2026cs.CL

Qwen-AgentWorld: Language World Models for General Agents

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

Yuxin Zuo, Zikai Xiao, Li Sheng +30

Qwen Team

World Models

Ask PDF

Apr 24, 2026cs.AI

Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond

As AI systems move from generating text to accomplishing goals through sustained interaction, the ability to model environment dynamics becomes a central bottleneck. Agents that manipulate objects, navigate software, coordinate with others, or design experiments require predictive environment models, yet the term world model carries different meanings across research communities. We introduce a "levels x laws" taxonomy organized along two axes. The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence. The second identifies four governing-law regimes: physical, digital, social, and scientific. These regimes determine what constraints a world model must satisfy and where it is most likely to fail. Using this framework, we synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery. We analyze methods, failure modes, and evaluation practices across level-regime pairs, propose decision-centric evaluation principles and a minimal reproducible evaluation package, and outline architectural guidance, open problems, and governance challenges. The resulting roadmap connects previously isolated communities and charts a path from passive next-step prediction toward world models that can simulate, and ultimately reshape, the environments in which agents operate. Code and resources are available at: https://github.com/matrix-agent/awesome-agentic-world-modeling.

Meng Chu, Xuan Billy Zhang, Kevin Qinghong Lin +47

World Models Multi-Agent Simulations World Artificial Intelligence Systems Model-Based Reinforcement Learning

Ask PDF

May 8, 2026cs.AI

EnvSimBench: A Benchmark for Evaluating and Improving LLM-Based Environment Simulation

Scalable AI agents training relies on interactive environments that faithfully simulate the consequences of agent actions. Manually crafted environments are expensive to build, brittle to extend, and fundamentally limited in diversity. A promising direction is to replace manually crafted environments with LLM-simulated counterparts. However, this paradigm hinges on an unexamined core assumption: LLMs can accurately simulate environmental feedback. In practice, LLM-simulated environments suffer from hallucinations, logical inconsistencies, and silent state drift failures that corrupt agent reward signals and compound the construction costs that the paradigm was designed to eliminate. To address this gap, we propose EnvSimBench with four contributions: 1) We provide the first formal definition and operationalization of Environment Simulation Ability (EnvSim Ability) as a quantifiable research objective. 2) We construct EnvSimBench, a rigorous benchmark covering 400 samples across 167 diverse environments, equipped with verifiable labels and fine-grained difficulty stratification along three axes. 3) Systematic evaluations reveal that all state-of-the-art language models suffer from a universal state change cliff: they achieve near-perfect accuracy on tasks when the environment state remains invariant, yet fail catastrophically when multiple states need simultaneous updates. This finding exposes EnvSim Ability as a critical yet largely unaddressed capability gap. 4) We design a constraint-driven simulation pipeline that substantially reduces hallucination, boosts environment synthesis yield by 6.8%, and cuts costs by over 90%. Overall, EnvSimBench serves as both a diagnostic framework and a practical optimization path for reliable LLM-based environment simulation, establishing a foundation for scalable agent training. Code and data are available at https://github.com/cookieApril/EnvSimBench

Yi Liu, TingFeng Hui, Wei Zhang +4

Beijing University of Posts and Telecommunications · The Hong Kong University of Science and Technology · Chongqing University

Synthetic Environments Simulation-Based Reinforcement Learning Environment

Ask PDF