Specification-Driven Generation and Evaluation of Discrete-Event World Models via the DEVS Formalism

Zheyu Chen; Huiteng Zhuang; Zhuohuan Li; Chuanhao Li

cs.AIMar 4, 2026

Specification-Driven Generation and Evaluation of Discrete-Event World Models via the DEVS Formalism

Authors: Zheyu Chen, Huiteng Zhuang, Zhuohuan Li, Chuanhao Li

Organizations: Zhili College, Tsinghua University · School of Transportation Science and Engineering, Beihang University · Department of Industrial Engineering, Tsinghua University

Abstract

World models are central to LLM agents that must evaluate actions over long horizons. Yet much existing work focuses on environments governed by physical dynamics or spatial structure, whereas many high-impact domains, including supply chains, procurement networks, and business processes, evolve through discrete events, timing constraints, and causal dependencies. These settings call for discrete-event world models. Existing approaches to constructing world models often fall near two extremes: hand-engineered simulators provide consistency and reproducibility, but are costly to build and adapt; neural models are flexible, but can suffer from compounding inconsistency over long-horizon rollouts. We seek a principled middle ground by synthesizing discrete-event world models online from natural-language specifications, retaining the reliability of explicit simulators while gaining the adaptability of neural models. We adopt the DEVS formalism and introduce a staged LLM-based generation pipeline that separates structural inference over component interactions from component-level event and timing logic. For evaluation, we develop benchmark suites in which simulators emit structured event traces, which are then validated against specification-derived temporal, causal, and semantic constraints. This enables reproducible verification and localized diagnostics. Together, these contributions produce world models that remain consistent over long-horizon rollouts, can be verified from observable behavior, and can be synthesized efficiently on demand during online execution.

Figures & tables

Figure 1: Illustrative example of the generation and execution of a discrete-event world model for a warehouse robot fleet restocking task. To facilitate planning, a natural-language specification is translated into a world model, which exposes a standardized execution interface for interventions (e.g., fleet size, dispatching rule, charger count) and emits a structured event trace and performance metrics. We propose to operationalize such a world model as a DEVS model composed of interacting atomic and coupled components, each representing an entity with local state, event-handling logic, and timing semantics. Interactions among entities are realized through event messages routed via component ports, while event timing is governed by component-specific time-advance and transition functions. This enables systematic what-if analysis and comparative evaluation for online planning.

Figure 2: Illustration of DEVS-Gen . A natural-language specification is transformed into a discrete-event world model via a two-stage pipeline; its generated artifacts, including the model hierarchy and event stream, can be used to build visualization frontends for animation and interactive inspection.

Scenario	Domain	Key Dynamics	Words	Ents	Evts	Ref. LOC	Scale
SEIRD	Bio-Math	Continuous (ODE)	507	5	1	745	S
ABP	Network	Stop-and-Wait	854	4	5	573	S
Barbershop	Service	Blocking & Signals	700	3	2	570	S
IOBS	Banking	Pipeline w/ Routing	822	6	7	703	M
OTrain	Transport	Schedule-driven	755	4	4	961	M
FileTransfer	Network	Dual-Loop FSM	750	8	12	937	L

Table 1: Benchmark scenarios summary. The dataset covers diverse domains and dynamic characteristics, ordered by their overall scale. We further quantify complexity through specification length (Words), number of entities (Ents), distinct event types (Evts), and reference code size (Ref. LOC).

		Single-Pass (No Execution)			Iterative (w/ Execution)
Model	Metric	DEVS-Gen (Ours)	OpenH-L	SWE-L	OpenH	SWE
GPT-5.2	$\text{Score}_{\text{beh}}$	0.87 $\pm$ 0.27	0.91 $\pm$ 0.23	0.00 $\pm$ 0.00	0.90 $\pm$ 0.24	0.82 $\pm$ 0.36
GPT-5.2	$\text{Score}_{\text{ope}}$	0.90 $\pm$ 0.20	0.95 $\pm$ 0.22	0.00 $\pm$ 0.00	0.95 $\pm$ 0.22	0.86 $\pm$ 0.36
GLM-4.7	$\text{Score}_{\text{beh}}$	0.75 $\pm$ 0.23	0.75 $\pm$ 0.27	0.56 $\pm$ 0.42	0.91 $\pm$ 0.09	0.70 $\pm$ 0.41
GLM-4.7	$\text{Score}_{\text{ope}}$	0.95 $\pm$ 0.24	0.90 $\pm$ 0.30	0.76 $\pm$ 0.44	1.00 $\pm$ 0.00	0.76 $\pm$ 0.44
Qwen3-30B	$\text{Score}_{\text{beh}}$	0.59 $\pm$ 0.32	0.60 $\pm$ 0.38	0.66 $\pm$ 0.24	0.61 $\pm$ 0.38	0.73 $\pm$ 0.29

Table 2: Main Results on Effectiveness. We group the frameworks into Iterative (utilize execution feedback to debug) and Single-Pass (generate models directly). Bold indicates the best performance within each respective group. DEVS-Gen function best for single-pass generation and achieves performance highly comparable to fully iterative agents.

Figure 3: Time-to-Success distribution across all evaluated models. The x-axis represents the cumulative success rate, and the y-axis indicates the wall-clock time required to achieve it. While baseline agents show low initial latency, they hit a vertical wall before reaching an

80\%

success rate due to non-terminating debugging cycles. In contrast, DEVS-Gen translates computation time into continuous task completion, reliably scaling to near

100\%

success.

	GLM-4.7 (Large)		Qwen3-30B (Small)
Setup	$\text{Score}_{\text{ope}}$ $\uparrow$	$\text{Score}_{\text{beh}}$ $\uparrow$	$\text{Score}_{\text{ope}}$ $\uparrow$	$\text{Score}_{\text{beh}}$ $\uparrow$
single (Unconstrained)	0.74	0.57	0.74	0.51
single-xdevs (Strict xdevs )	0.45 ( $\downarrow$ 0.29)	0.30 ( $\downarrow$ 0.27)	0.66 ( $\downarrow$ 0.08)	0.43 ( $\downarrow$ 0.08)
DEVS-Gen (Ours)	0.95 ( $\uparrow$ 0.50 )	0.74 ( $\uparrow$ 0.44 )	0.82 ( $\uparrow$ 0.16 )	0.59 ( $\uparrow$ 0.16 )

Table 3: Ablation on Framework Constraints. We report

\text{Score}_{\text{ope}}

and

\text{Score}_{\text{beh}}

. Directly enforcing strict structural rules ( single-xdevs ) causes a performance drop across both metrics compared to free-form coding ( single ). DEVS-Gen utilizes the PlanTree scaffold to absorb this complexity, surpassing the unconstrained baselines.

Figure 4: Ablation study on synthesis latency (using GPT-5.2). The chart compares the wall-clock time required for the Planning and Generation phases under serial and parallel execution modes. While the acceleration in the Planning phase is limited by the current benchmark scale, the Generation phase achieves a $4.7\times$ speedup, validating the scalability of the approach.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Term	Definition
Core Concepts
World Model	The functional and conceptual predictive engine used by an autonomous agent to forecast the consequences of its actions and simulate environment dynamics over time. In this work, it specifically refers to models targeting discrete-event environments for long-horizon what-if reasoning.
Simulator ( $\mathcal{M}$ )	The concrete, executable software artifact (e.g., Python code) that operationalizes the world model. It acts as a black-box engine that accepts external interventions as inputs and emits observable event traces as outputs.
Event Trace ( $\mathcal{T}$ )	A chronological, structured sequence of discrete events emitted by the simulator during execution. Each record captures the timestamp, emitting entity, event type, and payload, serving as the observable behavior for specification-driven evaluation.
Environment Specification ( Spec )	The natural-language description guiding the simulator generation, partitioned into operational configurations ( ${\texttt{Spec}}_{ope}$ ) and behavioral descriptions ( ${\texttt{Spec}}_{beh}$ ).
DEVS Formalism

Appendix

Table 8

Figure 5: Visualized PlanTree hierarchy for the ABP Model. The root model recursively decomposes into sub-models until atomic primitives are reached.

Figure 6: The final connection of the ABP DEVS model.

Explore similar work

World Agent: Can Language Models Keep a World Running?
Sep 26, 2026Weixing Chen, Weipeng Zhang, Nan An +2World Model Evaluation World Models
Sun Yat-sen University · South China University of Technology · Guangdong Key Laboratory of Big Data Analysis and Processing +2
Self-Evolving World Models for LLM Agent Planning
Jun 29, 2026Xuan Zhang, Wenxuan Zhang, See-Kiong Ng +1Test-Time Adaptation World Model Learning
National University of Singapore · Singapore University of Technology and Design · Singapore Management University
World State Generator
Sep 21, 2026Sungheon Jeong, Sanggeon Yun, Ryozo Masukawa +3LLM Agent Planning LLM Planning
University of California, Irvine · Northeastern University

Sep 26, 2026J/K move · Enter open · S save

World Agent: Can Language Models Keep a World Running?

Weixing Chen, Weipeng Zhang, Nan An, Yang Liu+1

Sun Yat-sen University · South China University of Technology · Guangdong Key Laboratory of Big Data Analysis and Processing+2

World Model Evaluation World Models Causal World Models Causal Reasoning in Language Models Temporal Reasoning in Language Models

Open paper AskPDF arXiv

World models are moving from generating realistic frames to generating playable worlds, yet whether a delivered world can keep running is not tested anywhere. Existing evaluations stop at generation, at delivery, or at single-step transitions, and each stops at a different point along the way. Correct local state transitions or intermediate outcomes do not guarantee a correctly organized causal event flow. We propose the world agent task, which moves the evaluation point of world generation from the moment of delivery to the continued operation that follows. In this task, a model is not asked to generate a world. It is held responsible for keeping the world running, which requires coordinating events and carrying forward their consequences to constrain subsequent evolution. We instantiate the task in WorldAgent-Benchmark with two complementary tracks. In the maintenance track, the model must ground the events of a continuous narrative into correct transitions of the explicit world state while respecting causal, temporal, and concurrency constraints. In the deduction track, the model must predict how the world will evolve under partial observations and act toward a goal. The maintenance track combines LLM-assisted semantic judgments with programmatic validation and scoring, while the deduction track is evaluated entirely programmatically. Individual judgments are auditable against world states and execution logs, and scores can be recomputed from the saved judgments and execution records. Across 8 models, scores decline steadily as pre-built structure is removed from the world, and causal-relation checking is the weakest component for every model. The benchmark makes the continued operation of a world measurable and distinguishes local completion from failures in event organization. Code and dataset will be released on https://github.com/HCPLab-SYSU/WorldAgent-Benchmark.

Abstract

Explore similar work

World Agent: Can Language Models Keep a World Running?

Self-Evolving World Models for LLM Agent Planning

World State Generator