Specification-Driven Generation and Evaluation of Discrete-Event World Models via the DEVS Formalism
Authors: Zheyu Chen, Huiteng Zhuang, Zhuohuan Li, Chuanhao Li
Organizations: Zhili College, Tsinghua University · School of Transportation Science and Engineering, Beihang University · Department of Industrial Engineering, Tsinghua University
World models are central to LLM agents that must evaluate actions over long horizons. Yet much existing work focuses on environments governed by physical dynamics or spatial structure, whereas many high-impact domains, including supply chains, procurement networks, and business processes, evolve through discrete events, timing constraints, and causal dependencies. These settings call for discrete-event world models. Existing approaches to constructing world models often fall near two extremes: hand-engineered simulators provide consistency and reproducibility, but are costly to build and adapt; neural models are flexible, but can suffer from compounding inconsistency over long-horizon rollouts. We seek a principled middle ground by synthesizing discrete-event world models online from natural-language specifications, retaining the reliability of explicit simulators while gaining the adaptability of neural models. We adopt the DEVS formalism and introduce a staged LLM-based generation pipeline that separates structural inference over component interactions from component-level event and timing logic. For evaluation, we develop benchmark suites in which simulators emit structured event traces, which are then validated against specification-derived temporal, causal, and semantic constraints. This enables reproducible verification and localized diagnostics. Together, these contributions produce world models that remain consistent over long-horizon rollouts, can be verified from observable behavior, and can be synthesized efficiently on demand during online execution.
Figures & tables
Figure 1: Illustrative example of the generation and execution of a discrete-event world model for a warehouse robot fleet restocking task. To facilitate planning, a natural-language specification is translated into a world model, which exposes a standardized execution interface for interventions (e.g., fleet size, dispatching rule, charger count) and emits a structured event trace and performance metrics. We propose to operationalize such a world model as a DEVS model composed of interacting atomic and coupled components, each representing an entity with local state, event-handling logic, and timing semantics. Interactions among entities are realized through event messages routed via component ports, while event timing is governed by component-specific time-advance and transition functions. This enables systematic what-if analysis and comparative evaluation for online planning.
Figure 2: Illustration of DEVS-Gen . A natural-language specification is transformed into a discrete-event world model via a two-stage pipeline; its generated artifacts, including the model hierarchy and event stream, can be used to build visualization frontends for animation and interactive inspection.
Scenario
Domain
Key Dynamics
Words
Ents
Evts
Ref. LOC
Scale
SEIRD
Bio-Math
Continuous (ODE)
507
5
1
745
S
ABP
Network
Stop-and-Wait
854
4
5
573
S
Barbershop
Service
Blocking & Signals
700
3
2
570
S
IOBS
Banking
Pipeline w/ Routing
822
6
7
703
M
OTrain
Transport
Schedule-driven
755
4
4
961
M
FileTransfer
Network
Dual-Loop FSM
750
8
12
937
L
Table 1: Benchmark scenarios summary. The dataset covers diverse domains and dynamic characteristics, ordered by their overall scale. We further quantify complexity through specification length (Words), number of entities (Ents), distinct event types (Evts), and reference code size (Ref. LOC).
Single-Pass (No Execution)
Iterative (w/ Execution)
Model
Metric
DEVS-Gen (Ours)
OpenH-L
SWE-L
OpenH
SWE
GPT-5.2
Scorebeh
0.87 ± 0.27
0.91 ± 0.23
0.00 ± 0.00
0.90 ± 0.24
0.82 ± 0.36
Scoreope
0.90 ± 0.20
0.95 ± 0.22
0.00 ± 0.00
0.95 ± 0.22
0.86 ± 0.36
GLM-4.7
Scorebeh
0.75 ± 0.23
0.75 ± 0.27
0.56 ± 0.42
0.91 ± 0.09
0.70 ± 0.41
Scoreope
0.95 ± 0.24
0.90 ± 0.30
0.76 ± 0.44
1.00 ± 0.00
0.76 ± 0.44
Qwen3-30B
Scorebeh
0.59 ± 0.32
0.60 ± 0.38
0.66 ± 0.24
0.61 ± 0.38
0.73 ± 0.29
Table 2: Main Results on Effectiveness. We group the frameworks into Iterative (utilize execution feedback to debug) and Single-Pass (generate models directly). Bold indicates the best performance within each respective group. DEVS-Gen function best for single-pass generation and achieves performance highly comparable to fully iterative agents.
Figure 3: Time-to-Success distribution across all evaluated models. The x-axis represents the cumulative success rate, and the y-axis indicates the wall-clock time required to achieve it. While baseline agents show low initial latency, they hit a vertical wall before reaching an 80% success rate due to non-terminating debugging cycles. In contrast, DEVS-Gen translates computation time into continuous task completion, reliably scaling to near 100% success.
GLM-4.7 (Large)
Qwen3-30B (Small)
Setup
Scoreope↑
Scorebeh↑
Scoreope↑
Scorebeh↑
single (Unconstrained)
0.74
0.57
0.74
0.51
single-xdevs (Strict xdevs )
0.45 ( ↓ 0.29)
0.30 ( ↓ 0.27)
0.66 ( ↓ 0.08)
0.43 ( ↓ 0.08)
DEVS-Gen (Ours)
0.95 ( ↑ 0.50 )
0.74 ( ↑ 0.44 )
0.82 ( ↑ 0.16 )
0.59 ( ↑ 0.16 )
Table 3: Ablation on Framework Constraints. We report Scoreope and Scorebeh . Directly enforcing strict structural rules ( single-xdevs ) causes a performance drop across both metrics compared to free-form coding ( single ). DEVS-Gen utilizes the PlanTree scaffold to absorb this complexity, surpassing the unconstrained baselines.
Figure 4: Ablation study on synthesis latency (using GPT-5.2). The chart compares the wall-clock time required for the Planning and Generation phases under serial and parallel execution modes. While the acceleration in the Planning phase is limited by the current benchmark scale, the Generation phase achieves a 4.7× speedup, validating the scalability of the approach.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Term
Definition
Core Concepts
World Model
The functional and conceptual predictive engine used by an autonomous agent to forecast the consequences of its actions and simulate environment dynamics over time. In this work, it specifically refers to models targeting discrete-event environments for long-horizon what-if reasoning.
Simulator ( M )
The concrete, executable software artifact (e.g., Python code) that operationalizes the world model. It acts as a black-box engine that accepts external interventions as inputs and emits observable event traces as outputs.
Event Trace ( T )
A chronological, structured sequence of discrete events emitted by the simulator during execution. Each record captures the timestamp, emitting entity, event type, and payload, serving as the observable behavior for specification-driven evaluation.
Environment Specification ( Spec )
The natural-language description guiding the simulator generation, partitioned into operational configurations ( Specope ) and behavioral descriptions ( Specbeh ).
DEVS Formalism
Appendix
Table 8
Figure 5: Visualized PlanTree hierarchy for the ABP Model. The root model recursively decomposes into sub-models until atomic primitives are reached.
Figure 6: The final connection of the ABP DEVS model.
World models are moving from generating realistic frames to generating playable worlds, yet whether a delivered world can keep running is not tested anywhere. Existing evaluations stop at generation, at delivery, or at single-step transitions, and each stops at a different point along the way. Correct local state transitions or intermediate outcomes do not guarantee a correctly organized causal event flow. We propose the world agent task, which moves the evaluation point of world generation from the moment of delivery to the continued operation that follows. In this task, a model is not asked to generate a world. It is held responsible for keeping the world running, which requires coordinating events and carrying forward their consequences to constrain subsequent evolution. We instantiate the task in WorldAgent-Benchmark with two complementary tracks. In the maintenance track, the model must ground the events of a continuous narrative into correct transitions of the explicit world state while respecting causal, temporal, and concurrency constraints. In the deduction track, the model must predict how the world will evolve under partial observations and act toward a goal. The maintenance track combines LLM-assisted semantic judgments with programmatic validation and scoring, while the deduction track is evaluated entirely programmatically. Individual judgments are auditable against world states and execution logs, and scores can be recomputed from the saved judgments and execution records. Across 8 models, scores decline steadily as pre-built structure is removed from the world, and causal-relation checking is the weakest component for every model. The benchmark makes the continued operation of a world measurable and distinguishes local completion from failures in event organization. Code and dataset will be released on https://github.com/HCPLab-SYSU/WorldAgent-Benchmark.
Weixing Chen, Weipeng Zhang, Nan An +2
Sun Yat-sen University · South China University of Technology · Guangdong Key Laboratory of Big Data Analysis and Processing +2
World models offer a principled way to equip long-horizon LLM agents with foresight: predictions of action consequences before execution. However, unreliable foresight can be ignored, misused, or even degrade downstream decision-making. In this paper, we introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen. WorldEvolver integrates three modules: (i) Episodic Memory, which exploits real action transitions through retrieval-based simulation; (ii) Semantic Memory, which extracts persistent heuristic rules from prediction-observation mismatches; and (iii) Selective Foresight, which filters low-confidence predictions before integrating them into agent reasoning context. We evaluate WorldEvolver on ALFWorld and ScienceWorld, measuring world model prediction accuracy on Word2World and downstream agent success rate on AgentBoard. Extensive experiments show that WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate, demonstrating that test-time memory revision enhances both predictive fidelity and planning performance.
Xuan Zhang, Wenxuan Zhang, See-Kiong Ng +1
National University of Singapore · Singapore University of Technology and Design · Singapore Management University
Language agents solve complex tasks through plans and actions. A single step the world refuses puts the goal out of reach, and what the agent does next decides the task. Prompted planners fail at exactly this point, rewriting the refused step in new words, meeting the same refusal, and burning the attempt budget without moving. They fail because the plan was never tied to the world, so a refusal has nothing in the plan to attach to. A world is where a task runs, and it has its own rules, its own admissible actions, and its own constraints. We build synthetic worlds across 7 domains and extract training data from them. A program enforces each world's rules and grades its goal, and every world is admitted only if its goal is reachable from its initial state. Agents run inside and leave verified failures paired with repairs that carried the run to a state the world certified, a record of about 226K trajectories. On this record we train the World State Generator, a model that writes a plan as checkable states of the world and keeps that plan aligned with the world it runs in. That alignment is what a plan written in language lacks, since the world it runs in has physical limits, logical dependencies, and required orders the language never states, and the plan encounters these rules only when a state fails. WSG takes that failure as the rule the world has stated and rewrites the remaining states to obey it, so the plan bends to the world as the run goes on. Across 7 public benchmarks, WSG raises end-to-end success for two open models near 30B parameters over prompting and brings to the level of proprietary model.
Sungheon Jeong, Sanggeon Yun, Ryozo Masukawa +3
University of California, Irvine · Northeastern University