Language-conditioned traffic simulation enables scalable test scenarios steered by natural language. The goal is to control selected agents while surrounding traffic reacts to their observed behaviour. For example, a vehicle instructed to cut in late should cause the following vehicle to brake only when the manoeuvre becomes observable. Yet simulators can leak private instructions across agents, causing surrounding vehicles to react before the instructed behaviour becomes observable. Such premature reactions defuse desired events and alter the ensuing closed-loop scenario. Existing benchmarks measure what agents do, not what they could have known, leaving this failure unmeasured. We formalise it as intent leakage and address it through evaluation, data and modelling. Our paired intervention test holds an instructed agent's executed trajectory fixed while varying its private instruction, isolating leakage-induced changes in surrounding agents. We also construct WOMD-PACT, an automatically generated, verifiable dataset of 531,099 WOMD scenes with timed behaviour tags with reaction episodes identified through a counterfactual ghost test and grounded private instructions. Finally, we introduce CaFe, a dual-stream diffusion-forcing simulator fine-tuned in closed loop that restricts inter-agent interactions to public beliefs, preserving legitimate anticipation without leaking private intent. Our evaluation reveals that existing conditioning designs displace 23-83% of nearby uninstructed agents through private-intent leakage, even when the instructed agent's executed motion is held fixed. CaFe eliminates measurable leakage while maintaining comparable WOSAC realism (0.763), demonstrating that causal faithfulness can be enforced without sacrificing measured simulation realism.
Figures & tables
Dataset
Base
Verif.
Private
Symm.
Inter.
Timed
ProSim-Instruct [ 37 ]
WOMD
✗
✗
✓
✗
∼
InterDrive [ 4 ]
WOMD, nuPlan
∼
✗
∼
✓
✗
WOMD-Reas. [ 21 ]
WOMD
∼
✗
✗
∼
∼
InstructWaymo [ 12 ]
WOMD
✓
–
✗
✗
✗
doPlan [ 31 ]
nuPlan
✓
–
✗
✗
∼
WOMD-PACT (ours)
WOMD
✓
✓
✓
✓
✓
Table 1: Instruction datasets for trajectory-level driving. We compare factual verification, private per-agent instructions, symmetric coverage, interaction identification and timing at behaviour onset. ∼ indicates partial support and – denotes single-agent or inapplicable settings.
Figure 2: Measuring intent leakage. We replay the instructed agent’s motion without its private instruction, compare surrounding agents across rollouts, and quantify leakage along two dimensions: the fraction of affected agents and their mean displacement.
Figure 3: CaFe. The private stream conditions each agent on its own instruction, while a language-blind pass of the frozen backbone predicts public beliefs from committed motion. In inter-agent attention, an agent reads its own private features and the public beliefs of others, never their private futures or instructions. After each one-second commit, both streams update from the new observable history. The example on the right shows how public beliefs change as an agent’s motion reveals its intention.
Leakage
Realism ↑
Accuracy and safety ↓
Method
Affected (%) ↓
Displacement (m) ↓
Meta
Kin.
Inter.
Map
minADE
Coll. (%)
Off-road (%)
1 s
4 s
8 s
1 s
4 s
8 s
ECoSim [ 6 ]
0.1
7.1
22.6
1.54
1.46
3.08
0.7685
0.4642
0.8032
0.8977
1.507
3.49
11.54
Backbone (no text)
n/a
n/a
n/a
n/a
n/a
n/a
0.7599
0.4493
0.7935
0.8940
1.417
4.34
11.68
Inter-agent attention [ 16 , 4 , 6 ]
0.1
11.2
26.7
0.78
1.40
2.77
0.7628
0.4624
0.7903
0.8990
1.362
5.82
12.22
Broadcast instructions [ 37 , 4 ]
22.2
69.2
82.7
0.87
2.32
4.14
0.7651
0.4694
0.7917
0.9000
1.435
6.06
12.18
Table 2: Intent leakage and realism on 1,000 WOMD-PACT validation scenes with 32 rollouts each. Variants reproduce the information routes of the cited works on a shared backbone with identical training data, sampler and leak-free guidance, isolating what each agent can observe. ECoSim [ 6 ] is evaluated as published as an external autoregressive baseline. Affected is the percentage of nearby moving uninstructed agents displaced by more than 0.5 m when instructed agents replay identical motion with their instructions removed, evaluated on the 746 eligible scenes (750 for ECoSim). Displacement is the mean displacement among affected agents. Realism and its components follow WOSAC 2025. n/a means that the leakage test is inapplicable (no-text backbone) or that mean displacement is undefined because no agents were affected. Bold indicates the best result among variants sharing our backbone.
Method
Clause (%) ↑
Reacts (%) ↑
B hits A (%) ↓
Meta ↑ (guidance-free)
No language
38.0
81.7
5.8
0.7494
Single stream
40.9
82.4
5.3
0.7466
History only
35.8
79.5
9.8
0.7238
CaFe (ours)
40.8
81.7
5.8
0.7481
Table 3: Instruction success and replay response quality. Clause satisfaction and responder metrics use eight matched rollouts per item. Responder metrics use fixed-motion replay with unprompted B .
Method
Guide
Meta ↑
Coll. ↓
Off-road ↓
Backbone
No
0.7494
5.6
13.7
Yes
0.7599
4.34
11.68
Single stream
No
0.7466
8.8
14.8
Yes
0.7628
5.82
12.22
CaFe (Ours)
No
0.7481
7.1
14.5
Yes
0.7631
4.81
12.10
Table 4: Effect of guidance. WOSAC 2025 realism, collision and off-road rates without and with guidance. Guided CaFe has 0.0% affected agents at 8 s.
Autonomous driving scenario generation creates realistic, controllable traffic situations for training and testing vehicles, especially in rare and safety-critical cases.
Authors are with CARIAD SE, Germany. · Author is with the Institute of Automotive Engineering (FZD), TU Darmstadt, Germany. · Author is with the Technical University of Munich, Germany.