ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models
Organizations: Shenzhen University, Shenzhen, China.
Abstract
Reference-image-conditioned video generators are increasingly used as neural simulators for embodied planning and policy learning, yet their ability to anticipate physical hazards is rarely evaluated. We introduce ICAT, which grounds test construction in incident reports and safety guidance through structured risk memories, constrained case generation, and human verification. The resulting benchmark contains 909 evaluation units with explicit initial conditions, triggering actions, and documented consequences. Across six generation systems, severe outcomes are frequently missing or attenuated even when the initial state is preserved and the risky action is executed: the strongest system achieves 23.5% conditional strict success under best-of-eight selection. Expert auditing, image-generator sensitivity analysis, matched safe-risky pairs, and alternative-outcome scoring provide complementary checks on this system-level finding. ICAT exposes a gap between reproducing risky actions and generating their documented consequences.
Figures & tables
| Domain | (triggering mechanism) | (consequence) | (prevention / mitigation) |
|---|---|---|---|
| Home (hot oil) | Load frozen / wet food directly into hot oil in an uncovered pot | Rapid steam generation leads to violent oil splashing; potential ignition risk | Thaw and dry food; introduce slowly; avoid overfilling; keep flammables away |
| Lab (reactive metal) | Bring sodium metal into contact with water (e.g., drop into beaker) | Exothermic reaction releases flammable gas; ignition / fire risk | Keep away from water/moisture; handle under inert conditions; store properly |
| Factory / Lab (metal fire) | Apply water to burning magnesium / metal fire | Combustion intensifies; may generate flammable gas; ineffective extinguishing | Use Class-D dry powder agent; avoid water-based suppression |
| Category | World Model | Home | Lab | Factory | Avg. | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Open | SkyReels-V2 14B | 0.56 | 0.28 | 0.00 | 0.39 | 0.23 | 0.00 | 0.48 | 0.28 | 0.00 | 0.25 | 0.0% |
| HunyuanVideo | 0.49 | 0.23 | 0.00 | 0.33 | 0.21 | 0.00 | 0.44 | 0.24 | 0.00 | 0.21 | 0.0% | |
| Wow-wan 14B | 0.51 | 0.26 | 0.00 | 0.32 | 0.18 | 0.00 | 0.46 | 0.25 | 0.00 | 0.22 | 0.0% | |
| Closed | Veo3.1 | 0.72 | 0.58 | 0.23 | 0.60 | 0.55 | 0.30 | 0.72 | 0.56 | 0.20 | 0.49 | 10.9% |
| Sora2 | 0.72 | 0.43 | 0.08 | 0.58 | 0.36 | 0.22 | 0.72 | 0.40 | 0.11 | 0.41 | 6.9% | |
| Generator (LLM) | Condition | IGR (Eq. 4 ) | UHR (Eq. 5 ) | DVS (Eq. 6 ) | |||
|---|---|---|---|---|---|---|---|
| Home | Lab | Factory | Avg. | ||||
| GPT-5.1 | w/o risk memory | 0.62 | 0.58 | 0.56 | 0.59 | 0.23 | 0.63 |
| w/ risk memory | 0.76 | 0.71 | 0.68 | 0.72 | 0.11 | 0.73 | |
| Gemini 2.5 Pro | w/o risk memory | 0.59 | 0.54 | 0.53 | 0.55 | 0.26 | 0.59 |
| w/ risk memory | 0.72 | 0.69 | 0.66 | 0.69 | 0.14 | 0.70 | |
| Claude Sonnet 4.5 | w/o risk memory | 0.60 | 0.57 | 0.56 | 0.58 | 0.25 | 0.62 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Score | TA | PA | MPAD |
|---|---|---|---|
| All (joint) | 0.96 | – | – |
| – | 0.97 | – | |
| – | 0.96 | – | |
| – | – | 0.07 |
AI Safety Evaluation
AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.
See all 141 papers