Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI Systems

Kailin Wang; Haoxiang Jie; Yaoyuan Yan; Jiacheng Zhou; Zhiyou Heng

cs.ROAug 14, 2026

Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI Systems

Authors: Kailin Wang, Haoxiang Jie, Yaoyuan Yan, Jiacheng Zhou, Zhiyou Heng

Organizations: AI Lab, Country Garden Services Group · Fudan University · Omni AI

Abstract

EV-WM represents candidate quality with feature and event scores, but these scores do not explicitly record an unmet task predicate, a route label for an available correction mechanism, or a post-correction acceptance result. We present Onto-EV-WM, an ontology-grounded diagnosis and verification-gated correction interface layered above EV-WM rather than a replacement world-model architecture. The implemented task-local TBox defines entity types, predicate signatures, and constraints; source-specific grounding maps predicted or simulator-observed states to task ABoxes; and deterministic rules retain each missing predicate and its arguments when assigning a route label. Learned or heuristic proposers remain separate from this symbolic interface; native task predicates determine acceptance, and the bounded protocol determines whether a failed verification is retried. In the aligned PointMaze evaluation, EV-WM and Onto-EV-WM both report 94% success, with mean final-state distances of 0.90573 and 0.61177, respectively; the separately budgeted search reaches 100% success. On LIBERO-Goal, the ontology represents failed task conditions as typed records, retains their predicate arguments, and associates them with the declared source/joint correction route and predicate-gated acceptance; the complete configuration reports 93.8% corrected-window success on seed 0 and 94.05 +- 0.30% across four evaluation-sampling seeds. On the fixed 10,030-task LIBERO-Plus registry, Onto-EV-WM succeeds on 8,526 tasks (85.00%), with suite-level success rates of 65.98% for LIBERO-10, 91.39% for LIBERO-Goal, and 91.38% for both LIBERO-Object and LIBERO-Spatial. These numbers report the performance of the complete ontology-grounded configurations under the tested simulator protocols; an ontology-only causal share is not measured separately, and real-robot recovery is not evaluated.

Figures & tables

Figure 1: Ontology-grounded world-model architecture for failure diagnosis and closed-loop repair. a , Observation, goal, and candidate-action histories condition the world model $F_{\theta}$ , which produces illustrative imagined rollouts. b , The reusable TBox specifies entity types and predicate signatures, while the fact decoder $G_{\psi}$ instantiates a predicted ABox. Comparing it with the task-required ABox exposes a missing in_region relation as a typed failure. c , The gate either preserves the reference path or invokes a compatible plan-, state-, or action-level execution mechanism. The verifier accepts a predicate-valid outcome; otherwise, feedback returns to the candidate action window for bounded retry.

Component	Status	Role
Visual encoder and feature rollout	Reused	Imagine candidate future features
Event prediction and scoring	Reused	Estimate task progress and event scores
TBox, ABox grounding, and diagnosis	New	Type task assertions and expose missing predicates
Failure gate and execution routing	New	Retain the reference or select a plan-, state-, or action-level mechanism
Native task predicate	Evaluation tool	Verify the resulting state before acceptance
Bounded accept/retry control	New	Terminate on success or update the next candidate window

Table 1: Relationship to EV-WM. “New” denotes a component introduced by the ontology-grounded interface, not evidence that the interface alone causes the measured gain.

Class	Examples or signatures	Ontological role	Repair-loop use
Object/Entity	bowl, plate, drawer, mug	predicate argument	bind task entities
Region/Site	stove_front, caddy_right, rack_site	spatial landmark	ground target areas
Relation	on, inside, in_region, right_of	typed predicate	encode goal assertions
Action/Event	move, place, open, completed	transition/progress	describe proposed change
Constraint/Margin	contact, joint_limit, target_margin	condition/scalar	represent feasibility and threshold
Failure	predicate_mismatch, relation_missing, rack_contact	diagnostic class	name the unmet condition

Table 2: Core vocabulary of the operational robot task ontology.

Typed gate	State-correction family	Example tasks	Intended effect
source_joint_pose	learned source/joint delta	drawers, stove, placement	correct object or joint state
object_relation_predicate	object-relative correction	bowl on plate, containment	satisfy target relation
rack_contact_site	rack-local pose correction	bottle on rack	satisfy contact predicate
source_joint_residual	verifier-checked candidate	source/joint residuals	retain predicate-valid state

Table 3: Typed gates and state-correction families. Only the learned source/joint qpos -delta route contributes to the quantitative LIBERO Goal10 state-correction result; the other rows define schema and qualitative routing semantics.

Field	Setting
Evaluation unit / horizon	25-step demonstration window, $H=25$
Windows	50 per task, 500 per sampling seed; seeds 0–3
Checkpoint	One fixed learned source/joint qpos -delta head
Proposal source	Learned delta on replay failures only
Verifier	Native LIBERO check_success
State application	Direct simulator qpos mutation

Table 4: LIBERO Goal10 evaluation protocol. Sampling seeds change the evaluated window sample while the learned checkpoint remains fixed.

Method	Setting	Success rate	Mean state distance
DINO-WM baseline	aligned	0.90	0.93568
EV-WM	aligned	0.94	0.90573
Onto-EV-WM	aligned	0.94	0.61177
Onto-EV-WM + diagnosis-guided search	larger budget, success-first	1.00	0.86504

Table 5: PointMaze random-state planning results, including the published EV-WM baseline ( Wang et al., 2026 ) . The first three rows use the same 50-trial setting. The final row changes both search budget and success-first selection and is not directly aligned.

Sampling seed	Event success	Replay success	Corrected success	Rescued / replay failures
0	392/500	436/500	469/500	33/64
1	379/500	428/500	471/500	43/72
2	398/500	435/500	469/500	34/65
3	401/500	440/500	472/500	32/60
Mean $\pm$ s.d.	$78.50\!\pm\!1.95\%$	$86.95\!\pm\!1.00\%$	$94.05\!\pm\!0.30\%$	—
Pooled	1570/2000	1739/2000	1881/2000	142/261 (54.4%)

Table 6: LIBERO Goal10 sampled-window results across four evaluation-sampling seeds. Event reports the event-score CEM result; “Rescued” counts replay failures converted to predicate-valid outcomes by direct simulator-state correction. Mean and sample standard deviation are calculated across evaluation-sampling seeds, not independently trained models.

Figure 4: LIBERO Goal10 sampled-window evaluation. a , Paired replay and failure-gated corrected success for four evaluation-sampling seeds. b , Replay failures and the subset recovered by the same fixed learned correction checkpoint. Seeds vary the sampled evaluation windows rather than model training; the protocol uses neither held-out demonstrations nor full episodes.

Model	Long	Goal	Object	Spatial	Average
$\pi_{0.5}$	77.9	79.3	89.9	92.0	84.6
OpenVLA-OFT	66.4	63.0	66.5	84.0	69.6
$\pi_{0}$	61.3	63.4	74.7	78.6	69.3
RIPT-VLA	67.5	58.0	64.3	85.8	68.4
$\pi_{0}$ -FAST	43.4	57.5	72.7	74.4	61.6
UniVLA	39.9	40.7	36.7	55.5	42.9

Table 7: LIBERO-Plus comparison with representative methods. Average is the reported aggregate benchmark success rate; because suite sizes differ, it is not the unweighted mean of the four suite percentages. The best and second-best results are shown in bold and underlined, respectively.

Robot Failure Recovery

Robot failure recovery studies how robots detect, diagnose, and respond to errors, damage, and unexpected physical interactions.

See all 73 papers

Jun 11, 2026

EV-WM: Event-Verified World Models for Long-Horizon Robotic Manipulation

Pretrained-feature world models provide a useful substrate for robot imagination, but visual or latent prediction alone does not determine whether an imagined future satisfies task-relevant predicates. Long-horizon manipulation requires progress signals that are relational, predicate-level, and physically grounded: whether an object has moved, whether a drawer or contact state has changed, whether a placement predicate is satisfied, and whether a candidate future is reliable enough for execution. We introduce EV-WM, a predicate-grounded verification framework for world-model planning. EV-WM rolls out candidate futures in pretrained visual-feature space, decodes them into structured event states, and scores them using task-progress, semantic-consistency, physical-feasibility, and uncertainty terms. The verifier guides sampling-based planning, gates candidate actions, and, in the contact-sensitive LIBERO wine-rack setting, selects among PPO-generated proposals. Across navigation, deformable-object, wall-constrained, and language-described manipulation studies, EV-WM shows that predicate-grounded verification can make feature-space world-model planning more interpretable and better aligned with task progress.

Kailin Wang, Haoxiang Jie, Yaoyuan Yan +2

1AI Lab, Country Garden Services Group · 2Fudan University · 3Omni AI

Long-Horizon Robotic Manipulation World Models for Robotics World Model-Based Planning Model-Based Planning

Ask PDF

Oct 6, 2026

World Models Dream of Success: Diagnosing and Repairing Failure Insensitivity in Robot World Models

Robot world models support policy evaluation, planning, and synthetic data generation, but these applications require predictions that distinguish successful actions from failures. Across four released checkpoints from two architecture families, we observe weak sensitivity to action changes and success-like predictions on verified failures. Although recent work incorporates failures into model training, which data can repair released checkpoints without changing their architecture or training objective still remains underexplored. To this end, we introduce CureWM, which constructs alternative actions from successful demonstrations across a severity grid, verifies their outcomes through execution in simulation or on hardware, and fine-tunes released models on the resulting failures and surviving successes alongside nominal demonstrations. This construction provides controlled action contrasts from shared starting contexts. On 484 held-out LIBERO failure counterfactuals, optimism falls from 80% after fine-tuning on the official data to 30--43% across four independently fine-tuned CureWM models (38% mean). In two separate evaluations on a physical robot arm, failure predictions scored as success-like by a latent-distance diagnostic decrease from 90% after fine-tuning on successful demonstrations alone to 33% with CureWM. With failure counts per task, successful replay data, and training budget matched, counterfactual failures yield a success--failure value gap of 0.124, compared with 0.014 for freshly collected on-policy failures. These findings support execution-verified counterfactual replay for post-hoc repair and show why reduced optimism must be evaluated alongside success--failure discrimination. Code is available at https://github.com/jiuyixu25/CureWM.

Jiuyi Xu, Xiao Hu, Meida Chen +3

Colorado School of Mines · Northeastern University · Institute for Creative Technologies, University of Southern California +1

World Models for Robotics Action-Conditioned World Models World Model Evaluation Counterfactual Learning

Ask PDF

Oct 5, 2026

Future Anchored Verification and Online Recovery for World Action Models

World action models (WAMs) have emerged as a promising paradigm for robotic manipulation. They act by first predicting how a task should be performed and then decoding the actions from that future. However, the remaining actions are invalid once execution drifts from the prediction. Simply replanning from the already out of distribution state rarely restores what the task still requires; existing execution monitors decide when to stop, but not what to restore. We observe that the answer is already in hand: the future the WAM predicted before acting depicts exactly the states it intended to pass through. We introduce FAVOR (Future Anchored Verification and Online Recovery), a lightweight framework that keeps these predicted frames as anchors and uses them for verification and recovery. An Anchor Verifier compares each observation with its anchor, together with the executed actions, to flag deviations that break the task. Anchor-Guided Recovery uses a vision-language model to turn the flagged anchor into a short corrective instruction. Under strengthened instruction guidance, the WAM executes this instruction to return to the intended future. It then resumes the task. FAVOR raises the task success of the base WAM from 97.85% to 98.10% on LIBERO and from 72.60% to 72.98% on LIBERO-Plus without modifying the policy.

Zhibin Qin, Zhenxiong Tan, Xinchao Wang

National University of Singapore

World Models for Robotics Robotics World Action Models

Ask PDF

Abstract

Robot Failure Recovery

Explore similar work

EV-WM: Event-Verified World Models for Long-Horizon Robotic Manipulation

World Models Dream of Success: Diagnosing and Repairing Failure Insensitivity in Robot World Models

Future Anchored Verification and Online Recovery for World Action Models