Organizations: Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
Recent advances in Vision-Language-Action (VLA) models have enabled robots to execute increasingly complex tasks. However, VLA models trained through imitation learning struggle to operate reliably in dynamic environments and often fail under Out-of-Distribution (OOD) conditions. To address this issue, we propose Robot-Conditioned Normalizing Flow (RC-NF), a real-time monitoring model for robotic anomaly detection and intervention that ensures the robot's state and the object's motion trajectory align with the task. RC-NF decouples the processing of task-aware robot and object states within the normalizing flow. It requires only positive samples for unsupervised training and calculates accurate robotic anomaly scores during inference through the probability density function. We further present LIBERO-Anomaly-10, a benchmark comprising three categories of robotic anomalies for simulation evaluation. RC-NF achieves state-of-the-art performance across all anomaly types compared to previous methods in monitoring robotic tasks. Real-world experiments demonstrate that RC-NF operates as a plug-and-play module for VLA models (e.g., pi0), providing a real-time OOD signal that enables state-level rollback or task-level replanning when necessary, with a response latency under 100 ms. These results demonstrate that RC-NF noticeably enhances the robustness and adaptability of VLA-based robotic systems in dynamic environments.
Figures & tables
Figure 2: Overview of our framework. Our Robot-Conditioned Normalizing Flow (RC-NF) operates as a real-time runtime monitor for robotic manipulation tasks. (Left) SAM2 extracts object masks from video streaming, which are then grid-sampled into point sets. Task prompts are encoded using spherical uniform encoding, and robot proprioception provides joint, gripper, and pose states. (Center) Our RC-NF leverages these signals as conditions within the affine coupling layers in the proposed RCPQNet (Sec . 3.3 ) to apply K invertible transformations and compute anomaly scores for the current task. (Right) When the anomaly score exceeds the threshold, the Anomaly Detection and Handling module triggers corrective behaviors by task replanning for task-level OOD and task rollback for state-level OOD.
Figure 3: Robot-Conditioned Point Query Network (RCPQNet) is introduced as the affine coupling layer in RC-NF that generates shift and scale parameters.
Method
Gripper Open
Gripper Slippage
Spatial Misalignment
Average
AUC
AP
AUC
AP
AUC
AP
AUC
AP
GPT-5 [ 29 ]
0.9137
0.9642
0.8941
0.8720
0.4904
0.4015
0.8500
0.8507
Gemini 2.5 Pro [ 8 ]
0.8644
0.9333
0.8633
0.8505
0.5167
0.4271
0.8186
0.8313
Claude 4.5 [ 4 ]
0.8754
0.9401
0.8551
0.8285
0.5292
0.4290
0.8214
0.8249
FailDetect [ 47 ]
0.7883
0.9032
0.6665
0.6932
0.6557
0.5820
0.7181
0.7700
RC-NF(Ours)
0.9312
0.9781
0.9195
0.9180
0.9676
0.9585
0.9309
0.9494
Table 1: Quantitative comparison of anomaly detection performance on the LIBERO-Anomaly-10 benchmark.
Figure 4: Visualization of the anomaly scores for the task Pick up the book and place it in the back compartment of the caddy . The x-axis denotes the time steps, and the y-axis indicates the anomaly score. The red dashed line marks tanomaly .
Row
Method
Gripper Open
Gripper Slippage
Spatial Misalignment
Average
AUC
AP
AUC
AP
AUC
AP
AUC
AP
1
RC-NF(Ours)
0.9312
0.9781
0.9195
0.9180
0.9676
0.9585
0.9309
0.9494
2
w/o Task Embedding
0.8769
0.9603
0.8668
0.8680
0.8139
0.8118
0.8643
0.9008
3
w/o Robot State
0.6327
0.8621
0.7443
0.8116
0.8929
0.8617
0.7152
0.8401
4
w/o Pos. Residual branch
0.9045
0.9712
0.8971
0.9085
0.8543
0.8072
0.8947
0.9225
5
w/o Dyn. Shape branch
0.7666
0.9234
0.7763
0.8108
0.1022
0.2755
0.6841
0.7899
Table 2: Ablation study of the proposed Robot-Conditioned Point Query Network (RCPQNet). RC-NF (Ours) denotes the full model.
Figure 5: A real-world comparison of π0 and π0 + RC-NF (ours) during the task placing the ball into the open drawer , when the drawer closes midway.
Figure 6: Comparison of π0 + RC-NF (Ours) and π0 model in handling two consecutive task-level OOD events during real-world deployment. t1a represents the moment when the ball is repositioned on the table. t2a marks when the ball rolls behind the gripper. t3r indicates when control is returned to the VLA after homing. The timeline of π0 + RC-NF (Ours) is shown in Fig. 7 .
Figure 7: Anomaly score plot for a state-level OOD sequence where the ball is repositioned at t1a and rolls behind the gripper at t2a . The yellow curve indicates the detection of an anomaly, with the trajectory corrected by the homing procedure. The green curve represents the operation of the VLA model π0 .
SAM2 Inference
Grid Sampling
RC-NF Inference
Other Latency
Overall (RTX 3090)
50 ms
1.7 ms
30 ms
5 ms
86.7 ms
Table 9
Figure 8: Diagram illustrating Spherical Uniform Encoding. The left side displays the ten tasks from the training set along with their descriptions, while the right side provides an example demonstration.
Figure 9: Comparison of performance metrics before and after incorporating three tasks with spatial misalignment (left, front and right) into the training set.
Figure 10: Prompts for generating bounding boxes with Gemini 2.5 Pro
Figure 11: VLMs prompts for robotic anomaly detection scoring
Vision-Language-Action (VLA) models have recently advanced robotic manipulation by translating natural-language instructions and visual observations into control actions. However, existing VLAs are primarily trained on successful expert demonstrations and lack structured supervision for failure diagnosis and recovery, limiting robustness in open-world scenarios. To address this limitation, we propose the Robotic Failure Analysis and Correction (RoboFAC) framework. We construct a large-scale failure-centric dataset comprising 9,440 erroneous manipulation trajectories and 78,623 QA pairs across 53 scenes in both simulation and real-world environments, with systematically categorized failure types. Leveraging this dataset, we develop a lightweight multimodal model specialized for task understanding, failure analysis, and failure correction, enabling efficient local deployment while remaining competitive with large proprietary models. Experimental results demonstrate that RoboFAC achieves a 34.1% higher failure analysis accuracy compared to GPT-4o. Furthermore, we integrated RoboFAC as an external supervisor in a real-world VLA control pipeline, yielding a 29.1% relative improvement across four tasks while significantly reducing latency relative to GPT-4o. These results demonstrate that RoboFAC enables systematic failure diagnosis and recovery, significantly enhancing VLA recovery capabilities. Our model and dataset are publicly available at https://github.com/MINT-SJTU/RoboFAC.
Zewei Ye, Weifeng Lu, Minghao Ye +4
School of AI, Shanghai Jiao Tong University · Harbin Institute of Technology, Shenzhen
Robust robotic manipulation requires not only predicting how the scene evolves over time, but also recognizing task-relevant objects in complex scenes. However, existing VLA models face two limitations. They typically act only on the current frame, while future prediction and object-aware reasoning are often learned in separate latent spaces. We propose OFlow (injecting Object-Aware Temporal Flow Matching into VLAs), a framework that addresses both limitations by unifying temporal foresight and object-aware reasoning in a shared semantic latent space. Our method forecasts future latents with temporal flow matching, factorizes them into object-aware representations that emphasize physically relevant cues while filtering task-irrelevant variation, and conditions continuous action generation on these predictions. By integrating OFlow into VLA pipelines, our method enables more reliable control under distribution shifts. Extensive experiments across LIBERO, LIBERO-Plus, MetaWorld, and SimplerEnv benchmarks and real-world tasks demonstrate that object-aware foresight consistently enhances robustness and success.
Kuanning Wang, Ke Fan, Chenhao Qiu +5
Fudan University Shanghai, China · University of Southern California Los Angeles, CA, USA · KAUST Saudi Arabia
Vision-language-action (VLA) models have emerged as generalist robotic controllers capable of mapping visual observations and natural language instructions to continuous action sequences. However, VLAs provide no calibrated measure of confidence in their action predictions, thus limiting their reliability in real-world settings where uncertainty and failures must be anticipated. To address this problem we introduce ReconVLA, a reliable conformal model that produces uncertainty-guided and failure-aware control signals. Concretely, our approach applies conformal prediction directly to the action token outputs of pretrained VLA policies, yielding calibrated uncertainty estimates that correlate with execution quality and task success. Furthermore, we extend conformal prediction to the robot state space to detect outliers or unsafe states before failures occur, providing a simple yet effective failure detection mechanism that complements the action-level uncertainty. We evaluate ReconVLA in both simulation and real robot experiments across diverse manipulation tasks. Our results show that conformalized action predictions consistently improve failure anticipation, reduce catastrophic errors, and provide a calibrated measure of confidence without retraining or modifying the underlying VLA.
Lingling Chen, Zongyao Lyu, William J. Beksi
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, USA