Detecting Silent Action-Interface Mismatch with Calibrated Relational World Models

Rui Zhu; Xurui Li; Haixu Tang

cs.RONov 4, 2026

Detecting Silent Action-Interface Mismatch with Calibrated Relational World Models

Authors: Rui Zhu, Xurui Li, Haixu Tang

Organizations: Yale University · Ant Group · Indiana University

Abstract

A robot action can satisfy shape and range constraints yet carry the wrong semantics: axes may be permuted, signs inverted, rotations rescaled, or gripper polarity reversed. We study learned state-action-effect relations as offline contracts for detecting these mismatches and ranking their likely fields. Clean held-out transitions calibrate alarms; a fixed inverse vocabulary supports closed-set localization. On ten LIBERO Spatial tasks comprising 432 source episodes, a capacity-controlled comparison at 2.5% empirical clean false-positive rate gives 46.1% recall for a ridge forward residual, 61.0% for a four-component joint Gaussian mixture, and 65.7% for the exact conditional of that same mixture. The joint score retains a 14.9 percentage-point advantage over ridge, while conditioning improves recall by a further 4.7 points. Joint scoring instead has higher localization: 95.8% versus 94.6%. These matched-rate comparisons are retrospective diagnostics, not deployment calibration guarantees. Earlier Object/Goal results broaden tasks within the LIBERO convention; DROID, all three HYDRA tasks, short temporal faults, small offsets, exact repair, and simulated mitigation fail their respective gates, while xArm contributes one task only. The evidence supports a bounded offline diagnostic and a detection-localization tradeoff, rather than joint-density superiority or cross-robot safety.

Figures & tables

Score	Matched recall (%)	Calibration FPR (%)	Localization (%)
Ridge forward	46.08 [41.58, 50.17]	2.63	92.70
Joint GMM	60.96 [56.90, 64.28]	3.22	95.81
Conditional GMM	65.66 [62.61, 68.40]	2.85	94.62

Table 1: Post-decision Spatial comparison over 432 source episodes. Recall is at 2.5% matched empirical clean FPR; brackets are conditional paired task/episode bootstrap 95% intervals. Achieved FPR uses a separate, calibration-only 2.5% nominal threshold and equal-episode weighting. Matched FPR uses clean rows within each split. Localization includes all valid fault rows, not just alarms.

Figure 1: Retrospective clean-reference FPR comparison on the same Spatial episodes. All methods use the same train/calibration/test partitions. Bands are pointwise 95% task/episode bootstrap intervals conditional on fitted scores and reference thresholds. They are not simultaneous bands or deployment-FPR guarantees.

Scope	Gate	Result	Supported interpretation
Small X-offset	0/10	GMM recall 0.0340	Explanation may rank well below alarm threshold
Temporal faults	0/10	median gain 0.0421	Short latency/hold/chunk order unsupported
DROID-100	fail	recalls 0.0193–0.0865	Structural compatibility is not detector transfer
xArm Lift	1 task	GMM recall 0.6016	One-task offline sanity only
HYDRA	0/3	recalls 0.1063–0.1213	Multi-task external convention fails
Exact repair	0/10	median within-0.01 0.4930	Localization is not reliable action repair

Table 2: Material boundaries retained in the main paper. “Pass” always refers to the frozen gate for that row; proxy and simulator rows are not physical trials.

Level	Statistical unit	May support
Fresh locked replay	task summaries over unseen split seeds	Detector ordering and clean-FPR compliance on recorded tasks
Clustered confirmation	task, then immutable source episode	Finite-source Spatial effect and paired uncertainty
Selection-frozen reuse	previously used task–episodes	Mechanism, heterogeneity, and failure description only
Post-hoc severity	task summaries at fixed severities	Explanatory sensitivity, not rescue of a failed family
External convention	source-defined episodes and tasks	Transfer only when the independently frozen gate passes
Simulator proxy	simulator episodes under matched budgets	Only the specified mitigation comparison, never physical safety

Table 1: Evidence hierarchy. “May support” is deliberately narrower than what the underlying metric could be used to suggest.

Suite	Tasks	Transitions	Task–episodes	Episodes/task
LIBERO Spatial	10	52,538	432	35–47
LIBERO Object	10	56,262	383	31–41
LIBERO Goal	10	51,614	428	33–50
Total	30	160,414	1,243	31–50

Table 2: Immutable source units. A task–episode is the inferential cluster; a transition is not treated as an independent trial.

Status	Family	Exact transformation
Supported	Motion scale	$a_{1:6}\leftarrow 0.5a_{1:6}$
Supported	XY permutation	$(a_{x},a_{y})\leftarrow(a_{y},a_{x})$
Supported	X sign	$a_{x}\leftarrow-a_{x}$
Supported	Rotation units	$a_{4:6}\leftarrow a_{4:6}/57.2957795$
Supported	Gripper polarity	$a_{7}\leftarrow 1-a_{7}$
Stress test	Fixed X offset	$a_{x}\leftarrow a_{x}+0.05$

Table 3: Frozen mutation vocabulary. Motion transformations act on the first six coordinates unless a smaller coordinate set is named.

Task	$\Delta R$	GMM loc.	$\Delta L_{f}$	$\Delta L_{j}$
0	0.1605	0.9632	0.0281	0.0151
1	0.2431	0.9656	0.0232	0.0172
2	0.1555	0.9577	0.0310	0.0181
3	0.2653	0.9466	0.0414	0.0287
4	0.0826	0.9417	0.0305	0.0197
5	0.1035	0.9650	0.0315	0.0129

Table 4: Fresh Spatial task results for the four-component GMM.

\Delta R

is GMM-minus-forward recall;

\Delta L_{f}

and

\Delta L_{j}

are localization gains over forward and joint Gaussian.

Effect	Estimate	Paired 95% CI	FWER lower	Holm $p$
GMM $-$ forward recall	0.16753	[0.13642, 0.20297]	0.13401	0.00293
GMM $-$ forward loc.	0.03089	[0.02679, 0.03541]	0.02645	0.00293
GMM $-$ joint loc.	0.01837	[0.01522, 0.02198]	0.01497	0.00293

Table 5: Fresh 432-episode Spatial effects. The sign-flip family has ten paired task summaries, so its smallest attainable unadjusted one-sided value is 1/1024.

Family	Recall effect	Loc. effect	Interpretation
Scale	0.2620	0.0098	Forward wins both
XY permutation	0.0353	-0.0248	Recall–location tradeoff
X sign	-0.0242	-0.0277	Joint wins both
Rotation units	0.0990	-0.0168	Only family passing Pareto gate
Gripper polarity	0.1054	-0.0036	Recall–location tradeoff

Table 6: Forward-minus-joint Gaussian effects by transformation family. Positive localization favors forward; negative localization favors joint.

Factor	Setting	Recall gain	95% interval	Loc.	Max FPR
Components	2	0.0725	[0.0407,0.1149]	0.9561	0.0443 (fail)
	4	0.1609	[0.1264,0.2049]	0.9585	0.0458
	8	0.1634	[0.1274,0.2118]	0.9566	0.0424
Calibration cap	100	0.1453	[0.1194,0.1892]	0.9607	0.0412
	200	0.1540	[0.1269,0.1983]	0.9607	0.0353
	400	0.1482	[0.1271,0.2024]	0.9607	0.0362

Table 7: GMM sensitivity. Intervals apply to GMM-minus-forward recall; FPR is the largest task value.

Source	Units	Detector result	Classification
DROID-100	100 ep.; pooled	scale/swap/sign recall 0.019/0.078/0.087	fail transfer; sparse identity
xArm Lift	800 ep.; 1 task	GMM/forward recall 0.602/0.505; loc. 0.929; FPR 0.024	One-task offline sanity
HYDRA	570 ep.; 3 tasks	GMM recall 0.106/0.110/0.121; loc. 0.728–0.792	0/3, fail transfer

Table 8: External-convention outcomes. All use low-dimensional source-semantic actions; video and simulator assets are excluded.

ID	Role	Wall (s)	Max RSS (bytes)
042b	fresh GMM recall	465.445	220,004,352
043	fresh localization	225.377	250,036,224
044	Object/Goal GMM	43.810	960,430,080
051	static metadata baseline	51.542	241,270,784
053a	split coverage	0.899	42,762,240
054a	clustered effects	46.422	588,038,144

Table 9: Exact CPU resource evidence for the ten retained main-evidence paths.

Claim	Command/script locator	Result locator
GMM recall	run-gmm-confirmation-042b.sh	gmm-joint-confirmation-042b.json
GMM localization	run-gmm-localization-043.sh	gmm-localization-confirmation-043.json
Suite repeat	run-cross-suite-gmm-044.sh	gmm-cross-suite-confirmation-044.json
Static comparator	run-metadata-displacement-baseline-051.sh	metadata-displacement-baseline-051.json
Split coverage	run-statistical-unit-audit-053a.sh	statistical-unit-audit-053a.json
Clustered effects	run-clustered-effect-intervals-054a.sh	clustered-effect-intervals-054a.json

Table 10: Key evidence locators. Paths are relative to the private project root.

FPR (%)	Ridge forward	Joint GMM	Conditional GMM
1.00	34.13	52.27	56.49
2.50	46.08	60.96	65.66
5.00	58.41	68.62	72.83

Table 11: Matched empirical clean FPR: mutation recall in percent. All rows use the same finite source episodes, so different FPRs are correlated descriptive views.

Score	Nominal 1%	Nominal 2.5%	Nominal 5%
Ridge forward	1.16 / 31.34	2.63 / 44.13	5.05 / 57.26
Joint GMM	1.71 / 51.09	3.22 / 60.86	5.62 / 68.50
Conditional GMM	1.34 / 54.76	2.85 / 65.04	5.32 / 72.64

Table 12: Calibration-only operating points in percent. Each cell gives achieved clean FPR / mutation recall; calibration targets are nominal, not achieved rates.

Comparison	Effect	95% interval	97.5% interval
Joint $-$ ridge	14.88	[12.07, 17.98]	[11.72, 18.43]
Joint $-$ conditional	-4.70	[-6.32, -3.50]	[-6.58, -3.39]

Table 13: Primary matched-FPR paired effects, in percentage points. The wider intervals control the two declared comparisons by Bonferroni under the stated conditional bootstrap interpretation.

Spatial task	Episodes	Ridge	Joint	Conditional
0	46	47.98	62.57	67.76
1	42	45.12	66.84	69.45
2	39	45.59	61.01	65.07
3	35	30.80	54.84	57.54
4	45	39.22	46.67	57.66
5	43	58.18	67.80	72.93

Table 14: Task-level recall at matched 2.5% empirical clean FPR, in percent. Task means equal-weight their source episodes and five represented families.

Evaluation	Pass	Gain vs. action/range	Loc.	Max FPR
Spatial, 2,000 seeds/task	10/10	0.3157	0.8567	0.0279
Object + Goal, 500/suite	2/2	0.2989	0.8474	0.0286
Five-family Spatial, 4,000/task	10/10	0.3793	0.9134	0.0279
Five-family Object + Goal, 4,000/suite	2/2	0.3579	0.8993	0.0284
Forward Spatial, fresh 2,000/task	10/10	0.3739	0.9285	0.0287
Forward Object + Goal, fresh 2,000/suite	2/2	0.3474	0.9092	0.0279

Table 15: Locked replay evidence. Gains are absolute recall differences. “Seeds” are repeated episode-split seeds over fixed data, not independent trajectories.

Evidence	Comparison	Effect	95% interval
Spatial splits	GMM $-$ forward recall	0.15663	task median
Spatial splits	GMM $-$ forward loc.	0.02932	[0.02472, 0.03553]
432 episodes	GMM $-$ forward recall	0.16753	[0.13642, 0.20297]
432 episodes	GMM $-$ forward loc.	0.03089	[0.02679, 0.03541]
432 episodes	GMM $-$ joint loc.	0.01837	[0.01522, 0.02198]
Metadata baseline	metadata $-$ GMM recall	-0.31196	[-0.33328, -0.24729]

Table 16: Paired historical evidence for the four-component GMM. Achieved FPR is not matched. The clustered rows first average repeated split appearances within each source episode.

Robotic Anomaly Detection

See all 19 papers

Same Label, Different Interface: Beyond Action-Space Labels in Robot Learning
Oct 28, 2026Arshia Sangwan, Zhongyang ZhangRobot Policy Learning Robotic Control
New York University · University of California San Diego
Cross-Embodiment Robot Foundation World Models with Latent Actions
Oct 7, 2026Huang Huang, Sriram Yenamandra, Arjun Majumdar +9World Models for Robotics Action-Conditioned World Models
Department of Computer Science, Stanford University, US · Work partially done while at Meta FAIR Robotics.
Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models
Sep 11, 2026Jianman Lin, Shailesh Shailesh, Zhongyi Luo +1Robot Foundation Models Vision-Language-Action Models
School of Future Technology, South China University of Technology, Guangzhou, China · National University of Singapore, Singapore · Nanyang Technological University, Singapore

Oct 28, 2026J/K move · Enter open · S save

Same Label, Different Interface: Beyond Action-Space Labels in Robot Learning

Arshia Sangwan, Zhongyang Zhang

New York University · University of California San Diego

Robot Policy Learning Robotic Control Robotic Manipulation

Open paper AskPDF arXiv

Robot-learning studies often compare position and velocity actions as though each label identified a single experimental treatment, yet the label alone does not determine what the robot executes. Before affecting robot motion, bounded policy output passes through filtering, physical scaling, reference construction, feedback, timing, and effort limits. We study how these omitted properties affect attribution under one shared learner and training budget. Across three contact-rich Factory tasks (PegInsert, GearMesh, and NutThread), we compare five joint-space interfaces, with ten independently trained policies per condition. We first remain within the same position-feedback law and vary the executed interface: substituting only the physical action scale as the directly modified component changes sample-median evaluation success by 0.645-1.000 across tasks, while removing an exponential moving average produces task-dependent changes. We then move to a cross-law comparison and choose a velocity scale that exactly matches the position interface's local raw-torque command gain. This calibration raises the sample median on GearMesh and NutThread, with no median change on PegInsert. Yet clear residual sample-median gaps remain on PegInsert and GearMesh, while the NutThread interval includes zero. Thus, even this explicit equality leaves feedback and held-action dynamics unmatched and does not establish position-velocity trained-outcome equivalence. Action-interface comparisons should therefore report the executed pathway and state the physical quantity they match.

Abstract

Robotic Anomaly Detection

Explore similar work

Same Label, Different Interface: Beyond Action-Space Labels in Robot Learning

Cross-Embodiment Robot Foundation World Models with Latent Actions

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models