Error detection is crucial in industrial training, healthcare, and assembly quality control. Most existing work assumes a single-view setting and cannot handle the practical case where a third-person (exo) demonstration is used to assess a first-person (ego) imitation. We formalize Ego→Exo Imitation Error Detection: given asynchronous, length-mismatched ego and exo videos, the model must localize procedural steps on the ego timeline and decide whether each is erroneous. This setting introduces cross-view domain shift, temporal misalignment, and heavy redundancy. Under a unified protocol, we adapt strong baselines from dense video captioning and temporal action detection and show that they struggle in this cross-view regime. We then propose SAVA-X, an Align-Fuse-Detect framework with (i) view-conditioned adaptive sampling, (ii) scene-adaptive view embeddings, and (iii) bidirectional cross-attention fusion. On the EgoMe benchmark, SAVA-X consistently improves AUPRC and mean tIoU over all baselines, and ablations confirm the complementary benefits of its components. Code is available at https://github.com/jack1ee/SAVAX.
Figures & tables
Method
AUPRC on Validation
AUPRC on Test
0.3
0.5
0.7
Mean
tIoU
0.3
0.5
0.7
Mean
tIoU
Dense Video Captioning (DVC) baselines
PDVC [ 57 ]
28.21
20.48
7.95
18.88
58.58
25.74
18.08
4.79
16.20
57.98
Exo2EgoDVC [ 35 ]
31.33
20.27
7.49
19.69
59.06
26.26
16.30
5.42
15.99
58.15
Temporal Action Localization (TAL) baselines
ActionFormer [ 63 ]
31.37
15.41
2.63
16.47
48.89
26.96
12.88
2.40
14.08
48.25
Table 1: Comparison on EgoMe validation and test split. Left: results on validation set . Right: results on the test set . We report AUPRC for the error class at multiple tIoU thresholds (0.3, 0.5, 0.7), their mean, and standalone temporal IoU (tIoU) for localization quality.
Modules
AUPRC
tIoU
AS
SVE
BiX
0.3
0.5
0.7
Mean
28.21
20.48
7.95
18.88
58.58
✓
30.90
22.60
9.21
20.90
58.88
✓
31.64
22.87
9.37
21.29
59.27
✓
33.08
21.86
8.23
21.06
58.27
✓
✓
30.89
24.26
10.32
21.82
58.96
Table 2: Ablation on EgoMe validation split without a variant label column. AS: Adaptive Sampling; SVE: Scene-Adaptive View Embedding; BiX: Bidirectional Cross-Attention Fusion.
Variant
Input frame rate (fps)
Avg
1
5
10
20
w/o AS
20.48
20.92
19.96
21.09
20.61
w AS
22.15
22.20
23.74
21.45
22.39
w AS, Lvic , Lsel
23.51
22.60
22.63
22.65
22.85
Table 3: Results of the AS at different input frame rates and ablation of the regularization term. Metric is [email protected].
Fusion Variant
AUPRC
tIoU
0.3
0.5
0.7
Mean
Simple concatenation
Concat (Channel)
28.21
20.48
7.95
18.88
58.58
Concat (Time)
28.91
21.05
7.85
19.27
58.15
Attention Type
BiX
33.08
21.86
8.23
21.06
58.27
Table 4: Fusion variants on EgoMe validation split. Concat (Channel) concatenates Ego/Exo along the feature channel; Concat (Time) concatenates along the temporal axis. BiX denotes bidirectional cross-attention fusion; BiX-Deformable replaces cross-attention with deformable attention. The last block ablates the bidirectional mechanism into single-direction flows.
Method
AUPRC on Validation
AUPRC on Test
0.3
0.5
0.7
Mean
tIoU
0.3
0.5
0.7
Mean
tIoU
Dense Video Captioning (DVC) baselines
PDVC [ 57 ]
68.41
46.21
12.72
42.45
58.58
66.53
43.88
12.30
40.90
57.98
Exo2EgoDVC [ 35 ]
67.52
44.52
12.43
41.49
59.06
65.37
42.27
11.40
39.68
58.15
Temporal Action Localization (TAL) baselines
ActionFormer [ 63 ]
65.87
31.68
4.54
34.03
48.89
63.25
29.20
4.17
32.20
48.25
Table A.1: Comparison on EgoMe validation and test split. Left: results on validation set . Right: results on the test set . We report AUPRC for the correct class at multiple tIoU thresholds (0.3, 0.5, 0.7), their mean, and standalone temporal IoU (tIoU) for localization quality.
Egocentric video understanding studies how AI interprets first-person video to recognize actions, track interactions, reason about space and time, and assist users.