World models and video understanding models are increasingly deployed in dynamic visual environments, yet their robustness to physical adversarial threats remains largely underexplored. We study whether a single physical adversarial texture can mislead the victim model across different real-world scenarios where observing viewpoints vary along uncontrolled object--camera trajectories. We identify that heterogeneous video semantics and trajectories induce poorly aligned gradients for optimizing the adversarial texture. To address that, we propose the Trajectory-aware adversarial Alignment and Refinement Attack (TARA), which incorporates trajectory-aware optimization to capture the evolving geometric variations of a printed texture under real-world scenarios. To improve cross-instance optimization compatibility, TARA incorporates trajectory-aware contrastive alignment to encourage attacked representations from diverse video--trajectory instances to approach a shared adversarial region and then performs trajectory-aware task refinement to translate the aligned representation into task-level prediction errors. For reproducible evaluation, we develop a geometry-grounded tracking pipeline that provides physically plausible adversarial regions with temporally consistent trajectories on EPIC-KITCHENS-100 and Something-Something V2. We further collect real-world print-and-recapture videos to validate attack effectiveness under physical deployment conditions. Together, the annotated benchmarks and real-world captured data form a reproducible testbed for future evaluation. Notably, our results reveal that a single printable adversarial texture can persistently disrupt world models and video understanding models under physical conditions, highlighting the fundamental security challenge.
Figures & tables
Figure 1: Illustration of physical adversarial texture on action anticipation under various trajectories.
Figure 2: Illustration of the observation trajectory of adversarial texture.
Figure 3: Analysis of trajectory-aware contrastive alignment.
Figure 4: Overall framework of TARA. A black texture is rendered along the video-trajectory instance and optimized through trajectory-aware contrastive alignment and task refinement.
Dataset
Task
Clips
Texture Train/Eval.
Annotated Frames
Homography Coverage
Mask Rate
Texture Area
Texture Motion
EK100
Anticipation
605
356 / 249
19,251
73.94%
7.56%
1.95%
20.65%
SSV2
Recognition
328
184 / 144
10,204
89.78%
9.70%
2.37%
7.66%
Real-world
Recog. / Antic.
348
0 / 348
8,645
81.27%
4.28%
2.97%
15.17%
Table 1: Statistics of the geometry-grounded trajectory benchmarks.
Figure 5: Distribution of surface-observation trajectories across the evaluation benchmarks.
Models
V-JEPA2-L
V-JEPA2.1-B
AVT-B
Overall
Setting
Train
Val
Train
Val
Train
Val
Avg. Train
Avg. Val
Black texture
4.8
3.3
5.7
5.0
8.2
5.0
6.2
4.4
Random texture
5.5
4.7
8.9
5.5
7.4
6.1
7.3
5.4
EOT
30.3
25.1
59.3
57.1
32.9
25.0
40.8
35.7
Patch-Fool
6.5
9.0
46.9
27.9
28.2
18.3
27.2
18.4
UPA-RFAS
9.5
10.4
17.1
15.7
17.6
18.3
14.7
14.8
Table 2: Comparison performance on EK100 future action anticipation. Top1 ASR (%, ↑ ) is reported on our split Train and Val data, with best and second-best results highlighted in bold and underlined.
Models
V-JEPA2-L
VideoMAE-B
InternVideo2
Overall
Setting
Train
Val
Train
Val
Train
Val
Avg. Train
Avg. Val
Black texture
9.2
5.6
5.7
10.8
4.0
2.9
6.3
6.4
Random texture
7.4
4.9
1.1
1.8
2.1
0.3
3.5
2.3
EOT
25.5
30.6
14.9
16.9
15.8
13.2
18.7
20.2
Patch-Fool
12.5
8.3
13.7
16.9
5.1
2.2
10.4
9.1
UPA-RFAS
9.8
3.5
14.3
16.9
11.9
9.6
12.0
10.0
Table 3: Comparison performance on SSV2 video action recognition. Top1 ASR (%, ↑ ) is reported on our split Train and Val data, with best and second-best results highlighted in bold and underlined.
Train
Val
Method
PSNR ↓
MSE (×10−3)↑
PSNR ↓
MSE (×10−3)↑
Clean
17.37
24.42
17.19
24.33
Black patch
16.48
27.70
16.51
26.74
Random patch
17.10
24.88
17.12
24.37
EOT
16.30
29.96
16.25
29.03
Patch-Fool
15.94
33.83
16.41
28.48
Table 4: Attack performance on SSV2 future frame generation using DiLA.
Tasks
Action Anticipation
Action Recognition
Models
V-JEPA2-L
V-JEPA2.1-B
V-JEPA2-L
InternVideo2
EOT
29.7
58.8
21.2
40.3
Patch-Fool
17.6
23.5
10.6
35.5
UPA-RFAS
18.9
17.6
11.8
43.5
BTC-UAP
14.9
17.6
12.9
24.2
PhyCamo
13.5
29.4
17.6
38.7
Table 5: Performance on real-world captured data. The first six rows are digital projection results using recorded homographies and masks; the last two rows are physical print-and-recapture results.
Figure 6: Performance of TARA under varying levels of simulated print-and-capture distortions.
Components
V-JEPA2-L
V-JEPA2.1-B
AVT-B
Overall
TA
CA
TR
Train
Val
Train
Val
Train
Val
Avg. Train
Avg. Val
×
✓
✓
10.4
10.9
8.0
5.7
9.4
3.3
9.3
6.6
✓
×
✓
32.9
22.7
68.6
58.3
40.0
28.3
47.2
36.4
✓
✓
×
29.0
22.7
64.0
65.0
36.5
30.0
43.2
39.2
✓
✓
✓
37.7
26.1
70.6
67.1
37.6
30.0
48.6
41.1
Table 6: Ablation study of our TARA on the EK100 dataset. Top-1 ASR (%, ↑ ) is reported. TA, CA, and TR denote trajectory awareness, contrastive alignment, and task refinement, respectively.
Figure 7: Visualization of TARA texture. The top row shows geometry-grounded texture rendering on EK100 and SSV2 videos, and the bottom row presents physical print-and-recapture examples.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Visualization of DiLA reconstruction results under TARA. Each column is a different validation clip.
Seen Content
Unseen Content
Method
Seen Traj.
Unseen Traj.
Seen Traj.
Unseen Traj.
Static center
17.6
17.6
13.5
9.5
EOT
32.4
24.3
32.4
16.2
PhyCamo
17.6
20.3
14.9
12.2
Patch-Fool
12.2
13.5
10.8
10.8
UPA-RFAS
13.5
13.5
9.5
5.4
Appendix
Table 7: Generalization across video content and observation trajectories on EK100 with V-JEPA2-L. Top-1 ASR (%, ↑ ) is reported on 74 eligible clip instances per condition.
Dataset / Model
Seed
Train
Val
Overall
EK100 / V-JEPA2-L
0 (reported)
37.7
26.1
32.1
1
37.9
26.1
32.2
2
36.8
24.6
31.0
Mean ± SD
37.5±0.6
25.6±0.9
31.8±0.7
SSV2 / V-JEPA2-L
0 (reported)
38.6
34.7
36.9
1
38.4
34.0
36.5
Appendix
Table 8: Random-seed stability of TARA. Final-epoch Top-1 ASR (%). Mean and sample SD are computed across all three seeds; seed 0 is also reported in the main results.
Figure 9: Visualization of adversarial textures under different imaging distortions on EK100 and SSV2. Patch and capture blur substantially suppress fine-grained texture details.
Figure 10: Additional trajectory visualizations across digital and physical settings. The first and third rows show digitally projected textures on EK100 and SSV2, respectively, while the second and fourth rows show real-world print-and-recapture recordings with physically attached textures.
Table 10: Geometry-estimation settings for newly annotated trajectories.
Figure 11: Human-guided surface-trajectory construction in an EK100 clip. (1) Manual attachment-region selection. (2) Homography estimation from tracked points. (3) Visible-support estimation under foreground occlusion. (4) Flagging of an inconsistent projection for human review.
Figure 12: Visualization of the clips in action anticipation. The first two samples are rendered from the EK100 dataset, and the last two samples are our real-world print-and-capture data.
Figure 13: Visualization of the clips in action recognition. The first two samples are rendered from the SSV2 dataset, and the last two samples are our real-world print-and-capture data.