We present V-JEPA 2.1, a family of self-supervised models that learn dense, high-quality visual representations for both images and videos while retaining strong global scene understanding. The approach combines four key components. First, a dense predictive loss uses a masking-based objective in which both visible and masked tokens contribute to the training signal, encouraging explicit spatial and temporal grounding. Second, deep self-supervision applies the self-supervised objective hierarchically across multiple intermediate encoder layers to improve representation quality. Third, multi-modal tokenizers enable unified training across images and videos. Finally, the model benefits from effective scaling in both model capacity and training data. Together, these design choices produce representations that are spatially structured, semantically coherent, and temporally consistent. Empirically, V-JEPA 2.1 achieves state-of-the-art performance on several challenging benchmarks, including 7.71 mAP on Ego4D for short-term object-interaction anticipation and 40.8 Recall@5 on EPIC-KITCHENS for high-level action anticipation, as well as a 20-point improvement in real-robot grasping success rate over V-JEPA-2 AC. The model also demonstrates strong performance in robotic navigation (5.687 ATE on TartanDrive), depth estimation (0.307 RMSE on NYUv2 with a linear probe), and global recognition (77.7 on Something-Something-V2). These results show that V-JEPA 2.1 significantly advances the state of the art in dense visual understanding and world modeling.
Figures & tables
Figure 1: V-JEPA 2.1 unlocks high-quality dense features. We compute PCA on patch features extracted from the same image or video and map the top three components to RGB channels for both V-JEPA 2 (ViT-g) and V-JEPA 2.1 (ViT-G). Our novel V-JEPA 2.1 produces dense representations with strong spatial and temporal consistency, learning semantically coherent features where similar objects map to the same PCA components.
Figure 2: V-JEPA 2.1 ViT-G performance across dense and global prediction tasks. We show the relative improvements of V-JEPA 2.1 compared to the previous V-JEPA 2 ViT-g model ( Assran et al., 2025 ) . We also report the performance of previous SOTA models using a frozen backbone evaluation: DINOv3 ( Siméoni et al., 2025 ) is the reference model for depth estimation, object tracking, semantic segmentation, SSv2 action recognition, and image classification; InternVideo2s ( Wang et al., 2024b ) for K400 action recognition, STAformer ( Mur-Labadia et al., 2024 ) for short-term object interaction anticipation, and PlausiVL ( Mittal et al., 2024 ) for action anticipation. Tasks where V-JEPA 2.1 ViT-G obtains SOTA in frozen-backbone evaluation are underlined .
Figure 3: Influence of the context loss Lctx . We show PCA visualizations of the feature map representations learned with V-JEPA 2 and with a model trained using V-JEPA 2 plus our Lctx loss. While V-JEPA features only show fragmented local spatial structure, explicitly supervising unmasked regions with Lctx leads to feature maps that exhibit coherent spatial structure. Similar semantic parts (e.g., heads of dogs, wheels of cars) are mapped to the same PCA components.
Figure 4: V-JEPA 2.1 Detailed Architecture. Images and videos are processed by respectively either a 2D or 3D Convolutional patch embedding. Then, 3D Rotational Positional Encoding (RoPE) and learnable modality embedding are added. The x− encoder processes the visible tokens and outputs multi-level embeddings formed by concatenating the normalized output from intermediate encoder blocks. Then, a MLP fuses this multi-level representation and reduces its dimensionality. These context tokens are concatenated with learnable mask tokens carrying spatio-temporal positional information. The predictor processes the combined sequence and produces multi-level predictions for the masked tokens. Training uses two different losses: (i) an L1 loss on masked-token predictions (the original V-JEPA objective), and (ii) a distance-weighted L1 loss on nearby context tokens, both supervised using the y -encoder multi-level outputs.
Figure 5: Impact of individual components of our novel V-JEPA 2.1 training recipe. The ablation is conducted starting from a ViT-L architecture, on single-image semantic segmentation on ADE20k, and action classification on SSv2. Introducing Weighted-Context Self-Supervision with the context loss significantly improves segmentation, at the cost of classification. Deep Self-Supervision allows to retrieve the performance, and further improving segmentation. Scaling the image data with VisionMix 163M dataset, combined with a Multi-Modal Tokenizer further improve the results. Finally, the recipe scales with model size and high resolution cool-down .
Table 6
Source
Samples
Type
Total Hours
V-JEPA2 weights
V-JEPA2.1 weights
SSv2 ( Goyal et al., 2017 )
168K
EgoVideo
168
0.056
0.170
Kinetics ( Carreira et al., 2019 )
733K
ExoVideo
614
0.188
0.010
Howto100M ( Miech et al., 2019 )
1.1M
ExoVideo
134K
0.318
0.100
ImageNet ( Deng et al., 2009 )
1M
Images
n/a
0.250
YT-1B ( Zellers et al., 2022 )
19M
ExoVideo
1.6M
0.188
0.720
LVD-142M ( Oquab et al., 2023 )
142 M
Curated images
n/a
Table 3: Comparison between VideoMix22M and VisionMix163M. We increase the visual diversity of the pretraining dataset by increasing the number of images, replacing ImageNet with LVD-142M, and update the YT-1B sampling weight to promote data-source with more visual diversity.
Figure 6: Short-Term Anticipation task. Given a video observed up to time t (denoted V:t ), the model predicts a set of future object–interaction events. Each prediction includes: (i) a bounding box localizing the object that will be interacted with, (ii) the noun class of that object, (iii) a verb class describing the upcoming interaction, and (iv) the anticipation time δ , i.e., the number of seconds between the current frame and the start of the interaction. Ground-truth labels are derived from the contact frame Vt+δ , but are expressed in the coordinate frame of the last observed frame Vt .
Model
AP b
AP b+N
AP b+V
AP b+ δ
AP b+N+V
AP b+ δ +N
AP b+ δ +V
AP All
mAP N
mAP N+V
mAP N+ δ
mAP All
Results Reported in the Literature
StillFast ( Ragusa et al., 2023 )
-
-
-
-
-
-
-
-
20.2
10.3
7.17
3.96
STAformer ( Mur-Labadia et al., 2024 )
38.3
28.3
16.6
12.27
12.8
8.89
5.47
4.06
29.4
15.3
9.94
5.67
GANO ( Thakur et al., 2023 )
45.3
27.0
12.2
16.6
6.54
9.0
4.18
2.47
-
-
-
-
Encoders using our same protocol
DINOv2 ViT-L ( Oquab et al., 2023 )
45.1
37.8
21.9
14.4
17.6
10.6
7.27
5.25
28.3
15.2
9.22
5.25
Table 4: Short-Term Object Interaction Anticipation on Ego4D. Following Grauman et al. (2022) , we report different Top-5 Average Precision (AP) and mean Average Precision (mAP) metrics, where the winning score is the mAP All. We compare results reported in the literature with different frozen SSL encoders (DINOv2, DINOv2, V-JEPA 2 and our V-JEPA 2.1) extended with an attentive-based predicted head. V-JEPA 2.1 obtains the state-of-the-art due to its high-quality dense features and predictive capabilities.
Figure 7: Short-Term Anticipation (STA) qualitative results. We compare our model predictions with the ground-truth data. V-JEPA 2.1 predicts multiple scenarios, where the predicted object and verb classes correspond with plausible human-object interactions.
Method
Param.
Action Anticipation
Verb
Noun
Action
InAViT ( Roy et al., 2024 )
160M
51.9
52.0
25.8
Video-LLaMA ( Zhang et al., 2023 )
7B
52.9
52.0
26.0
PlausiVL ( Mittal et al., 2024 )
8B
55.6
54.2
27.6
V-JEPA 2 ViT-g ( Assran et al., 2025 )
1B
63.6
57.1
39.7
V-JEPA 2.1 ViT-g
1B
63.6
56.2
38.4
Table 5: Action Anticipation on EK100. Comparison with the state-of-the-art on the EK100 Action Anticipation benchmark. We report mean-class recall-at-5 for verb, noun and action on the validation set of EK100. V-JEPA 2.1 offers comparable performance to V-JEPA 2 for the ViT-g 1B model, but shows improved performance for the ViT-G 2B model, setting a new state-of-the-art for the task.
Planning Details
Method
#Samples
Iter.
Horizon
Time
Reach
Grasp
Pick-&-Place
VJEPA 2 ( Assran et al., 2025 )
800
10
1
3 sec.
100%
60%
80%
VJEPA 2.1 (ours)
800
10
1
3 sec.
100%
70%
80%
300
15
8
14 sec.
100%
80%
80%
Table 6: Planning Performance. Comparing closed-loop robot manipulation of a cup using MPC with VJEPA 2 versus VJEPA 2.1. In both cases we leverage the cross-entropy method ( Rubinstein, 1997 ) for optimizing the sequence of actions using a single A100 GPU. For each robot skill, we evaluate each model across 10 tasks and average the results. The improved depth understanding and dense features of VJEPA 2.1 lead to a 20% improvement in the success rate of Grasp. Moreover, qualitatively, we find that any failures on Pick-and-Place and Grasp are a result of poor planning over gripper actions, as opposed to failures in spatial understanding (e.g., closing the gripper too soon, such that you cannot grasp the object, or slightly opening the gripper while in transit leading to the object being dropped.
Figure 8: Zero-shot evaluation of VJEPA 2 and VJEPA 2.1 models on a table-top Franka Panda robot arm with a parallel-jaw gripper, demonstrating Grasp of a cup using visual goal specification via model-predictive control. VJEPA 2.1 features better encode depth information, leading to an improvement in robot manipulation when grasping the cup involves reasoning over actions along the depth axis of the camera.
Figure 9: Planning Navigation Trajectories in Latent Space. V-JEPA 2.1 enables 10× faster and more accurate navigation planning compared to ( Bar et al., 2025 ) . We show PCA visualizations of 8 denoising steps of the planned latent trajectory between a start frame and a goal frame.
Model
Time
Recon
Tartan Drive
Scand
Sacson
Average
ATE
RTE
ATE
RTE
ATE
RTE
ATE
RTE
ATE
RTE
NWM ( Bar et al., 2025 )
103.2 (s)
1.146
0.339
5.831
1.219
1.113
0.297
4.037
0.928
3.032
0.696
V-JEPA 2.1 ViT-g
10.6 (s)
1.146
0.338
5.758
1.200
1.094
0.299
3.904
0.921
2.975
0.690
V-JEPA 2.1 ViT-G
10.6 (s)
1.179
0.340
5.687
1.187
1.038
0.285
4.054
0.938
2.990
0.688
Table 7: Open Loop Navigation Planning. We finetune V-JEPA 2.1 on robot navigation datasets and compare the open-loop planning performance to NWM. Reporting the average planning time (seconds), normalized Average Trajectory Error (ATE) and Relative Trajectory Error (RTE). V-JEPA 2.1 representation enables 10× speed-up without sacrificing performance.
Method
Param.
Depth
Segmentation
VOS
NYUv2 ↓
KITTI ↓
ADE20k
Citysc.
VOC
DAVIS-S
YT VOS-S
7B models
Web-DINO ( Fan et al., 2025 )
7B/14
0.466
3.158
42.7
68.3
76.1
57.2
43.9
DINOv3 ( Siméoni et al., 2025 )
7B/16
0.309
2.346
55.9
81.1
86.6
71.1
74.1
Models less than 2B parameters
PEcore ( Bolya et al., 2025 )
2B/14
0.590
4.119
38.9
61.1
69.2
48.2
34.7
Table 8: Performance on dense visual tasks. We report the Root Mean Squared Error (RMSE) for depth estimation (NYUv2, KITTI), mean Intersection-over-Union (mIoU) for semantic segmentation (ADE20K, Cityscapes, VOC12), and the J&F -Mean for video object segmentation (DAVIS, YouTube-VOS). Following Siméoni et al. (2025) , ADE20K and VOC12 are evaluated at an input resolution of 448×448 for models with patch size 14 and 512×512 for those with patch size 16. Cityscapes, NYUv2, and KITTI use their default evaluation resolutions. For DAVIS and YouTube-VOS, we evaluate with the shorter image side set to 480 px (patch size 16) or 420 px (patch size 14). V-JEPA 2.1 VIT G demonstrates strong performance on dense vision tasks. Notably, it achieves state-of-art performances on NYUv2 depth estimation, outperforming a DINOv3 backbone with 7 billion parameters.
Figure 10: Depth estimation comparison on NYU and KITTI datasets. While V-JEPA 2 captures the overall scene geometry, its predictions lack local consistency and precise boundary structure. In contrast, our V-JEPA 2.1 produces sharper, more coherent, and fine-grained depth maps.
Figure 11: Semantic segmentation qualitative results in VOC12 and Cityscapes datasets. The previous version, V-JEPA 2, produced sparse semantic masks due to the presence noisy feature maps. However, V-JEPA 2.1 achieves competitive performance with state of the art methods such as DINOv3 ViT-H+.
Figure 12: Qualitative comparative of V-JEPA 2.1 ViT-G vs DINOv3 ViT-H + . V-JEPA 2.1 captures fine-grained details (i.e the traffic light contour) and obtains better scale consistency. In the two rightmost columns, DINOv3 mislabel the depth of the green object on the bed, and the TV, and detect them closer to the camera than they are.
Figure 13: Video Object Segmentation qualitative examples. Given the ground-truth object masks for the initial frame, we propagate the instance masks to subsequent frames based on patch similarity within the V-JEPA 2.1 embedding space. All videos are processed at a resolution where the shorter side is set to 480 pixels.
Method
Param.
Motion Understanding
Appearance Understanding
SSv2
Diving-48
K400
IN1K
Results Reported in the Literature
VideoMAEv2 ( Wang et al., 2023 )
1B
56.1
–
82.8
71.4
InternVideo2-1B ( Wang et al., 2024b )
1B
67.3
–
87.9
–
VideoPrism ( Zhao et al., 2024 )
1B
68.5
71.3
87.6
–
DINOv3 ViT-H+ ( Siméoni et al., 2025 )
0.8B
69.8
–
86.7
87.9
Table 9: Performance on global tasks. We report Top-1 classification accuracy on action recognition benchmarks (SSv2, Diving-48, and K400) and image classification (IN1K). Our protocol uses a resolution of 256×256 by default, and 384×384 for V-JEPA 2 ViT-g and V-JEPA 2.1; and uses 16×2×3 inputs for SSv2 (16-frame clips, 2 temporal crops, 3 spatial crops), 16×8×3 for K400, and 32×4×3 for Diving-48. V-JEPA 2.1 VIT-G achieves state-of-art performance on the SSv2 action recognition asks while being competitive on appearance understanding tasks.
Method
Params Enc / LLM
Avg.
PerceptionTest Test Acc
MVP Paired-Acc
TempCompass multi-choice
TemporalBench (MBA-short QA)
TOMATO Acc
TVBench Acc
MVBench Acc
≤8B Video Language Models Results Reported in the Literature
InternVL-2.5 ( Chen et al., 2024 )
300M/7B
52.1
68.9
39.9
68.3
24.3
29.4
61.6
72.6
Qwen2VL ( Wang et al., 2024a )
675M/7B
47.0
66.9
29.2
67.9
20.4
31.5
46.0
67.0
Qwen2.5VL ( Qwen Team et al., 2025 )
1B/7B
49.7
70.5
36.7
71.7
24.5
24.6
50.5
69.6
PLM 8B ( Cho et al., 2025 )
1B/8B
56.7
82.7*
39.7
72.7
28.3
33.2
63.5
77.1
VJEPA 2 ViT-g 384 ( Assran et al., 2025 )
1B/8B
59.5
84.0*
44.5
76.9
36.7
40.3
60.6
73.5
Table 10: VideoQA results. We report the performance of VJEPA 2.1 trained with Llama 3.1 8B backbone on several popular temporal video understanding datasets. We train on a modified subset of the PerceptionLM ( Cho et al., 2025 ) data, and compare to VJEPA 2 ( Assran et al., 2025 ) by training it on the same data regime. VJEPA 2.1 achieves comparatively better performance than VJEPA 2 on several video understanding tasks which require both rich visual information and temporal understanding. For PerceptionTest ( Pătrăucean et al., 2023 ) , we report the validation accuracy, as the test server to compute the test accuracy (indicated by *) is no longer active.
Figure 14: Comparison of dense features produced by different SSL methods on a single image. We compare DINOv2 ViT-g ( Oquab et al., 2023 ) , DINOv3 ViT-H+ ( Siméoni et al., 2025 ) and V-JEPA 2 ViT-g ( Assran et al., 2025 ) against V-JEPA 2.1 ViT-G. Images are resized so that their shorter side is set to 1024 pixels, after which they are processed using the 2D convolution image tokenizer.
Figure 15: V-JEPA 2.1 unlocks dense features from video. Qualitative results on Ego4D (1080px), Cityscapes (1080px), Diving48 (768px), and MOCA (768px) illustrate the strong temporal consistency of the dense features produced by our ViT-G model, particularly on dynamic objects. Since we process entire video sequences, we employ the 3D convolutional video tokenizer.
Model
# Params
Distilled
SSv2
K400
IN1K
ADE20K
Citys.
VOC12
NYU ( ↓ )
KITTI ( ↓ )
DAVIS
ViT-G
2B
77.7
87.7
85.5
47.9
73.5
85.0
0.307
2.461
69.0
ViT-L
300M
74.2
84.3
81.8
42.0
66.7
79.6
0.381
2.914
64.5
ViT-L
300M
✓
76.5
86.9
84.9
46.7
71.0
83.9
0.336
2.490
68.7
ViT-B
80M
✓
74.5
85.1
84.0
42.0
64.5
78.0
0.408
2.850
67.0
Table 11: Benefits of Distillation. Distilling the V-JEPA 2.1 ViT-G into smaller models (ViT-L, 300M; ViT-B, 80M) yields consistent improvements over training from scratch and approaches the performance of the ViT-G teacher.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Primary Phase
Cooldown Phase
Number of frames
16
64
Frames per Second
4.0
4.0
Video Crop Size
256
384
Image Crop Size
256
512
Random Resize Aspect Ratio
[0.75, 1.35]
[0.75, 1.35]
Random Resize Scale
[0.3, 1.0]
[0.3, 1.0]
Appendix
Table 12: Pretraining hyperparameters for the Primary and Cooldown phases. Hyperparameters are identical across all our ViT architectures (ViT-L, -g and -G).
Deep Self-Supervision
ADE20k
NYU
Diving-48
Last-Layer
4-Layers
Last-Layer
4-Layers
Last-Layer
4-Layers
✗
34.9
39.1
0.513
0.463
85.8
86.9
✔
42
43.9
0.381
0.370
87.2
88.1
Appendix
Table 13: Impact of deep self-supervision on the need for multi-scale evaluation. We conduct experiments with our V-JEPA 2.1 ViT-L trained from scratch. Last-Layer indicates that evaluation is conducted with only the last layer of the encoder as input to the probe. 4-Layers indicates that evaluation is conducted by concatenating 4 intermediate layers of the encoder and feeding it to the probe.
Model
Res
SSv2
D-48
K400
IN1K
EK100
ADE20k
Citys.
VOC
NYU
KITTI
DAVIS
YT-VOS
ViT-g
256
76.7
88.6
86.2
84.1
35.7
47.8
71.7
84.6
0.359
2.611
68.0
72.0
ViT-g
384
76.9
89
87
84.8
38.4
47.8
71.8
84.7
0.350
2.601
67.4
71.3
ViT-G
256
77.1
89.5
87.3
84.8
38.8
47.8
73.2
84.7
0.313
2.472
68.6
72.7
ViT-G
384
77.7
89.2
87.7
85.5
40.8
47.9
73.5
85.0
0.307
2.461
69.0
72.6
Appendix
Table 14: Impact of pretraining spatial resolution. We compare models trained during the cooldown phase at 256×256 spatial resolution against 384×384 .
Progress in AI has largely been driven by methods that assume less. As compute and data increase, approaches with weaker inductive biases generally outperform those with stronger assumptions. This is particularly characteristic of the field of Visual Representation Learning, where approaches have gone from being dominated by Supervised Learning, to Weakly Supervised Learning, to the now widespread success of Self-Supervised Learning without human labels. Yet, even modern Self-Supervised Learning approaches still depend on strong inductive biases such as augmentations, masking, or cropping. If this trend holds, even these remaining biases should become bottlenecks at scale -- and our experiments confirm this: the optimal strength of inductive biases decreases as data grows. This motivates the search for approaches that rely on fewer assumptions. To this end, we introduce Temporal Difference in Vision (TDV), a new paradigm for self-supervised learning from video that avoids existing inductive biases, relying instead on a causal assumption that the past causes the future. TDV functions by jointly training an image encoder and a motion encoder so that the current frame's representation plus the encoded motion equals the next frame's representation. Despite not leveraging any strong inductive biases, TDV matches state-of-the-art recipes on dense spatial tasks, laying the foundation for representation learning without strong assumptions.
Self-supervised video foundation models learn rich spatiotemporal representations, yet it remains unclear what visual concepts these representations encode, where they emerge across transformer layers, and how they are geometrically organized. In this work, we tackle these three questions through a systematic layer-wise analysis of V-JEPA 2 and VideoMAE-v2. We leverage lightweight probes trained to discover three temporally grounded properties: (i) camera motion understanding, (ii) intuitive physics, and (iii) anomaly detection. Both models encode camera motion, with best results (>90 ROC AUC) emerging at 60-70% of network depth, and achieve moderate anomaly detection performance (>60 ROC AUC), but remain near chance on intuitive-physics tasks, suggesting a limited encoding of deeper physical reasoning. Beyond classification, we find that temporal features from individual videos form smooth low-dimensional trajectories in representation space, suggesting that camera motion is not only linearly decodable but also geometrically organized. Based on these results, we apply geometry-aware spline-based steering in the model's latent representations to interpolate camera motion, yielding steered videos with smoother trajectories and more coherent temporal progression than linear interpolation.
Sharon S. Musa, Fereshteh Forghani, Harrish Thasarathan +3
Lassonde School of Engineering, York University, Toronto, Canada · Vector Institute, Toronto, Canada · FAIR, Meta Superintelligence Labs +2
Machines that understand humans should perceive the present and anticipate the future. Existing human-centric vision model are pretrained on human images, set the state of the art in static dense perception, so motion and anticipation are out of reach. Here we present Human-JEPA, a human-centric vision model trained on video by anchored forecasting: dense targets are pinned to a frozen copy of the initialization, preventing a silent collapse of dense perception, and block masks are replaced by a pure past-to-future split, avoiding a five-point action tax and a seventeen-point re-identification collapse. Under frozen probes, Human-JEPA leads the pixel-anchored specialists on pose and person re-identification at 2.7 times fewer parameters, conceding high-resolution dense parsing, and its released predictor head is the first that does not degrade anticipation. A single safely adapted model thus serves both halves of understanding humans.
Hui Wei, Licai Sun, Guoying Zhao
ELLIS Institute Finland, Finland · Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland