cs.CVSep 30, 2026
SaveVR-JEPA: Learning Contrastive-State Latent Guidance for Generation-based Video Reasoning
Organizations: Shenzhen Campus of Sun Yat-sen University · Tsinghua Shenzhen International Graduate School · Shenzhen Loop Area Institute · Tencent · Mohamed bin Zayed University of Artificial Intelligence · UiT The Arctic University of Norway
Abstract
Reasoning through video generation offers a promising path toward visual intelligence by modeling latent visual states and their dynamics. However, current video generation models often lack explicit guidance on how these states should evolve, leaving generated trajectories prone to physical and structural inconsistencies that undermine reasoning reliability. While the Video Joint-Embedding Predictive Architecture (V-JEPA) provides rich spatiotemporal priors learned through latent prediction, these general priors do not naturally adapt to the logical reasoning capabilities required for complex visual tasks. To bridge this gap, we propose VR-JEPA, a framework that aligns the V-JEPA predictor with task-specific reasoning logic through localized contrastive-state learning and uses its predicted latent trajectories to guide video generation for visual reasoning. Specifically, (i) we pair successful trajectories with generated alternatives under the same input conditions and use discrepancies in their V-JEPA representations to identify informative states and tokens for localized contrastive supervision. (ii) We further equip the V-JEPA predictor with skill-specific experts trained on anchor-task data, allowing the model to adaptively specialize its shared spatiotemporal priors across diverse cognitive domains. Together with skill-specific experts, this contrastive supervision enables VR-JEPA to predict latent trajectories that provide task-specific logical guidance for video generation. Comprehensive experiments on the large-scale VBVR-Pro-Bench dataset demonstrate that VR-JEPA achieves an relative improvement over the cutting-edge generation-based reasoning baseline, significantly mitigating physical artifacts and enhancing logical consistency.
Figures & tables
Figure 1: Generation-based visual reasoning with VR-JEPA. Top: Comparison of different video reasoning paradigms: (1) direct video generation; (2) video generation guided by static DINOv3 features of the initial state; and (3) VR-JEPA, which adapts V-JEPA’s spatiotemporal priors to diverse reasoning skills and guides video generation through predicted latent trajectories. Bottom: Video reasoning results of VR-JEPA and VBVR-Wan2.2, showing that VR-JEPA generates trajectories that satisfy task requirements. Red dashed boxes highlight reasoning errors.
Figure 2: Overview of VR-JEPA. VR-JEPA comprises a Latent Reasoner and a Video Generator. The Latent Reasoner predicts a latent rollout from the initial scene and task instruction, with SR-MoE adaptively selecting expert combinations for different reasoning tasks. Its seven experts cover VIS (visual discrimination), NUM (numerical reasoning), IDN (object identity and persistence), SPA (spatial reasoning and planning), DYN (physical dynamics), CMP (compositional reasoning), and ABS (abstract rule inference). During predictor training, localized contrastive-state learning uses dense V-JEPA discrepancies between successful and generated videos to focus contrastive supervision on tokens corresponding to informative states and regions. The Video Generator conditions Wan2.2 DiT on the rollout to synthesize a reasoning video.
Figure 3: State selection for localized contrastive-state learning. Successful reference trajectories and task-matched generated candidates are compared in dense V-JEPA feature space. After state-specific quantile subtraction, large residual discrepancies select states for contrastive supervision (green checks), while small discrepancies exclude them (red crosses).
| In-Domain (ID) (%) | Out-of-Domain (OOD) (%) | ||||||||||||
| Models | Overall (%) | Avg. | Abst. | Know. | Perc. | Spat. | Trans. | Avg. | Abst. | Know. | Perc. | Spat. | Trans. |
| Open-source Video Models | |||||||||||||
| Wan2.2-I2V-A14B | 18.2 | 15.7 | 12.3 | 15.2 | 15.0 | 17.0 | 23.3 | 20.7 | 29.2 | 17.9 | 13.8 | 19.5 | 31.8 |
| LTX-2.3-I2AV | 11.2 | 10.6 | 9.3 | 12.3 | 9.7 | 13.6 | 5.1 | 11.9 | 21.7 | 19.8 | 7.0 | 9.1 | 5.9 |
| Wan2.1-I2V-14B-720P | 10.0 | 10.5 | 8.7 | 14.6 | 12.4 | 9.9 | 4.0 | 9.5 | 13.9 | 8.7 | 7.6 | 12.3 | 5.2 |
| Wan2.2-TI2V-5B | 9.4 | 6.6 | 5.0 | 7.4 | 7.7 | 9.2 | 2.5 | 12.2 | 22.2 | 9.2 | 9.1 | 6.3 | 11.6 |
Table 1: Main results on VBVR-Pro-Bench ( Xu et al., 2026 ) . We report overall, in-domain (ID), and out-of-domain (OOD) scores, followed by results for abstraction (Abst.), knowledge (Know.), perception (Perc.), spatial reasoning (Spat.), and transformation (Trans.). Avg. denotes the mean of task scores weighted by the number of samples per task. All scores are reported as percentages. Higher is better. Bold and underlined values indicate the best and second-best results.
Figure 4: Qualitative comparison on VBVR-Pro-Bench. Given the initial state and instruction, we compare the reasoning results of VR-JEPA with those of a video reasoning model (VBVR-Wan2.2), an open-source video model (LTX-2.3-I2AV), and two proprietary video models (Kling VIDEO 3.0 and Seedance 2.0). Green and red dashed boxes highlight correct results and reasoning errors, respectively.
Figure 5: Qualitative ablation of localized contrastive-state learning (LCL) and skill-routed mixture of experts (SR-MoE). Top and middle: adding LCL and SR-MoE to the dense baseline, respectively. Bottom: adding SR-MoE to the LCL-equipped predictor to obtain the full model. Green and red dashed boxes highlight correct results and reasoning errors, respectively.
| Variant | Components | Overall (%) | In-Domain (ID) (%) | Out-of-Domain (OOD) (%) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LCL | SR-MoE | Avg. | Abst. | Know. | Perc. | Spat. | Trans. | Avg. | Abst. | Know. | Perc. | Spat. | Trans. | ||
| Dense baseline | ✗ | ✗ | 53.9 | 63.4 | 53.8 | 58.9 | 58.9 | 68.8 | 91.4 | 44.4 | 43.3 | 44.5 | 31.9 | 59.3 | 75.3 |
| + LCL | ✓ | ✗ | 55.1 | 64.9 | 52.6 | 61.2 | 61.3 | 71.6 | 93.0 | 45.3 | 44.1 | 38.5 | 33.1 | 65.4 | 75.8 |
| + SR-MoE | ✗ | ✓ | 55.5 | 65.2 | 54.0 | 61.4 | 60.7 | 69.8 | 95.6 | 45.9 | 42.8 | 42.3 | 34.3 | 64.2 | 77.3 |
| Full model | ✓ | ✓ | 56.0 | 65.9 | 56.3 | 63.8 | 61.8 | 69.1 | 91.4 | 46.1 | 41.5 | 42.1 | 35.2 | 65.2 | 77.5 |
Table 2: Ablation of localized contrastive-state learning (LCL) and skill-routed mixture of experts (SR-MoE). Scores are reported as percentages; higher is better.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Training strategy | Trainable modules | Task success (%) | ||
|---|---|---|---|---|
| TCI | LoRA | ID | OOD | |
| VBVR-Wan2.2 | ✗ | ✗ | 55.4 | 45.1 |
| LoRA-only tuning | ✗ | ✓ | 57.4 | 45.5 |
| TCI-only tuning | ✓ | ✗ | 63.1 | 43.7 |
| Joint tuning | ✓ | ✓ | 65.9 | 46.1 |
Table 4: Ablation of the training strategy. We compare tuning the trajectory-conditioning interface (TCI), LoRA parameters, or both, with VBVR-Wan2.2 as the baseline. ID and OOD task-success scores are reported on VBVR-Pro-Bench as percentages; higher is better.
| Routing strategy | Task success(%) | |
|---|---|---|
| ID | OOD | |
| Shared FFN only | 64.2 | 44.7 |
| Uniform routing | 63.9 | 44.7 |
| Random routing | 64.8 | 44.8 |
| Learned routing | 65.9 | 46.1 |
Table 5: Ablation of the SR-MoE routing strategy. We compare learned routing with uniform routing, random routing, and a shared-FFN-only variant to assess the contribution of adaptive expert composition. Shared FFN only disables the expert branches of the trained full model at inference, retaining the shared FFNs. ID and OOD task-success scores are reported on VBVR-Pro-Bench as percentages; higher is better.
Figure 6: Qualitative comparison of SR-MoE routing strategies. We compare video reasoning results with learned routing, shared FFN only, uniform routing, and random routing. The shared-FFN-only variant disables expert branches at inference. Disrupting learned expert coordination degrades reasoning performance in the illustrated examples. Green and red dashed boxes highlight correct and incorrect results, respectively.
| Rollout condition | Intervention | Task success (%) | |
| ID | OOD | ||
| Predicted rollout | Use the model-predicted trajectory | 65.9 | 46.1 |
| No conditioning | Remove all trajectory conditioning | 51.5 | 44.9 |
| Repeated initial state | Repeat the initial state across all rollout steps | 62.5 | 43.3 |
| Cross-sample rollout | Use the predicted trajectory from another sample | 58.5 | 45.9 |
Table 6: Ablation of latent trajectory guidance. We test whether task success depends on the sample-specific temporal content of the rollout, rather than merely the presence of an additional conditioning signal.
Figure 7: Additional qualitative comparisons with open-source models on VBVR-Pro-Bench. Given the initial state and instruction, we compare the video reasoning results of VR-JEPA with VBVR-Wan2.2 and LTX-2.3-I2AV. Green and red dashed boxes highlight correct results and reasoning errors, respectively.
Figure 8: Additional qualitative comparisons with proprietary video models on VBVR-Pro-Bench. Given the initial state and instruction, we compare the video reasoning results of VR-JEPA with Kling VIDEO 3.0 and Seedance 2.0. Green and red dashed boxes highlight correct results and reasoning errors, respectively.
Figure 9: Qualitative comparison of visual representations on VBVR-Pro-Bench. Given the initial state and instruction, we compare video reasoning results guided by, from left to right: (1) predicted latent rollouts in V-JEPA 2.1 space (VR-JEPA), (2) initial-state DINOv3 features, and (3) predicted latent rollouts in DINOv3 space. Green and red dashed boxes highlight correct results and reasoning errors, respectively.
Figure 10: Qualitative ablation of latent trajectory guidance. We compare video reasoning results using the predicted rollout with three interventions: removing trajectory conditioning, repeating the initial state across rollout steps, and substituting a rollout from another sample. Red dashed boxes highlight reasoning errors.
Figure 11: Task-dependent expert routing. Mean routing weights for ten tasks, averaged across the two SR-MoE predictor layers (layer4 and layer5). Rows denote tasks and columns denote the seven skill-specific experts; darker cells indicate larger weights. The distributions show both concentrated expert selection and allocations spread across multiple experts.
Figure 12: Probing expert specialization through skill removal. We remove the highest-weight expert for each of seven examples and examine whether the resulting errors reflect the loss of its associated reasoning skill. From top to bottom, the removed experts correspond to visual discrimination, numerical reasoning, object identity and persistence, spatial reasoning and planning, physical dynamics, compositional reasoning, and abstract rule inference. Green and red dashed boxes highlight correct results and reasoning errors, respectively.