DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation

Abolfazl Meyarian; Amin Karimi Monsefi; Rajiv Ramnath; Ser-Nam Lim

cs.CVMar 26, 2026

DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation

Authors: Abolfazl Meyarian, Amin Karimi Monsefi, Rajiv Ramnath, Ser-Nam Lim

Organizations: Path Robotics, Columbus, OH, USA · The Ohio State University, Columbus, OH, USA · University of Central Florida, Orlando, FL, USA

Abstract

Flow-matching video generators produce temporally coherent, high-fidelity outputs yet routinely violate elementary physics because their reconstruction objectives penalize per-frame deviations without distinguishing physically consistent dynamics from impossible ones. Contrastive flow matching offers a principled remedy by pushing apart velocity-field trajectories of differing conditions, but we identify a fundamental obstacle in the text-conditioned video setting: semantic-physics entanglement. Because natural-language prompts couple scene content with physical behavior, naive negative sampling draws conditions whose velocity fields largely overlap with the positive sample's, causing the contrastive gradient to directly oppose the flow-matching objective. We formalize this gradient conflict, deriving a precise alignment condition that reveals when contrastive learning helps versus harms training. Guided by this analysis, we introduce DiReCT (Disentangled Regularization of Contrastive Trajectories), a lightweight post-training framework that decomposes the contrastive signal into two complementary scales: a macro-contrastive term that draws partition-exclusive negatives from semantically distant regions for interference-free global trajectory separation, and a micro-contrastive term that constructs hard negatives sharing full scene semantics with the positive sample but differing along a single, LLM-perturbed axis of physical behavior; spanning kinematics, forces, materials, interactions, and magnitudes. A velocity-space distributional regularizer helps to prevent catastrophic forgetting of pretrained visual quality. When applied to Wan 2.1-1.3B, our method improves the physical commonsense score on VideoPhy by 16.7% and 11.3% compared to the baseline and SFT, respectively, without increasing training time.

Figures & tables

Figure 1: Comparison of zeroshot Wan-2.1-1.3B against the same model trained using DiReCT on a few prompts from VideoPhy. In the top example, the baseline drives the car backward, violating forward kinematics, while DiReCT produces a consistent forward trajectory. In the middle example, the baseline’s surfer progressively merges with the sail rig, losing bodily structure, whereas DiReCT maintains the surfer’s integrity and mass throughout. In the bottom example, the baseline keeps the wood stationary and upright despite the flowing current, while DiReCT generates realistic downstream motion consistent with buoyancy and flow dynamics.

Figure 2: Overview of the DiReCT framework. The contrastive objective is decomposed into two complementary scales: a macro-contrastive term draws random negatives from semantically distant clusters (MaNS), providing clean global trajectory separation, while a micro-contrastive term uses physics-perturbed hard negatives (MiNS) that share scene semantics but violate a targeted physical dimension, enabling fine-grained physics discrimination.

Figure 3: Macro-contrastive negative sampling (MaNS). Prompts are encoded via the text encoder of the video generative model, globally pooled, and partitioned into $K$ semantic regions. For each positive, negatives are drawn exclusively from different partitions, ensuring minimal velocity-field overlap and preventing the gradient conflict identified in Proposition 1 from Appendix 0.A .

Figure 4: Micro-contrastive hard negative generation (MiNS). For each anchor prompt, an LLM (Qwen2.5-7B-Instruct) perturbs a single physics dimension while preserving scene semantics. The perturbed prompt is rendered by the base model to produce a hard negative video whose velocity trajectory is physically inconsistent with the anchor.

Baselines
Method	VideoPhy(%) $\uparrow$			WorldModelBench $\uparrow$
	SA	PC	AVG	Instr.	Common Sense		Physics Adherence		Total
	SA	PC	AVG	Instr.	Frame	Temp.	Mass	Penetr.	Total
Zero-shot	50.45	32.69	41.57	2.18	0.93	0.80	0.72	0.83	5.46
+ SFT	50.38	34.28	42.33	2.14	0.92	0.81	0.75	0.85	5.47
+ Random negatives ( $\Delta$ FM)	47.71	35.89	41.80	2.12	0.91	0.80	0.73	0.84	5.40

Table 1: Ablation on negative sampling strategy. All fine-tuned variants use the same base model and training data. VideoPhy measures the physical plausibility of generated videos. WorldModelBench evaluates physical commonsense and physics adherence across diverse scenarios. Best results are shown in bold , second best underlined .

Model	Params	VideoPhy (%) $\uparrow$			WorldModelBench $\uparrow$
		SA	PC	AVG	Instr.	Common Sense		Physics Adherence		Total
		SA	PC	AVG	Instr.	Frame	Temporal	Mass	Penetr.	Total
CogVideoX-2B [ 45 ]	2B	48.73	34.75	41.74	1.97	0.88	0.75	0.67	0.82	5.09
LTX [ 7 ]	2B	21.10	25.09	23.10	1.57	0.74	0.81	0.79	0.91	4.82
Allegro [ 52 ]	2.8B	51.27	27.24	39.25	1.91	0.87	0.79	0.76	0.89	5.22
CogVideoX-5B [ 45 ]	5B	47.30	42.79	45.05	2.01	0.91	0.80	0.75	0.86	5.33

Table 2: Comparison with state-of-the-art open-source video generation models of comparable scale. We report VideoPhy (SA, PC, AVG), WorldModelBench scores across instruction following, common sense, and physics adherence. Best results in bold , second best underlined .

Figure 5: Comparison of DiReCT with CogVideoX-2B and Allegro on a prompt from VideoPhy (top) and one from WorldModelBench (bottom).

Figure 6: Training dynamics in terms of VideoPhy scores. SFT starts to saturate beyond 5k steps, while DiReCT continues to improve and stabilizes around 15k steps, with the gap widening particularly on Physical Commonsense (PC).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: The prompt template used to generate the hard negatives.

$\lambda_{\mathrm{anc}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{anc}}$	SA	PC	AVG
0.1	48.57	33.50	41.04
0.2	50.38	34.28	42.33
0.3	50.40	31.69	41.05

Appendix

Table S1: Sensitivity to distributional anchoring strength

\lambda_{\mathrm{anc}}

. SFT only, no contrastive terms.

$\lambda_{\mathrm{rand}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{rand}}$	SA	PC	AVG
0.001	50.52	35.48	43.00
0.005	50.65	36.47	43.56
0.02	49.83	36.82	43.33

Appendix

Table S2: Sensitivity to macro-contrastive strength

\lambda_{\mathrm{rand}}

\lambda_{\mathrm{anc}}{=}0.2

\lambda_{\mathrm{hard}}{=}0.01

$\lambda_{\mathrm{hard}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{hard}}$	SA	PC	AVG
0.005	50.61	35.97	43.29
0.02	50.78	37.46	44.12
0.1	49.18	37.03	43.11

Appendix

Table S3: Sensitivity to micro-contrastive strength

\lambda_{\mathrm{hard}}

\lambda_{\mathrm{anc}}{=}0.2

\lambda_{\mathrm{rand}}{=}0.005

Method	Training Cost $\downarrow$
Method	Mem. (GB)	Time (s/iter)
SFT	23.29	4.94
DiReCT	30.45	4.95

Appendix

Table S4: Per-iteration training cost. Both methods use Wan 2.1-1.3B at

480\times 832

resolution (81 frames), global batch size 8, DeepSpeed ZeRO-2 on H200 GPUs.

$K$	VideoPhy (%) $\uparrow$
$K$	SA	PC	AVG
8	49.87	34.52	42.20
16	50.61	36.38	43.50
32	51.26	38.16	44.71
64	50.94	36.89	43.92
128	50.18	35.71	42.95

Appendix

Table S5: Effect of cluster count

K

in macro-contrastive sampling (MaNS). We vary the number of semantic clusters used for group-exclusive negative sampling. Too few clusters risk intra-cluster semantic diversity (gradient conflict); too many produce overly fragmented groups with noisy boundaries.

Method	Cosine $\uparrow$
$\Delta$ FM (random neg.)	$0.12\pm 0.21$
DiReCT -MaNS (ours)	$-0.05\pm 0.12$

Appendix

Table S6: Report of gradient alignment between flow-matching and contrastive loss terms for

\Delta

FM and DiReCT according to the cosine similarity metric.

Figure S2: Additional qualitative comparisons with CogVideoX-2B and LTX on VideoPhy prompts. Red boxes highlight physical violations in baseline outputs; green boxes indicate physically consistent behavior produced by DiReCT . In the top example, LTX causes the cookie to dissolve and lose structural integrity upon contact with the coffee, while CogVideoX-2B drops the cookie with no hand visible in frame; DiReCT preserves the cookie’s shape throughout the dipping motion with a plausible hand–cookie interaction. In the middle example, CogVideoX-2B fails to render the shoe entirely, showing only a brush, and LTX exhibits severe visual degradation with color bleeding and loss of object boundaries; DiReCT maintains coherent brush–shoe contact with a realistic polishing motion. In the bottom example, LTX produces unnatural spoon–yogurt interactions with the spoon merging into the mixture, while CogVideoX-2B generates implausible yogurt rigidity; DiReCT produces a physically plausible stirring motion in which the spoon and yogurt maintain distinct material properties throughout.

Contrastive Flow Matching

See all 9 papers

Mar 27, 2026

PhysVid: Physics Aware Local Conditioning for Generative Video Models

Generative video models achieve high visual fidelity but often violate basic physical principles, limiting reliability in real-world settings. Prior attempts to inject physics rely on conditioning: frame-level signals are domain-specific and short-horizon, while global text prompts are coarse and noisy, missing fine-grained dynamics. We present PhysVid, a physics-aware local conditioning scheme that operates over temporally contiguous chunks of frames. Each chunk is annotated with physics-grounded descriptions of states, interactions, and constraints, which are fused with the global prompt via chunk-aware cross-attention during training. At inference, we introduce negative physics prompts (descriptions of locally relevant law violations) to steer generation away from implausible trajectories. On VideoPhy, PhysVid improves physical commonsense scores by

\approx 33\%

over baseline video generators, and by up to

\approx 8\%

on VideoPhy2. These results show that local, physics-aware guidance substantially increases physical plausibility in generative video and marks a step toward physics-grounded video models.

Saurabh Pathak, Elahe Arani, Mykola Pechenizkiy +1

Eindhoven University of Technology

Physics-Guided Video Generation Physical Consistency in Video Generation Video Generation Physics-Informed Generative Modeling Controllable Video Generation

Ask PDF

Sep 8, 2026

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation

Video generation models have recently attracted substantial attention for their ability to generate visually compelling videos, yet ensuring physically consistent and plausible dynamics still remains a fundamental challenge, driving a growing line of research on physical realism in video generation. To address this challenge, motivated by the fact that physical regularities are primarily encoded in motion patterns, we propose PhysFlow, a novel two-stage framework for improving the physical plausibility of generated videos by decomposing video generation into motion-aware optical flow generation followed by motion-conditioned appearance synthesis. Specifically, PhysFlow consists of a physics-aware optical-flow video generator called PA-Flow and a flow-guided video generator called FlowRender. During the first stage, PA-Flow employs a physics-aware attention module to model how motion attributes and material properties influence global motion and local deformation, respectively, and generates an optical flow video as an explicit representation of motion. In the second stage, FlowRender leverages the decoupled motion representation as guidance to synthesize realistic textures and appearances, ultimately producing the final physically plausible video. To further support model training with explicit physical supervision, we construct PhysVideo, a physics-based video dataset generated with a physics engine and 3D-GS rendering, containing 10K foreground objects and 50K realistic video sequences with annotations of motion and material properties. Extensive experiments demonstrate that our proposed PhysFlow generates videos with superior physical plausibility while maintaining high visual fidelity compared with existing methods.

Cong Wang, Hanxin Zhu, Yonglin Tian +5

State Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Zhongguancun Academy, Beijing 100094, China +2

Optical Flow Physical Consistency in Video Generation Controllable Video Generation

Ask PDF

Jul 26, 2026

VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation

Modern video generation models can synthesize visually compelling and temporally coherent clips, yet controlling their physical behavior remains difficult with standard text and image conditions. The core challenge is a conditioning bottleneck: material response, contact interaction, deformation, and motion trajectory are continuous and relational physical cues that are hard to specify exhaustively in language but can be demonstrated naturally by video. We propose VIPER, a Visual In-Context Physics Reasoning framework for reference-guided image-to-video generation. Given a target image, a brief target prompt, and a reference video, VIPER treats the reference as a dense visual demonstration of the desired physical process rather than an appearance template. It uses a Multimodal Large Language Model (MLLM) to extract reference-derived physical cues and guide a pretrained image-to-video generator through a hierarchical training strategy, enabling physical behavior transfer while preserving the visual prior of the base generator. To support this setting, we construct VIPER-19K, a curated dataset with material, trajectory, and physical-impact annotations, together with filtered reference-target pairs. Experiments on an unseen validation set show that VIPER achieves stronger reference-video physical similarity and higher human preference than representative video generation and video-as-prompt baselines, while maintaining competitive general video quality. Qualitative results further demonstrate that VIPER can transfer reference-derived physical behavior to new target scenes without requiring carefully engineered prompts.

Tianxiao Chen, Hanmo Chen, Huajin Chen +3

Zhejiang University · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.

Video-Reference Video Generation Physical Consistency in Video Generation Image-to-Video Generation Physics-Guided Video Generation

Ask PDF

Abstract

Contrastive Flow Matching

Explore similar work

PhysVid: Physics Aware Local Conditioning for Generative Video Models

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation

VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation