DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation

Abolfazl Meyarian; Amin Karimi Monsefi; Rajiv Ramnath; Ser-Nam Lim

cs.CVMar 26, 2026

DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation

Authors: Abolfazl Meyarian, Amin Karimi Monsefi, Rajiv Ramnath, Ser-Nam Lim

Organizations: Path Robotics, Columbus, OH, USA · The Ohio State University, Columbus, OH, USA · University of Central Florida, Orlando, FL, USA

Abstract

Flow-matching video generators produce temporally coherent, high-fidelity outputs yet routinely violate elementary physics because their reconstruction objectives penalize per-frame deviations without distinguishing physically consistent dynamics from impossible ones. Contrastive flow matching offers a principled remedy by pushing apart velocity-field trajectories of differing conditions, but we identify a fundamental obstacle in the text-conditioned video setting: semantic-physics entanglement. Because natural-language prompts couple scene content with physical behavior, naive negative sampling draws conditions whose velocity fields largely overlap with the positive sample's, causing the contrastive gradient to directly oppose the flow-matching objective. We formalize this gradient conflict, deriving a precise alignment condition that reveals when contrastive learning helps versus harms training. Guided by this analysis, we introduce DiReCT (Disentangled Regularization of Contrastive Trajectories), a lightweight post-training framework that decomposes the contrastive signal into two complementary scales: a macro-contrastive term that draws partition-exclusive negatives from semantically distant regions for interference-free global trajectory separation, and a micro-contrastive term that constructs hard negatives sharing full scene semantics with the positive sample but differing along a single, LLM-perturbed axis of physical behavior; spanning kinematics, forces, materials, interactions, and magnitudes. A velocity-space distributional regularizer helps to prevent catastrophic forgetting of pretrained visual quality. When applied to Wan 2.1-1.3B, our method improves the physical commonsense score on VideoPhy by 16.7% and 11.3% compared to the baseline and SFT, respectively, without increasing training time.

Figures & tables

Figure 1: Comparison of zeroshot Wan-2.1-1.3B against the same model trained using DiReCT on a few prompts from VideoPhy. In the top example, the baseline drives the car backward, violating forward kinematics, while DiReCT produces a consistent forward trajectory. In the middle example, the baseline’s surfer progressively merges with the sail rig, losing bodily structure, whereas DiReCT maintains the surfer’s integrity and mass throughout. In the bottom example, the baseline keeps the wood stationary and upright despite the flowing current, while DiReCT generates realistic downstream motion consistent with buoyancy and flow dynamics.

Figure 2: Overview of the DiReCT framework. The contrastive objective is decomposed into two complementary scales: a macro-contrastive term draws random negatives from semantically distant clusters (MaNS), providing clean global trajectory separation, while a micro-contrastive term uses physics-perturbed hard negatives (MiNS) that share scene semantics but violate a targeted physical dimension, enabling fine-grained physics discrimination.

Figure 3: Macro-contrastive negative sampling (MaNS). Prompts are encoded via the text encoder of the video generative model, globally pooled, and partitioned into $K$ semantic regions. For each positive, negatives are drawn exclusively from different partitions, ensuring minimal velocity-field overlap and preventing the gradient conflict identified in Proposition 1 from Appendix 0.A .

Figure 4: Micro-contrastive hard negative generation (MiNS). For each anchor prompt, an LLM (Qwen2.5-7B-Instruct) perturbs a single physics dimension while preserving scene semantics. The perturbed prompt is rendered by the base model to produce a hard negative video whose velocity trajectory is physically inconsistent with the anchor.

Baselines
Method	VideoPhy(%) $\uparrow$			WorldModelBench $\uparrow$
	SA	PC	AVG	Instr.	Common Sense		Physics Adherence		Total
	SA	PC	AVG	Instr.	Frame	Temp.	Mass	Penetr.	Total
Zero-shot	50.45	32.69	41.57	2.18	0.93	0.80	0.72	0.83	5.46
+ SFT	50.38	34.28	42.33	2.14	0.92	0.81	0.75	0.85	5.47
+ Random negatives ( $\Delta$ FM)	47.71	35.89	41.80	2.12	0.91	0.80	0.73	0.84	5.40

Table 1: Ablation on negative sampling strategy. All fine-tuned variants use the same base model and training data. VideoPhy measures the physical plausibility of generated videos. WorldModelBench evaluates physical commonsense and physics adherence across diverse scenarios. Best results are shown in bold , second best underlined .

Model	Params	VideoPhy (%) $\uparrow$			WorldModelBench $\uparrow$
		SA	PC	AVG	Instr.	Common Sense		Physics Adherence		Total
		SA	PC	AVG	Instr.	Frame	Temporal	Mass	Penetr.	Total
CogVideoX-2B [ 45 ]	2B	48.73	34.75	41.74	1.97	0.88	0.75	0.67	0.82	5.09
LTX [ 7 ]	2B	21.10	25.09	23.10	1.57	0.74	0.81	0.79	0.91	4.82
Allegro [ 52 ]	2.8B	51.27	27.24	39.25	1.91	0.87	0.79	0.76	0.89	5.22
CogVideoX-5B [ 45 ]	5B	47.30	42.79	45.05	2.01	0.91	0.80	0.75	0.86	5.33

Table 2: Comparison with state-of-the-art open-source video generation models of comparable scale. We report VideoPhy (SA, PC, AVG), WorldModelBench scores across instruction following, common sense, and physics adherence. Best results in bold , second best underlined .

Figure 5: Comparison of DiReCT with CogVideoX-2B and Allegro on a prompt from VideoPhy (top) and one from WorldModelBench (bottom).

Figure 6: Training dynamics in terms of VideoPhy scores. SFT starts to saturate beyond 5k steps, while DiReCT continues to improve and stabilizes around 15k steps, with the gap widening particularly on Physical Commonsense (PC).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: The prompt template used to generate the hard negatives.

$\lambda_{\mathrm{anc}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{anc}}$	SA	PC	AVG
0.1	48.57	33.50	41.04
0.2	50.38	34.28	42.33
0.3	50.40	31.69	41.05

Appendix

Table S1: Sensitivity to distributional anchoring strength

\lambda_{\mathrm{anc}}

. SFT only, no contrastive terms.

$\lambda_{\mathrm{rand}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{rand}}$	SA	PC	AVG
0.001	50.52	35.48	43.00
0.005	50.65	36.47	43.56
0.02	49.83	36.82	43.33

Appendix

Table S2: Sensitivity to macro-contrastive strength

\lambda_{\mathrm{rand}}

\lambda_{\mathrm{anc}}{=}0.2

\lambda_{\mathrm{hard}}{=}0.01

$\lambda_{\mathrm{hard}}$	VideoPhy (%) $\uparrow$
$\lambda_{\mathrm{hard}}$	SA	PC	AVG
0.005	50.61	35.97	43.29
0.02	50.78	37.46	44.12
0.1	49.18	37.03	43.11

Appendix

Table S3: Sensitivity to micro-contrastive strength

\lambda_{\mathrm{hard}}

\lambda_{\mathrm{anc}}{=}0.2

\lambda_{\mathrm{rand}}{=}0.005

Method	Training Cost $\downarrow$
Method	Mem. (GB)	Time (s/iter)
SFT	23.29	4.94
DiReCT	30.45	4.95

Appendix

Table S4: Per-iteration training cost. Both methods use Wan 2.1-1.3B at

480\times 832

resolution (81 frames), global batch size 8, DeepSpeed ZeRO-2 on H200 GPUs.

$K$	VideoPhy (%) $\uparrow$
$K$	SA	PC	AVG
8	49.87	34.52	42.20
16	50.61	36.38	43.50
32	51.26	38.16	44.71
64	50.94	36.89	43.92
128	50.18	35.71	42.95

Appendix

Table S5: Effect of cluster count

K

in macro-contrastive sampling (MaNS). We vary the number of semantic clusters used for group-exclusive negative sampling. Too few clusters risk intra-cluster semantic diversity (gradient conflict); too many produce overly fragmented groups with noisy boundaries.

Method	Cosine $\uparrow$
$\Delta$ FM (random neg.)	$0.12\pm 0.21$
DiReCT -MaNS (ours)	$-0.05\pm 0.12$

Appendix

Table S6: Report of gradient alignment between flow-matching and contrastive loss terms for

\Delta

FM and DiReCT according to the cosine similarity metric.

Figure S2: Additional qualitative comparisons with CogVideoX-2B and LTX on VideoPhy prompts. Red boxes highlight physical violations in baseline outputs; green boxes indicate physically consistent behavior produced by DiReCT . In the top example, LTX causes the cookie to dissolve and lose structural integrity upon contact with the coffee, while CogVideoX-2B drops the cookie with no hand visible in frame; DiReCT preserves the cookie’s shape throughout the dipping motion with a plausible hand–cookie interaction. In the middle example, CogVideoX-2B fails to render the shoe entirely, showing only a brush, and LTX exhibits severe visual degradation with color bleeding and loss of object boundaries; DiReCT maintains coherent brush–shoe contact with a realistic polishing motion. In the bottom example, LTX produces unnatural spoon–yogurt interactions with the spoon merging into the mixture, while CogVideoX-2B generates implausible yogurt rigidity; DiReCT produces a physically plausible stirring motion in which the spoon and yogurt maintain distinct material properties throughout.

Contrastive Flow Matching

See all 9 papers

PhysVid: Physics Aware Local Conditioning for Generative Video Models
Mar 27, 2026Saurabh Pathak, Elahe Arani, Mykola Pechenizkiy +1Physics-Guided Video Generation Physical Consistency in Video Generation
Eindhoven University of Technology
PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation
Sep 8, 2026Cong Wang, Hanxin Zhu, Yonglin Tian +5Optical Flow Physical Consistency in Video Generation
State Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Zhongguancun Academy, Beijing 100094, China +2
VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation
Jul 26, 2026Tianxiao Chen, Hanmo Chen, Huajin Chen +3Video-Reference Video Generation Physical Consistency in Video Generation
Zhejiang University · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.

Mar 27, 2026J/K move · Enter open · S save

PhysVid: Physics Aware Local Conditioning for Generative Video Models

Saurabh Pathak, Elahe Arani, Mykola Pechenizkiy, Bahram Zonooz

Eindhoven University of Technology

Physics-Guided Video Generation Physical Consistency in Video Generation Video Generation Physics-Informed Generative Modeling Controllable Video Generation

Open paper AskPDF arXiv

Generative video models achieve high visual fidelity but often violate basic physical principles, limiting reliability in real-world settings. Prior attempts to inject physics rely on conditioning: frame-level signals are domain-specific and short-horizon, while global text prompts are coarse and noisy, missing fine-grained dynamics. We present PhysVid, a physics-aware local conditioning scheme that operates over temporally contiguous chunks of frames. Each chunk is annotated with physics-grounded descriptions of states, interactions, and constraints, which are fused with the global prompt via chunk-aware cross-attention during training. At inference, we introduce negative physics prompts (descriptions of locally relevant law violations) to steer generation away from implausible trajectories. On VideoPhy, PhysVid improves physical commonsense scores by

\approx 33\%

over baseline video generators, and by up to

\approx 8\%

on VideoPhy2. These results show that local, physics-aware guidance substantially increases physical plausibility in generative video and marks a step toward physics-grounded video models.

Abstract

Contrastive Flow Matching

Explore similar work

PhysVid: Physics Aware Local Conditioning for Generative Video Models

PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation

VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation