LPA-CWM: A Learned Physical Adjudicator for Motion Reasoning with Counterfactual World Models

Kunwei Wu; Xiang Liu; Guocai Yao; Junming Chen; Zhikang Chen; Min Zhang; Pengwei Wang; Sen Cui

cs.CVSep 12, 2026

LPA-CWM: A Learned Physical Adjudicator for Motion Reasoning with Counterfactual World Models

Authors: Kunwei Wu, Xiang Liu, Guocai Yao, Junming Chen, Zhikang Chen, Min Zhang, Pengwei Wang, Sen Cui

Organizations: National University of Singapore · Tsinghua University · Beijing Academy of Artificial Intelligence · The Hong Kong University of Science and Technology ·

Abstract

Counterfactual world models (CWM) extract motion from pretrained video predictors by comparing factual and intervened predictions, but uniform aggregation weights responses equally without explicitly incorporating physical priors. Our key insight is to incorporate physical priors into candidate reliability learning, motivating LPA-CWM with a lightweight Learned Physical Adjudicator (LPA). Trained on dense MOVi-F trajectories, the 3.0M-parameter LPA compares visual context and response structure across an unordered candidate set to predict relative weights; windowed localization and one paired re-evaluation recover motion with the CWM frozen. Existing video-level benchmarks do not directly assess motion correspondence, where low localization error can conceal missing trajectory segments. We introduce Completeness-aware Motion Correspondence (CMC), a ground-truth-anchored protocol jointly measuring localization, completeness, visibility, and continuity, counting missing predictions as failures on visible dynamic points. Across DAVIS, Kinetics, and RoboTAP, LPA-CWM improves all main CMC measures over Uniform CWM, with relative gains of 18.1%--60.0% in average Dynamic Correspondence Accuracy ( $\mathrm{DCA}_{\mathrm{avg}}$ ), and improves TAP-Vid First tracking accuracy (overview: https://LPA-CWM.github.io).

Figures & tables

Figure 1: Overview of our work. Uniform CWM can suffer from three characteristic failures in motion reasoning: drifting from moving objects, being distracted by competing motion, and losing track under appearance changes. Our Learned Physical Adjudicator (LPA) identifies reliable counterfactual motion evidence, enabling LPA-CWM to recover more coherent trajectories. We further introduce the CMC protocol to evaluate motion correspondence across localization, completeness, visibility, and continuity. Compared with external methods, Uniform CWM, and APA-CWM, LPA-CWM performs strongly across all four dimensions.

Figure 2: Overview of APA-CWM and LPA-CWM. Given a source–target RGB frame pair $(\mathbf{I}_{t},\mathbf{I}_{t+1})$ and a query point $\mathbf{p}_{t}$ , a frozen CWM generates motion candidates $\{\bm{\Delta}_{m}\}$ . APA-CWM derives candidate weights from the NSG-derived map $\mathcal{P}$ around candidate peaks $\widehat{\mathbf{p}}_{t+1}^{(m)}$ . LPA-CWM instead encodes local crops, full responses, the frame pair, and response statistics; the resulting tokens $\mathbf{z}_{m}$ are processed by a trainable Set Transformer to predict weights. The weighted response undergoes windowed localization and one CWM re-evaluation to obtain refined motion. During LPA training, endpoint errors from MOVi-F trajectories are converted into soft targets $\mathbf{q}$ .

Figure 3: Windowed localization progressively suppresses distant modes. Top: flattened maps from $\pi^{(0)}$ to $\pi^{(2)}$ . Bottom: the corresponding 3D views.

Figure 4: Qualitative motion correspondence. Five methods are compared on identical frames and visible queries across manipulation, human-action, and animal-motion scenes. Badges report mean EPE in pixels. LPA-CWM yields accurate, coherent correspondences across the displayed cases.

Figure 5: CMC summary on DAVIS. Higher is better.

Dataset	Method	CMC (Ours)							TAP-Vid First
Dataset	Method	Cov. $\uparrow$	[email protected] $\uparrow$	DCA $\uparrow$	CTS $\uparrow$	VisBA $\uparrow$	OF1 $\uparrow$	LCC $\uparrow$	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	AD $\downarrow$	MD $\downarrow$	OA $\uparrow$
DAVIS	SMURF	91.98	85.54	11.25	2.67	56.64	14.92	83.29	7.39	14.34	32.34	30.06	78.14
	GMRW	37.02	20.62	23.92	17.00	63.24	45.22	35.46	21.85	41.55	20.59	13.38	50.77
	DODUO	31.44	15.04	22.25	14.42	58.94	45.83	28.44	26.15	52.49	8.79	2.90	49.46
	CoTracker3	92.31	85.33	71.81	70.50	76.43	58.97	81.01	62.90	76.81	4.20	2.01	86.11
	RAFT	68.74	49.58	49.00	31.77	70.16	41.66	62.02	37.08	58.79	15.83	5.60	69.86

Table 1: Cross-dataset evaluation on DAVIS, Kinetics, and RoboTAP. LPA is trained exclusively on MOVi-F and directly evaluated on all three datasets without adaptation. CMC metrics are percentages. For TAP-Vid First, AJ,

<\delta_{\mathrm{avg}}^{x}

, and OA are percentages, while AD and MD are pixels. Bold marks the best result among the three CWM variants. Stronger external results are shaded separately.

Configuration	CMC (Ours)							TAP-Vid First
Configuration	Cov. $\uparrow$	[email protected] $\uparrow$	DCA $\uparrow$	CTS $\uparrow$	VisBA $\uparrow$	OF1 $\uparrow$	LCC $\uparrow$	AJ $\uparrow$	$<\delta_{\mathrm{avg}}^{x}\uparrow$	AD $\downarrow$	MD $\downarrow$	OA $\uparrow$
Architecture ablation
Uniform CWM (reference)	88.10	80.96	36.87	21.46	59.91	33.91	85.20	24.31	40.21	13.10	3.98	76.56
w/o candidate-set interaction	96.58	91.93	51.86	45.55	63.16	38.73	90.30	41.52	56.82	8.99	2.66	82.75
Response + scalar cues only	95.99	90.31	49.75	38.24	61.62	39.77	90.85	41.89	57.20	8.46	3.26	83.61
Full LPA-CWM	98.21	95.63	58.98	57.54	75.89	48.14	93.18	45.29	63.92	6.73	2.07	84.18

Table 2: Ablations on DAVIS under CMC and TAP-Vid First. All configurations use

M=10

and

R=1

. Architecture and objective variants use windowed localization; “Std.” and “Win.” denote standard and windowed localization. Metric conventions follow Table 1 ; best values within each block are bold, enabling direct comparison across design choices.

Panel of Figure 6 — Figure 6: Candidate weighting on MOVi-F validation videos. (a) Candidate-space weighted-coordinate EPE with video-macro means. (b) Weight versus ground-truth candidate rank; the inset reports top-1 hit rates. Error bars and shading show 95% video-cluster bootstrap intervals.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dim.	Statistic	Definition	Support
1	Normalized vertical displacement	Candidate $y$ displacement normalized by image height	Global
2	Normalized horizontal displacement	Candidate $x$ displacement normalized by image width	Global
3	Endpoint response	Response magnitude at the candidate endpoint	Single pixel
4	Global response mean	Mean response magnitude	Full response
5	Global response standard deviation	Population standard deviation of response magnitude	Full response
6	Global response median	Median response magnitude	Full response

Appendix

Table A.1: Candidate scalar descriptor used by LPA. The support column indicates the spatial region used to compute each statistic.

Figure A.1: Evidence chain for the NSG-derived analytic precursor on a real frame pair. The reference frame and observed change define the temporal evidence; score magnitude and score–change alignment produce the analytic reliability map $P$ . APA-CWM averages this map within a local neighborhood around CWM candidate peaks $\hat{p}_{t+1}^{(m)}$ . LPA-CWM does not use the diffusion score map; it learns candidate reliability from the features described in Section 3.3 . Heatmaps are independently normalized for visualization.

Dataset	Method	Cov. $\uparrow$	[email protected] $\uparrow$	DCA@1 $\uparrow$	DCA@2 $\uparrow$	DCA@4 $\uparrow$	DCA@8 $\uparrow$	DCA@16 $\uparrow$	DCA $\uparrow$	CTS4 $\uparrow$	CTS8 $\uparrow$
DAVIS	SMURF	91.98	85.54	1.72	4.11	8.72	14.07	27.62	11.25	2.67	3.33
	GMRW	37.02	20.62	11.47	21.20	26.10	28.57	32.25	23.92	17.00	17.00
	DODUO	31.44	15.04	6.76	18.23	25.89	29.59	30.79	22.25	14.42	15.04
	CoTracker3	92.31	85.33	40.55	64.40	80.49	85.20	88.41	71.81	70.50	74.82
	RAFT	68.74	49.58	29.93	46.92	53.02	56.58	58.53	49.00	31.77	34.59
	SEA-RAFT	57.76	31.25	27.93	40.57	47.35	49.01	50.18	43.01	27.43	30.54

Appendix

Table C.1: Full threshold-wise CMC accuracy results on DAVIS, Kinetics, and RoboTAP. All entries use the same 0–100 scale as Table 1 . Cov. and [email protected] denote Coverage and [email protected]; CTS4 and CTS8 denote CTS@(4,0.8) and CTS@(8,0.8), respectively. Results follow the presentation convention of Table 1 .

Method	DAVIS				Kinetics				RoboTAP
Method	VisBA $\uparrow$	OF1 $\uparrow$	LCC $\uparrow$	Frag. $\downarrow$	VisBA $\uparrow$	OF1 $\uparrow$	LCC $\uparrow$	Frag. $\downarrow$	VisBA $\uparrow$	OF1 $\uparrow$	LCC $\uparrow$	Frag. $\downarrow$
SMURF	56.64	14.92	83.29	21.54	62.69	35.11	82.40	9.66	57.17	28.66	78.28	16.35
GMRW	63.24	45.22	35.46	8.46	65.99	46.34	51.43	7.16	61.48	35.21	28.76	7.59
DODUO	58.94	45.83	28.44	12.88	78.37	58.39	59.83	16.82	73.28	53.09	47.15	17.33
CoTracker3	76.43	58.97	81.01	1.25	78.00	58.56	80.41	8.41	73.11	59.35	75.21	5.05
RAFT	70.16	41.66	62.02	7.62	74.93	60.72	71.71	9.38	70.60	44.27	69.46	22.24

Appendix

Table C.2: Visibility and continuity diagnostics for CMC on DAVIS, Kinetics, and RoboTAP. OF1 denotes occlusion F1. VisBA, OF1, and LCC are percentages; Frag. is the mean fragmentation count

\times 100

. Results follow the presentation convention of Table 1 .

Figure C.1: CMC threshold sensitivity on DAVIS. (a) DCA avg as the dynamic-track motion threshold varies over

\{2,4,8\}

pixels. (b) CTS@(4,

c

) as the coverage threshold varies over

\{0.6,0.7,0.8,0.9\}

with

L_{\rm motion}=4

pixels. Dashed vertical lines indicate the default settings.

Method	Visibility source	Threshold	Sampling
RAFT	Direct query-to-target forward–backward consistency	6 px	Forward at integer query; backward bilinear
SEA-RAFT	Direct query-to-target forward–backward consistency	6 px	Forward at integer query; backward bilinear
DODUO	Direct query-to-target forward–backward consistency	6 px	Forward at integer query; backward bilinear
CoTracker3	Native visibility from official offline predictor	0.9	Native sparse-query trajectory; no additional FB test
SMURF	Native occlusion map	0.5	Query-location lookup
GMRW	Official tracking utility with adjacent-frame forward–backward consistency	3 px	Bilinear at propagated positions

Appendix

Table D.1: Visibility adaptation for external estimators. Thresholds are defined on the corresponding evaluator grid.

Panel of Figure 15 — Figure E.1: Joint sensitivity to mask count and paired re-evaluation depth. Full LPA-CWM results on 40 fixed MOVi-F query groups. (a) Final deployed endpoint EPE. (b) End-to-end inference time per query. $R=0$ omits paired re-evaluation; $M=10,R=1$ is the deployed setting.

Figure F.1: Additional qualitative results on fourteen cases distinct from Figure 4 . Columns show SEA-RAFT, CoTracker3, Uniform CWM, APA-CWM, and LPA-CWM in this order. Every method receives the same frame pair and the same five query identities. A method uses one color throughout; continuous lines connect source queries to target predictions, and badges report mean EPE over the displayed queries.

Point Tracking

Point tracking follows selected or dense image points through video, enabling motion analysis, 3D reconstruction, and precise spatiotemporal understanding.

See all 35 papers

CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model
Sep 19, 2026Ziming Xu, Shuang Liang, Ruobing Han +10Causal World Models Action-Conditioned World Models
Aether AI · University of California, San Diego · Vanderbilt University
Entity-Centric World Models: Interaction-Aware Masking for Causal Video Prediction
May 14, 2026Santosh Kumar PaidiWorld Model Learning Causal World Models
Genentech, Inc.
ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Models
May 9, 2026Haotian Xue, Yipu Chen, Liqian Ma +4Action-Conditioned Video Generation World Models
Georgia Institute of Technology

Sep 19, 2026J/K move · Enter open · S save

CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model

Ziming Xu, Shuang Liang, Ruobing Han, Ziqiao Xi+9

Aether AI · University of California, San Diego · Vanderbilt University

Causal World Models Action-Conditioned World Models Video Prediction Embodied World Models

Open paper AskPDF arXiv

Embodied world models learn to predict future physical dynamics from visual observations and control signals, where physical knowledge is implicitly entangled within latent representations. We introduce CausalWM, a 16B embodied world model that performs explicit causal chain-of-thought reasoning before future video prediction. CausalWM organizes useful variables into a reasoning trajectory, allowing the model to progressively capture causal dependencies underlying physical evolution. To train CausalWM, we collect 31K hours embodied data and develop a three-stage paradigm consisting of large-scale video pre-training, causal CoT mid-training, and multi-objective RL post-training. Despite using only a limited set of supervised CoT variables, CausalWM exhibits emergent in-context learning capabilities, enabling contextual visual feature guidance and efficient few-step generation. CausalWM achieves state-of-the-art performance across language-conditioned, action-conditioned, single-view and multi-view benchmarks, including Top-1 performance on TriWorldBench leaderboard.

Abstract

Point Tracking

Explore similar work

CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model

Entity-Centric World Models: Interaction-Aware Masking for Causal Video Prediction

ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Models