SCOPE: Score-Isolated Agentic Optimization for Video World Models

Yuhua Jiang; Jiaming Wang; Qingbin Liu; Feifei Gao

cs.AIAug 15, 2026

SCOPE: Score-Isolated Agentic Optimization for Video World Models

Authors: Yuhua Jiang, Jiaming Wang, Qingbin Liu, Feifei Gao

Organizations: Tsinghua University · National University of Singapore · Tencent

Abstract

Video world models are increasingly used as simulators for planning and embodied decision making, yet improving them at inference time introduces a subtle evaluation problem: prompts, samplers, verifiers, and selectors may evolve together, making it difficult to attribute gains or prevent held-out feedback from shaping the final policy. We introduce \scope (\scopefullname), a framework for auditable inference-time adaptation of frozen video world models. \scope represents external controls as a typed state, updates this state only through bounded changes supported by development evidence, and freezes the resulting policy before held-out evaluation. On Physics-IQ benchmark, \scope improves over the exact frozen base by $+14.24$ (95% CI $[+8.10,+21.23]$ ). Controlled ablations further identify gains from scene specification, sampling, and learned selection, while the margin over the strongest matched agentic baseline remains unresolved. Cross-backbone and prospective evaluations reveal a complementary result: useful inference-time updates exist, but their benefits do not transfer uniformly across models and settings. Together, these findings suggest that reliable inference-time adaptation requires not only better proposals, but also a principled mechanism for deciding which updates should become part of the deployed system. Code is available at https://github.com/YuhuaJiang2002/SCOPE.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of Scope . Proposal mechanisms in the gray band are replaceable. Scope maintains (1) a typed state over text, noise, verifier, and reward controls; (2) a score-blind deployment choice with an exact Base fallback; (3) a development-time state update that accepts or rejects bounded edits; and (4) a score-isolation constraint that keeps held-out outcomes read-only. Only the external inference state changes; model weights remain frozen.

Method	Wan P-IQ	Wan $\Delta$ (95% CI)	CogVideoX P-IQ	CogVideoX $\Delta$ (95% CI)
Frozen Base	$20.70$	–	$22.91$	–
WMReward reward-only ( Yuan et al., 2026 )	$21.81$	$+1.11\;[-0.40,+3.17]$	$26.10$	$+3.19\;[+0.85,+6.04]$
VAE- $\theta$ reward	$22.33$	$+1.63\;[-0.67,+4.46]$	$25.14$	$+2.23\;[+0.02,+4.83]$
VIGOR reward-only ( Yin et al., 2026 )	$25.15$	$+4.44\;[+1.49,+7.96]$	$27.75$	$+4.83\;[+1.71,+8.44]$
uniform random 3-candidate selector ( Brown et al., 2024 ; Ma et al., 2025 )	$27.41$	$+6.71\;[+1.93,+11.45]$	$30.57$	$+7.65\;[+4.51,+10.97]$
LingBot-style Director $\to$ Pilot proxy † ( Gao et al., 2026 )	$29.98$	$+9.27\;[+2.37,+16.12]$	$34.91$	$+12.00\;[+6.76,+17.62]$

Table 1: Common-base Physics-IQ comparison across two frozen video backbones. All rows report the mean over 40 scene-level Physics-IQ composites. Within each backbone, methods share the same Base and evaluation protocol. The Scope –Qwen differences are

+2.07

on Wan (CI

[-0.27,+6.14]

) and

+0.52

on CogVideoX (CI

[-0.39,+1.65]

), and remain unresolved on both backbones. Absolute scores are not compared across backbones. The dagger denotes the separately generated fixed Director baseline; green indicates Base-relative intervals with positive lower bounds.

Figure 2: Evolution of the matched Physics-IQ policy across two frozen backbones. R0–R3 correspond to Base, the physics-card update, the rule-refinement update, and the final selected Scope state. Curves show the mean Physics-IQ score over 40 scenes, with shaded 95% scene-bootstrap confidence intervals. Scores are compared only within each backbone.

Figure 3: Example of retrieval-assisted generation on Newton’s cradle. Retrieval provides a textual concept card but no reference pixels or visual embeddings. Both Wan2.2 variants use the same temporal anchoring operation. The visualization for OURS additionally includes the disclosed endpoint-motion intervention and display-only hand replacements. This example is qualitative and is not used as evidence for the aggregate tool-use effect.

Ranking-eligible matched methods
Matched method	Wan Overall	Wan $\Delta$ (95% CI)	CogVideoX Overall	CogVideoX $\Delta$ (95% CI)
Frozen Base	$76.67$	–	$74.10$	–
$+$ shared global	$76.86$	$+0.19$ CI $[-2.13,+2.66]$	$74.41$	$+0.30$ CI $[-2.15,+2.77]$
LingBot-style Director $\to$ Pilot $+$ shared global †	$76.77$	$+0.10$ CI $[-2.25,+2.54]$	$74.29$	$+0.18$ CI $[-2.30,+2.69]$
Qwen-Image-Agent-style (SceneLang)	$77.02$	$+0.35$ CI $[-1.50,+2.32]$	$\mathbf{74.72}$	$\mathbf{+0.62}$ CI $[-1.37,+2.68]$
SceneLang $+$ shared global	$77.01$	$+0.34$ CI $[-1.86,+2.63]$	$71.57$	$-2.53$ CI $[-5.25,+0.12]$

Table 2: Cross-backbone results on P-AI (PAI-Bench-G). Deltas and paired 95% CIs are relative to the Frozen Base of the same backbone. SceneLang is the strongest fixed-policy comparator. The diagnostic Scope rows report the better of two variants per backbone and therefore do not represent a single transferable policy.

Method	Wan Mean †	Wan Weighted †	CogVideoX Mean †	CogVideoX Weighted †
Frozen Base ( Team Wan et al., 2025 ; Yang et al., 2025 )	$37.46$	$36.60$	$48.30$	$48.29$
Qwen-Image-Agent-style ( Zhang et al., 2026c )	$38.72$	$37.92$	$46.73$	$46.16$
LingBot-style Director $\to$ Pilot prompt proxy ‡ ( Gao et al., 2026 )	$41.27$	$42.84$	$47.39$	$\mathbf{49.62}$
GPT-5.4 first-frame skill (internal)	$38.31$	$39.33$	$46.67$	$47.99$
Scope (ours)	$\mathbf{41.75}$	$\mathbf{43.17}$	$\mathbf{47.76}$	$49.22$

Table 3: Cross-backbone results on OpenS2V-Eval (

n{=}180

per backbone). Scores are multiplied by

100

and compared only within each backbone. Mean † and Weighted † use the disclosed GPT-5.4 judge with a Qwen fallback proxy. ‡ denotes a prompt-only LingBot-style proxy.

Evaluation	Reference / Criterion	Outcome
Fresh Physics-IQ	Base	$+0.096\;[-0.887,+1.077]$
	Frozen router	$-1.019\;[-2.159,+0.146]$
Typed update	Random update	$-0.316\;[-5.984,+5.145]$
Tool router	Always-on	$\mathbf{+0.660}\;[+0.016,+1.315]$
	No tool	$+0.083\;[-0.191,+0.381]$
	Random routing	$+0.002\;[-0.281,+0.299]$

Table 4: Prospective evaluations. Positive effects favor the proposed method; PhyGround reports proposal coverage against its predeclared minimum.

Category	Variant	Comparison	$\Delta$ (95% CI)
Control	Text (SceneLang)	vs. Base	$+5.62\;[+0.55,+10.55]$
	Noise	vs. fixed sampler	$+1.26\;[+0.52,+2.01]$
	Reward	vs. matched random	$+3.33\;[+1.49,+5.30]$
	Reward	vs. self-score	$+0.66\;[-1.52,+2.92]$
	Region-state prior	identity metric	$-7.28\;[-13.33,-2.53]$
Update	Random update	vs. $\Omega_{0}$	$+4.53\;[+1.14,+9.17]$

Table 5: Ablation studies of Scope . Each row reports a matched comparison and its 95% CI. Component effects are protocol-specific and should not be added across rows.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Policy	Cards	Target	Confirmation	Pooled
Exact no-tool	$0$	$0.4071$	$0.4425$	$0.4189$
Always-on profiled card	$180$	$0.4021$	$0.4352$	$0.4132$
Fit-only group policy	$48$	$0.4096$	$0.4401$	$0.4197$
Same-budget stratified random	$48$	$0.4084$	$0.4424$	$0.4197$

Appendix

Table 6: Selective grounded-router final result. Absolute DisMo cosine similarities reuse the same two candidate arms and three seeds for all four policies. Cards count profiled prompt-card applications across

192

target and

96

confirmation tasks; they are not additional generated videos.

Result	Evaluation role	Authoritative artifact(s)
Matched 40-scene Physics-IQ procedure vs exact Base	same-universe system result	physiq_common_base_result.json ; matched publication in the release manifest
Retrospective LingBot harness proxy	same-protocol retrospective addition	matched score evidence and result in the release manifest
SceneLang text edit	component; three fixed seeds	scenelang_skill_table.json
Fixed sampler edit	component; four fixed seeds	sampler_axis_joint_fourseed_result.json
Supervised reward selector	scene-grouped OOF	reward_gain_ci.json
Integrated and new-task prospective failures	prospective negative / stop	pai_scope_lifecycle_staircase_result.json ; scope_v6_fit_group_final_result.json ; pai_wide_crc_human096_development_completion_r7.json ; pai_wide_completion_guard_development_completion.json

Appendix

Table 7: Result-to-artifact index. Each row links a reported experimental result to its smallest authoritative machine-readable source.

Setting	Independent unit	Evaluation role	Primary comparison
GPT-5.4 SceneLang	scene; 3 fixed seeds averaged	component evidence	SceneLang vs Base; CI conditional on these seeds
Learned reward	64 scenes; 179 nested views	component evidence	scene-grouped OOF reward vs matched random
Physics-IQ tool axis	scene; 3 nested views	preregistered negative ablation	web-only and web+SAM vs no-tool; incremental SAM vs web-only
PAI split6	task	same-split deployment diagnostic	all matched controls shown; post-hoc Scope separated
OpenS2V	category-stratified item	prospective deployment case	Scope vs LingBot; unresolved
Fresh PhyT2V evaluation	scene	deployment diagnostic	selected vs incumbent/fixed proposal; no expansion credit

Appendix

Table 8: Protocol and statistical status matrix. Views are nested within scenes for Physics-IQ; secondary, same-split, and post-hoc rows never receive promotion credit.

Method	SC	BC	MS	AQ	IQ	OC	IS	IB
Frozen base (absolute)	89.51	91.71	98.49	44.21	62.83	19.50	92.44	95.75
Shared global (Base parent)	-0.45	-0.18	-0.73	+0.61	+1.60	-0.12	-0.09	-0.02
Qwen-Image-Agent-style (SceneLang) ( Zhang et al., 2026c )	+0.89	+0.37	+0.13	+0.42	+0.50	-0.06	+0.66	+0.49
SceneLang + shared global	-0.03	+0.19	-0.65	+0.79	+2.25	-0.04	+0.10	+0.03
LingBot-style + safe fallback † ( Gao et al., 2026 )	+0.24	+0.10	+0.03	0.00	+0.28	-0.02	+0.13	+0.12
LingBot-style Director $\to$ Pilot + shared global † ( Gao et al., 2026 )	-0.16	+0.02	-0.70	+0.70	+2.17	-0.22	+0.06	+0.07

Appendix

Table 9: Complete official-72B PAI quality profiles. SC/BC are subject/background consistency; MS is motion smoothness; AQ/IQ are aesthetic/imaging quality; OC is overall text–video consistency; IS/IB are I2V subject/background fidelity. Frozen Base is absolute and all other rows are

\Delta

vs Base. All 12 main-panel rows are included.

Figure 4: Web-grounded complete paper submersion intervention. Both rows use the same straw-free seed- $20260785$ Wan2.2 baseline. The unexplained overhead straw is inpainted once in their shared conditioning frame. WITHOUT OURS shows the complete baseline frame, in which only the lower paper edge contacts the water. OURS applies a disclosed post-hoc full-frame intervention: the complete paper mask is compressed onto the dish bottom; every final paper pixel lies within the audited water ellipse; and water tint, caustics, refraction, and a ripple band are rendered after the paper. Both published A/B MP4s and the displayed frames are complete $832\!\times\!480$ images, with dish cropping disabled in the manifest. The post-absorption Wikimedia image is display-only and never enters either video. Retrieved pixels likewise never enter the video. The native four-seed prompt grid failed the qualitative gate and is not shown. This visualization is a qualitative post-hoc state intervention, not native Wan dynamics, an unbiased generation result, or benchmark evidence.

Adaptive Inference

Adaptive inference allocates computation, model choice, or data collection dynamically to improve efficiency while preserving reliable predictions.

See all 95 papers

RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation
Jul 1, 2026Byeongguk Jeon, Seonghyeon Ye, JaeHyeok Doo +4World Model Learning Robotic Policy Evaluation
1KAIST · 2Config
Video World Models Need Predictable Latents
Oct 26, 2026Mattia Scardecchia, Sherry YangVideo World Models World Model Learning
New York University
WorldAgen: Unified State-Action Prediction with Test-Time World Model Training
Sep 8, 2026Chi Wan, Kangrui Wang, Yuan Si +2Test-Time Adaptation Action-Conditioned World Models
Northwestern University

Jul 1, 2026J/K move · Enter open · S save

RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation

Byeongguk Jeon, Seonghyeon Ye, JaeHyeok Doo, Sungdong Kim+3

1KAIST · 2Config

World Model Learning Robotic Policy Evaluation World Model Evaluation Video World Models

Open paper AskPDF arXiv

Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with a task-progress-aware vision-language model scoring. To enable reliable long-horizon autoregressive world-model rollouts, we propose Step Forcing, which combines anchored and one-step self-forwarded contexts to reduce train-test mismatch while preserving action-observation dynamics. Together, these components enable RoboWorld to align strongly with real-world robot evaluation across tasks and environments, achieving Pearson's r = 0.989 and Spearman's

ρ

= 0.970.

Abstract

Adaptive Inference

Explore similar work

RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation

Video World Models Need Predictable Latents

WorldAgen: Unified State-Action Prediction with Test-Time World Model Training