Temporal Specification Guidance for Text-to-Video Generation

Avishree Khare; Hideki Okamoto; Bardh Hoxha; Georgios Fainekos; Rajeev Alur

cs.AIOct 14, 2026

Temporal Specification Guidance for Text-to-Video Generation

Authors: Avishree Khare, Hideki Okamoto, Bardh Hoxha, Georgios Fainekos, Rajeev Alur

Organizations: University of Pennsylvania · Toyota Motor North America (R&D)

Abstract

Text-to-video models often generate the visual concepts requested in natural language prompts but fail to satisfy their intended ordering, timing, and persistence requirements. We propose representing these requirements as specifications in a formal Temporal Logic, and introduce Temporal Specification Guidance (TSG), a training-free inference-time framework that turns temporal specifications into executable objectives for guiding video generation during sampling. TSG scores traces of visual concept evidence using differentiable quantitative semantics and uses the score's gradients to construct a specification-guided denoising direction. We instantiate this framework with video-text attention traces, efficient differentiable scores for Metric Temporal Logic, and gradient-derived attention biases. This instantiation requires neither an external concept classifier nor backpropagation through the video transformer. Its attention-steered forward pass yields a denoising direction that augments classifier-free guidance (CFG). On a benchmark of 220 prompts spanning 11 categories of temporal constraints, including ordered object entry and state transitions, this instantiation improves specification satisfaction accuracy over CFG by 9.5-20 percentage points across five models, with minimal memory overhead. These results demonstrate the potential of temporal specifications and differentiable satisfaction objectives as a flexible interface for controlling video generation.

Figures & tables

Panel of Figure 1 — Figure 1: TSG improves adherence to temporal requirements. Top: CFG produces an amber light for the first two-thirds while TSG follows the expected yellow–red–green progression. Bottom: CFG omits both requested rescue cues; TSG generates a flame signal while retaining the drone. Videos are generated with CogVideoX-5B.

Panel of Figure 2 — Figure 2: Temporal Specification Guidance (TSG). During an additional conditional pass, TSG scores each head’s attention traces against the specification and applies gradient-derived logit biases. The resulting prediction augments CFG. Transformer weights receive no gradients. The example presents videos generated with Wan2.2 5B.

Panel of Figure 3 — Figure 3: TSG benchmark with interval-timed temporal requirements from 11 categories. Shaded intervals indicate permitted event timing within the video duration; $\emptyset$ denotes absence, and “or” denotes mutually exclusive outcomes. Pictograms are illustrative.

Model	Method	Sat Acc. (%)	Img. quality	Latency (min)	Peak GPU mem.
CogVideoX 2B	Base	10.5% (+0.0%)	59.80 (+0.0)	0.75 $\pm$ 0.03 ( $1.00\times$ )	26.30 GiB
	TSG	30.5% (+20.0%)	56.85 (-3.0)	2.62 $\pm$ 0.38 ( $3.49\times$ )	26.33 GiB
CogVideoX 5B	Base	8.6% (+0.0%)	63.72 (+0.0)	1.82 $\pm$ 0.08 ( $1.00\times$ )	31.28 GiB
	TSG	26.8% (+18.2%)	62.18 (-1.5)	6.06 $\pm$ 0.86 ( $3.33\times$ )	31.31 GiB
Wan 2.1 1.3B	Base	15.9% (+0.0%)	69.25 (+0.0)	1.39 $\pm$ 0.02 ( $1.00\times$ )	30.48 GiB
	TSG	31.4% (+15.5%)	67.39 (-1.9)	2.35 $\pm$ 0.17 ( $1.69\times$ )	30.52 GiB

Table 1: TSG improves satisfaction accuracy on a benchmark of 220 prompts. Latency and Memory measurements are on B200 for all models, except for Wan2.2 A14B which was run on H200.

	Method	Temporal Acc $\uparrow$	Latency (min/video) $\downarrow$	Peak GPU mem (GiB) $\downarrow$
One object (N=80)
	Base	69.40% (0.0%)	1.29 $\pm$ 0.01 (1.00 $\times$ )	30.48 (1.00 $\times$ )
	TempoControl (one round)	72.44% (+3.0%)	2.23 $\pm$ 0.01 (1.72 $\times$ )	79.41 (2.61 $\times$ )
	TempoControl (full)	84.05% (+14.6%)	9.87 $\pm$ 2.10 (7.63 $\times$ )	79.43 (2.61 $\times$ )
	TSG	87.02% (+17.6%)	2.02 $\pm$ 0.01 (1.56 $\times$ )	30.52 (1.00 $\times$ )
Two object (N=82)

Table 2: TempoControl benchmark results with Wan2.1 1.3B. Parentheses show accuracy changes in percentage points and computational-cost ratios relative to CFG. Latency reports mean ± standard deviation; memory reports peak allocation. We present setup details in Appendix E.3 .

Panel of Figure 4 — Figure 4: Category-wise results and ablations. Left: mean satisfaction accuracy across five models, with min–max whiskers. Right: guidance-strength (top) and CFG–TSG (bottom) ablations.

Panel of Figure 5 — Figure 5: Representative examples comparing CFG-only sampling (top row of each panel) with TSG (bottom row) on ordered entry, three-state transition, and overlapping handover respectively. The first two examples are generated using Wan2.1 T2V-1.3B, and the third example is generated using Wan2.2 TI2V-5B in text-to-video mode.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Temporal behavior
Two-state transition	A subject or setting changes from state $A$ to $B$ .
Three-state transition	A subject or setting progresses through $A$ , $B$ , then $C$ .
Ordered entry of two entities	One entity is initially present; a second joins it and both remain.
Ordered entry of three entities	Two further entities enter in sequence, accumulating in the scene.
Entity handover with overlap	$A$ alone is followed by $A$ and $B$ together, then $B$ alone.
Entity exit and re-entry	An entity is present, becomes absent, then returns.

Appendix

Table 3: The 11 temporal constraint categories in the main benchmark, each containing 20 examples. Descriptions summarize the temporal structure; individual specifications also encode timing, absence, and persistence requirements.

Entity	Candidate states or attributes
airplane	configuration: landed, in flight
apple	condition: whole, sliced
balloon	inflation: visibly deflated, visibly inflated
bicycle	orientation: upright, lying on its side
bird	configuration: perched, in flight
boat	orientation: upright, capsized

Appendix

Table 4: Examples of object entities with corresponding states.

Entity	Candidate states or attributes
beach	illumination: bright daylight, orange horizon, dark night
city street	illumination: bright daylight, dark night; precipitation: no visible precipitation, visible falling rain, visible falling snow
farm field	ground cover: green crops, bare soil, snow-covered; precipitation: no visible precipitation, visible falling rain, visible falling snow
forest	dominant foliage: green foliage, autumn-colored foliage, completely bare branches, snow-covered trees; precipitation: no visible precipitation, visible falling rain, visible falling snow
kitchen	artificial lighting: lights visibly illuminated, no lights visibly illuminated
lake	surface: visible open water, completely ice-covered; visibility: far shore clearly visible, far shore obscured by fog; precipitation: no visible precipitation, visible falling rain, visible falling snow

Appendix

Table 5: Examples of scene and setting entities with corresponding states.

Model	Resolution	Frames/FPS	Latent frames	Steps	CFG
Wan 2.1 1.3B	$480\times 832$	$81/16$	21	50	6
Wan 2.2 TI2V 5B	$704\times 1280$	$121/24$	31	50	5
Wan 2.2 T2V A14B	$704\times 1280$	$81/16$	21	40	4 / 3
CogVideoX 2B	$480\times 720$	$49/8$	13	50	6
CogVideoX 5B	$480\times 720$	$49/8$	13	50	dynamic (6)

Appendix

Table 6: Generation settings for the TSG benchmark. Wan 2.2 A14B uses CFG scales 4 and 3 for its high- and low-noise experts, respectively. CogVideoX 5B uses its dynamic CFG schedule with scale parameter 6. All methods compared within a model share these settings. The TempoControl comparison uses the Wan 2.1 row.

T2V Generation

Text-to-video generation creates moving visual scenes from natural-language prompts, with growing control over motion, identity, events, and style.

See all 110 papers

Compositional Video Generation via Inference-Time Guidance
May 14, 2026Ariel Shaulov, Eitan Shaar, Amit Edenzon +2Diffusion Model Guidance Video Diffusion Models
1Tel-Aviv University · 2Independent Researcher · 3Bar Ilan University +1
TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models
Jul 29, 2026Taewon Kang, Matthias ZwickerDiffusion Model Guidance Video Diffusion Models
University of Maryland at College Park, United States
TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation
Apr 21, 2026Hongyu Zhang, Yufan Deng, Zilin Pan +6Image-to-Video Generation T2V Generation
Peking University, Shenzhen Graduate School · Zhejiang University · Nanjing University +3

May 14, 2026J/K move · Enter open · S save

Compositional Video Generation via Inference-Time Guidance

Ariel Shaulov, Eitan Shaar, Amit Edenzon, Gal Chechik+1

1Tel-Aviv University · 2Independent Researcher · 3Bar Ilan University+1

Diffusion Model Guidance Video Diffusion Models T2V Generation Video Generation Inference-Time Guidance Compositional Video Generation

Open paper AskPDF arXiv

Text-to-video diffusion models generate realistic videos, but often fail on prompts requiring fine-grained compositional understanding, such as relations between entities, attributes, actions, and motion directions. We hypothesize that these failures need not be addressed by retraining the generator, but can instead be mitigated by steering the denoising process using the model's own internal grounding signals. We propose CVG, an inference-time guidance method for improving compositional faithfulness in frozen text-to-video models. Our key observation is that cross-attention maps already encode how prompt concepts are grounded across space and time. We train a lightweight compositional classifier on these attention features and use its gradients during early denoising steps to steer the latent trajectory toward the desired composition. Built on a frozen VLM backbone, the classifier transfers across semantically related composition labels rather than relying only on narrow category-specific features. CVG improves compositional generation without modifying the model architecture, fine-tuning the generator, or requiring layouts, boxes, or other user-supplied controls. Experiments on compositional text-to-video benchmarks show improved prompt faithfulness while preserving the visual quality of the underlying generator.

Abstract

T2V Generation

Explore similar work

Compositional Video Generation via Inference-Time Guidance

TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models

TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation