When a Larger Guidance Scale Repairs Text-to-Video Failures and When It Does Not
Organizations: University of Maryland at College Park, United States
Abstract
When a text-to-video diffusion model ignores part of a prompt, the usual first fix is to raise the classifier-free guidance scale. This paper is a measurement study of when that fix works. On Mochi, the same increase repairs failures on rare compositions ("a snowy beach") beyond what generating again does, does not repair negations ("a highway with no vehicles"), where it fixes fewer failures than generating again does, and changes nothing for events that must happen late in a clip ("a sandcastle that a wave washes flat"). Which of these happens can be known in advance, from the prompt and one inexpensive score. The score, which we call demand, measures how strongly the denoiser reacts to the part of the prompt that the failed video leaves out, and it is read from the first quarter of denoising, before any video exists. Our main finding is a condition: raising the scale is effective only when the prompt contains no negation and no event tied to a specific time, and when demand is sufficiently high, with recovery clearly higher above a fitted threshold than below it. Published corrections for the three failure types, used as fixed probes, behave as this account predicts. A short decision rule follows: before generating again, a user can tell whether the most common fix is worth trying, and what to use when it is not.
Figures & tables
| Rare composition | Negation | Temporal | |
| Request | a snowy beach with waves | a highway at sunset with no vehicles | a sandcastle on a beach where |
| a wave washes it flat | |||
| Default (made instead) | a tropical beach with waves | a highway at sunset | a sandcastle on a beach |
| Failure rate, untreated | .472 | .365 | .495 |
| Failure rate, scale raised | .297 | .407 | .527 |
| Effect of raising the scale | helps ( ) | does not help (.171 re-seed .250) | no effect ( ) |
| (a) Correlation between demand and failure, within each failure type | |||
|---|---|---|---|
| Negation | Rare comp. | Temporal | |
| Mochi | |||
| HunyuanVideo | |||
| CogVideoX | |||
| (b) AUROC gained by adding demand to all text-only predictors | |||
| One slope | Slope per failure type | Rare comp. only | |
| Method | Failure rate | Recovery | Induction | McNemar |
| Negation, untreated .365 | ||||
| NEGATE † | .230 | .500 | .075 | |
| DCR | .365 | .240 | .138 | |
| TPD | .357 | .253 | .134 | |
| Scale-up | .407 | .171 | .165 | |
| Rare composition, untreated .472 | ||||
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Failure type | Category codes | What the categories vary |
|---|---|---|
| Negation [ 18 ] | AOC, LEN, INA, MNC, SFN, NMI, DNS, SND | what the prompt excludes, and how : Absent Object Consistency (AOC, a highway with no vehicles ); Late Emergence Negation (LEN, no performers appearing ); Implicit Natural-Only Attribute (INA, exclusion implied by only , with no negation word); Multi-Negation Composition (MNC, no desks, no chairs, and no books ); Structural Functional Negation (SFN, holding a tool but not using it ); Non-Inversion Mitigation (NMI, graded, not aggressive ); Double Negation Sensitivity (DNS, not unlit ); and Scoped Negation Disambiguation (SND, chasing a ball that is not moving ) |
| Rare composition [ 19 ] | ENV, TEMP, OBJ, ATTR, SCALE, CTX, MAT, DENS | which binding is made improbable : Environment Recomposition (ENV, a snowy beach ); Temporal Misalignment (TEMP, a rainbow at night ); Object Relocation (OBJ, a penguin in a sandy desert ); Attribute Rebinding (ATTR, a glowing wooden chair ); Scale Shift (SCALE, a giant apple next to a small house ); Contextual Relocation (CTX, a classroom inside a forest ); Material and State Conflict (MAT, a frozen waterfall in summer ); and Composition Density Variation (DENS, a nearly empty street at noon ) |
| Temporal [ 20 ] | ENTRY, SCENE, ONSET, STATE, CAUSAL, WEATHER_NIGHT, ACTIVITY, STRUCTURAL | what kind of change must emerge : entity entry, scene transition, onset of a process, state change, a causal consequence, a weather-to-night shift, an activity change, and structural transformation |
| Agreement | Cohen’s | 95% CI | ||
|---|---|---|---|---|
| Overall | 201 | 0.746 | 0.489 | |
| Negation | 58 | 0.793 | 0.593 | |
| Rare composition | 64 | 0.703 | 0.406 | |
| Temporal | 79 | 0.747 | 0.496 |
| Failure rate | Prompt-level agreement | |||||
|---|---|---|---|---|---|---|
| Failure type | Mochi | Hunyuan | CogVX | M/H | M/C | H/C |
| Negation | 0.365 | 0.442 | 0.388 | 0.496 | 0.474 | 0.480 |
| Rare composition | 0.472 | 0.375 | 0.518 | 0.260 | 0.162 | 0.190 |
| Temporal | 0.495 | 0.555 | 0.688 | 0.383 | 0.365 | 0.384 |
| Failure type | Method | Recovery | Induction | Net change in | McNemar |
|---|---|---|---|---|---|
| Negation | NEGATE (own type) | 0.500 | 0.075 | ||
| Rare composition | DCR (own type) | 0.698 | 0.289 | ||
| Temporal | TPD (own type) | 0.505 | 0.153 | ||
| Temporal | Scale-up | 0.328 | 0.386 |
| Method | Applied to | Slot that receives | Scored by |
|---|---|---|---|
| NEGATE | rare composition, temporal | forbidden-content slot | target failure type’s judge, vs. untreated |
| DCR | negation, temporal | common-counterpart slot | target failure type’s judge, vs. untreated |
| TPD | negation, rare composition | early-scene slot ( ) | target failure type’s judge, vs. untreated |
| Scale-up | negation, rare composition | none (scale only) | target failure type’s judge, vs. untreated |
| Failure type | Untreated | Scale-up | Recovery | Induction | Net | McNemar |
|---|---|---|---|---|---|---|
| Negation | .388 | .255 | .477 | .086 | ||
| Rare composition | .518 | .718 | .111 | .534 |
| Failure type | Replicate | Untreated | Scale-up | Net | Repaired | Broken | McNemar |
|---|---|---|---|---|---|---|---|
| Negation | 1 | .382 | .357 | 16 | 12 | .57 | |
| Negation | 2 | .376 | .382 | 11 | 12 | ||
| Negation | 3 | .365 | .340 | 12 | 8 | .50 | |
| Negation | pooled | .374 | .360 | 39 | 32 | .48 | |
| Temporal | 1 | .431 | .425 | 15 | 14 | ||
| Temporal | 2 | .525 | .450 | 25 | 13 | .07 |
| Failure type | Scale | Pairs | Recovery | Induction | Net recovery [95% CI] | McNemar |
|---|---|---|---|---|---|---|
| Rare comp. | 3.2 | 320 | .231 | .138 | ||
| Rare comp. | 6.4 | 320 | .333 | .069 | ||
| Rare comp. | 9.0 | 480 | .361 | .099 | ||
| Rare comp. | 12.7 | 320 | .419 | .059 | ||
| Negation | 6.4 | 156 | .133 | .083 | ||
| Negation | 9.0 | 470 | .222 | .109 |
| Pooled, failure-type fixed effects | Within one failure type, all predictors | ||||||
| Model | Complexity | Rarity | Encoder | All | Negation | Rare comp. | Temporal |
| Mochi | 0.569 | 0.595 | 0.545 | 0.607 | 0.819 | 0.523 | 0.637 |
| HunyuanVideo | 0.623 | 0.617 | 0.623 | 0.635 | 0.824 | 0.705 | 0.583 |
| CogVideoX | 0.660 | 0.669 | 0.664 | 0.694 | 0.771 | 0.520 | 0.648 |
| Pooled, one demand slope | One slope per failure type | Rare comp., category fixed effects | ||||||
| Model | Text only | demand | AUROC | Text only | demand | |||
| Mochi | 0.607 | 0.632 | 0.658 | 0.588 | 0.628 | |||
| HunyuanVideo | 0.635 | 0.691 | 0.739 | 0.748 | 0.782 | |||
| CogVideoX | 0.694 | 0.695 | 0.706 | 0.689 | 0.739 | |||
T2V Generation
Text-to-video generation creates moving visual scenes from natural-language prompts, with growing control over motion, identity, events, and style.
See all 118 papers