Frequency-Step Cache: Separating Synchronization and Repair for Efficient Autoregressive Video Diffusion

Kerry Xu; Param Misra; Zachary Stenglein; Athri Karunamurthy; Fanjiang Ye; Yuke Wang

cs.CVOct 11, 2026

Frequency-Step Cache: Separating Synchronization and Repair for Efficient Autoregressive Video Diffusion

Authors: Kerry Xu, Param Misra, Zachary Stenglein, Athri Karunamurthy, Fanjiang Ye, Yuke Wang

Organizations: Rice University

Abstract

Autoregressive (AR) video generation enables long-horizon synthesis but requires repeated forward passes through large diffusion transformers. Training-free caching reduces this cost by reusing intermediate computation, yet existing controllers often couple two decisions: when to synchronize the cache exactly and how much computation to repair between synchronizations. We introduce Frequency-Step Cache (FSC), which separates these roles. FSC schedules exact full-computation anchors on a dense-to-sparse trajectory, while a lightweight latent-space risk based on frequency-resolved drift and token age controls inter-anchor repair through whole-step replay, selective token recomputation, or full repair. On SkyReels-V2, whole-step reuse provides most of the acceleration, while selective token computation is most effective as fidelity repair. A controlled ablation shows that using one reactive signal for both synchronization and repair reduces fidelity at matched latency, with the same separation advantage observed on FramePack and Matrix-Game. Across multiple autoregressive video backbones, FSC improves speed-quality trade-offs. On SkyReels-V2, FSC-Speed reaches 2.36x speedup versus MotionCache's 2.275x while improving SSIM by 0.019 and PSNR by 0.5 dB. At MotionCache-matched speed, FSC-Quality improves SSIM by 0.031 and PSNR by 1.0 dB and lowers LPIPS by 0.024. Without retuning, FSC-Quality transfers to SkyReels-V2 14B at 2.12x speedup versus 1.79x for MotionCache.

Figures & tables

Figure 1: Three limitations of naive caching: reuse error depends on the step, token-selective repair provides limited speedup, and approximate anchors rarely help (Observations 1–3). SkyReels-V2, controlled profiling setting (App. A.1 ): (a) 14B; (b)–(d) 1.3B. Anchor counts differ because each panel is a separate controlled study. (a) Relative-

L_{2}

error from reusing each block’s previous-step residual (band: interquartile range over blocks and prompts). (b) SSIM vs. same-seed vanilla for three placements of

7

(

8

for uniform) exact anchors: dense-to-sparse (D2S,

0.861

) outperforms uniform (

0.856

) and sparse-to-dense (S2D,

0.830

). (c) Speedup vs. token fraction

q

. True sparse kernels reach

1.94\times

q{=}0.10

, far below their compute-proportional

2.72\times

; the dense-forward token-refresh reference stays at

\approx 1.23\times

by design (dashed: its ideal). Whole-step D2S reuse reaches

2.62\times

. (d) SSIM vs. speedup for anchor policies (“clean” circles: exact anchors; “approximate”:

\triangledown

pyramid kernels,

\triangle

block-skip gates,

\Diamond

other variants). Of

17

approximate variants, only three block-skip gates near

3.0\times

lie above the exact-anchor frontier (black line), by at most

0.008

SSIM. The pyramid-full variant has substantially lower SSIM than the clean anchor baseline.

Figure 2: Anchors are fixed before generation; full repairs are chosen by the risk

\rho_{s}

and leave the schedule unchanged. (a) When : dense warmup, then dense-to-sparse anchors. (b) How much : drift from

\mathbf{X}_{\mathrm{ref}}

and cache age form

\rho_{s}

, which selects replay (

C0

), token repair (

C1

–

C3

), or full repair. An anchor replaces

\mathbf{X}_{\mathrm{ref}}

; a full repair keeps it.

Figure 3: The staleness risk $\rho_{s}$ is strongly associated with exact reuse error (Spearman $0.893$ ) in an offline diagnostic. SkyReels-V2 1.3B, $4{,}896$ non-anchor decisions on the eight development prompts (App. C ). (a) $\rho_{s}$ versus the oracle reuse error $E_{s}$ ; within each prompt the Spearman correlation is $0.909$ – $0.972$ . (b) Mean, median, and $95$ th-percentile $E_{s}$ per $\rho_{s}$ decile; mean and median rise monotonically.

Model	Method	PFLOPs	Speedup	Latency	Mem	VBench	PSNR	SSIM	LPIPS
		$\downarrow$	$\uparrow$	(s) $\downarrow$	(GB) $\downarrow$	$\uparrow$	$\uparrow$	$\uparrow$	$\downarrow$
SkyReels-V2	Vanilla	37.7	1.000 $\times$	367	53.7	0.7977	–	–	–
	FlowCache	17.4	2.037 $\times$	180	58.1	0.7939	17.57	0.7946	0.3073
	MotionCache	15.4	2.275 $\times$	161	58.9	0.7898	17.34	0.7899	0.3366
	SeaCache-AR ∗	14.9	2.320 $\times$	158	58.6	0.7886	17.52	0.7919	0.3331
	FSC-Quality (ours)	14.8	2.275 $\times$	161	58.4	0.7900	18.38	0.8209	0.3122

Table 1: FSC-Quality matches MotionCache’s speed with higher fidelity, and FSC-Speed is faster while keeping higher SSIM and PSNR. Speed and PFLOPs: Bench (MAGI-1: speedups on the vanilla-clean prompt pool after per-method latency auditing, MotionCache and FSC PFLOPs on a

20

-prompt sample); VBench: the

505

Bench prompts of the evaluated dimensions (

500

on MAGI-1); PSNR/SSIM/LPIPS: Fid (

110

on MAGI-1). PFLOPs come from executed-token counters, warmup forwards included, on a different basis per backbone; Mem: peak memory (both App. D ). ∗ Our latency-calibrated SeaCache port (App. E.3 ). † MAGI-1 tests transfer through its native cache interface (Sec. 4.4 ). Best accelerated result per backbone in bold , second best underlined .

(a) Equal latency, SkyReels-V2 1.3B (Paired)
Method	Speedup $\uparrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
MotionCache	2.274 $\times$	0.7885	17.55	0.7789	0.3156
FlowCache (threshold $0.123$ )	2.305 $\times$	0.7922	17.46	0.7715	0.3125
SeaCache-AR (threshold $\delta{=}0.28$ )	2.321 $\times$	0.7908	17.68	0.7826	0.3222
FSC-Quality	2.275 $\times$	0.7883	18.87	0.8277	0.2840
(b) SkyReels-V2 14B, 1.3B configuration unchanged (14B)

Table 2: FSC maintains higher fidelity at matched latency and higher SSIM and PSNR on 14B without retuning. Paired: speedup on all

235

prompts, VBench on

126

, fidelity on

21

. 14B (

352\times 640

): VBench is evaluated on

25

of the

47

prompts; all other metrics use all

47

. Best in bold , second best underlined per block.

Figure 4: Operating-point robustness and the effect of separating synchronization from repair. (a) SkyReels-V2 1.3B; speedup: mean over the

235

Paired prompts; SSIM: their

21

prompts with vanilla references. Outside the MotionCache threshold sweep, FSC retains higher SSIM than the shared-stack baselines at comparable speed, and MotionCache’s released

\tau{=}0.10

is its latency-matched point. Lines join each method’s operating points in speed order (

\tau

, thr,

\delta

: MotionCache, FlowCache, and SeaCache-AR thresholds; rel.: released; FSC: FSC-Quality with

13

anchors and

r{=}1.40

, FSC-Quality, FSC-Speed, and two other

W{=}5

variants of Table 6 ). (b) Dev set (Table 3 ); speedup: mean over its

8

prompts. The purely reactive variant remains below FSC-Quality at every tested latency; its SSIM gap narrows from

0.13

(matched latency) to

0.07

(

31\%

more latency); hollow: anchors only, no repair.

Variant	SSIM $\uparrow$	LPIPS $\downarrow$	s/video $\downarrow$	Wins (of 8)
				SSIM	LPIPS
FSC-Quality: D2S anchors + repair	0.6641	0.2925	161	–	–
purely reactive, matched latency ( $r{=}1.076$ )	0.5332	0.4786	160	8	8
purely reactive, $13\%$ slower ( $r{=}0.631$ )	0.5575	0.4285	182	8	8
purely reactive, $31\%$ slower ( $r{=}0.250$ )	0.5976	0.3569	211	8	6
anchors only, no repair ( $13$ D2S anchors)	0.5940	0.3337	153	8	8

Table 3: Separating synchronization from repair: a purely reactive variant reduces SSIM by about

0.13

at matched latency; increasing its latency by

13\%

31\%

narrows but does not close the gap. SkyReels-V2, Dev set; rows modify FSC-Quality (D2S: dense-to-sparse; purely reactive: no anchors,

\rho_{s}

triggers repair and full synchronization;

r

: threshold scale); s/video: median wall-clock. Absolute scores are not comparable with Table 1 (different prompts and fidelity harness). The last two columns count the prompts on which FSC-Quality outperforms each variant; further variants in App. E.10 . Best in bold , second best underlined .

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Token budget $q$	0.10	0.15	0.20	0.30
Compute-proportional ideal (token refresh)	3.18 $\times$	2.93 $\times$	2.72 $\times$	2.37 $\times$
Compute-proportional ideal (true sparse)	2.72 $\times$	2.55 $\times$	2.40 $\times$	2.16 $\times$
Token refresh (dense-forward reference)	1.23 $\times$	1.23 $\times$	1.23 $\times$	1.23 $\times$
True sparse kernels	1.94 $\times$	1.83 $\times$	1.78 $\times$	1.65 $\times$
Whole-step D2S reuse (reference)	2.62 $\times$

Appendix

Table 4: True sparse token repair stays far below its compute-proportional ideal, and whole-step dense-to-sparse (D2S) reuse is faster. End-to-end speedup of the true sparse kernels and of the dense-forward token-refresh reference (flat by construction) versus their compute-proportional ideals (for token refresh, the speedup its schedule would reach if only the selected tokens were computed; SkyReels-V2 1.3B, profiling setting of App. A.1 ); the ideals differ because token refresh uses

3

exact anchors and true sparse kernels use

6

Figure 5: High-frequency residual content goes stale faster than low-frequency content, but the separation is content-dependent (Observation 4; SkyReels-V2 14B, six profiling prompts). (a) Relative-

L_{2}

reuse error versus cache age (number of reuse steps since the cached state was refreshed) for the low-frequency band of the block residuals (two levels of

2\times 2

average pooling) and the high-frequency remainder, each measured against the corresponding full-compute band; HF error grows faster in absolute terms and remains

\approx 2

–

2.5\times

the LF error across ages

1

–

10

(up to

2.7\times

for individual prompts; (b)). (b) The HF/LF error ratio from (a) for the lowest-error and highest-error profiling prompts (“easy prompt” and “hard prompt” in the legend); it stays above one throughout but is smaller on the highest-error prompt and shrinks with age.

Figure 6: The FSC risk

\rho_{s}

detects unsafe reuse (

E_{s}>0.05

) with ROC-AUC

0.930

. The

4{,}896

non-anchor decisions of Fig. 3 (our development set, SkyReels-V2 1.3B, development configuration described above). The oracle probe leaves the anchor reference and routing-state variables unchanged; the K/V-cache side effect is described above. Dashed: chance.

	RTX PRO 6000 (946)	RTX PRO 6000 (8)	H100 (10)	H100 (23)
Vanilla latency (s)	367	367	300	300
MotionCache speedup	2.275 $\times$	2.285 $\times$	2.278 $\times$	2.259 $\times$
FSC speedup	2.275 $\times$	2.274 $\times$	2.280 $\times$	2.277 $\times$
$\Delta$ SSIM (FSC $-$ MC)	$+0.031$ (72/102)	$+0.053$ (8/8)	$+0.021$ (7/10)	$+0.028$ (18/23)
$\Delta$ LPIPS (FSC $-$ MC)	$-0.024$ (66/102)	$-0.060$ (6/8)	$-0.009$ (5/10)	$-0.021$ (15/23)
Peak mem. vanilla / MC / FSC (GB)	53.7 / 58.9 / 58.4	–	53.7 / 59.0 / 58.5	53.7 / 58.4 / 58.5

Appendix

Table 5: At MotionCache-matched latency, FSC’s fidelity advantage has the same sign on both GPUs. Same code on two GPUs (SkyReels-V2 1.3B, seed

1024

; speedup and fidelity against the local vanilla; paired FSC

-

MotionCache differences with win counts). RTX columns: the FSC-Quality point of Table 1 (

946

prompts; fidelity on its

102

-prompt subset) and a separate

8

-prompt re-measurement: eight prompts of the

102

-prompt fidelity subset, one per MotionCache-protocol dimension and disjoint from the development set; vanilla, released MotionCache (

\tau{=}0.10

) and FSC-Quality were generated in one session on the same GPU, and fidelity is against that session’s vanilla with the ablation-harness metrics of Table 3 , so this column is comparable neither with the

946

-prompt column nor with the development-set Table 11 ; RTX peak memory (fresh-process protocol above) was measured for the

946

-prompt column only; H100 columns: the

W{=}7

n{=}9

configuration (

10

prompts) and FSC-Quality’s anchor schedule with

q_{\mathrm{scale}}{=}0.8

(

23

prompts; see text); H100 peak memory comes from the benchmark worker processes, and in the

23

-prompt column MotionCache ran after FSC in the same process and inherited FSC’s per-window cache-clearing flag, which lowers its resident memory (

58.4

vs.

59.0

GB) but not its outputs;

W

: warmup,

n

: anchors,

q_{\mathrm{scale}}

: repair-fraction scale.

Figure 7: FSC-Speed exhibits smaller deviation from the vanilla trajectory than MotionCache over time. SkyReels-V2 1.3B, two prompts, same seed; rows: vanilla, MotionCache, and FSC-Speed; columns: frames at four timestamps. Early frames are nearly identical and deviations accumulate with time: MotionCache drifts in object identity and layout (left: the cap morphs and the paper drifts away) and in framing and color (right: progressive zoom and saturation).

Configuration (relative to FSC-Quality)	Speedup $\uparrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
FSC-Quality (Table 1 )	2.275 $\times$	0.7900	18.38	0.8209	0.3122
$W{=}7$ , $n{=}9$ , $r{=}1.75$ , $q_{\mathrm{scale}}{=}1.00$ , $p_{\mathrm{LF}}{=}0.1$	2.277 $\times$	0.7853	18.31	0.8202	0.3199
$W{=}5$ , $r{=}1.55$ (FSC-Speed, Table 1 )	2.356 $\times$	0.7892	17.87	0.8088	0.3313
$W{=}5$	2.378 $\times$	0.7892	17.88	0.8091	0.3316
$W{=}5$ , tail anchors at steps $44$ and $47$	2.445 $\times$	0.7854	17.79	0.8073	0.3464
MotionCache	2.275 $\times$	0.7898	17.34	0.7899	0.3366

Appendix

Table 6: Every SkyReels-V2 FSC operating point improves PSNR and SSIM over MotionCache at a VBench aggregate within

0.005

of it. Full

946

-prompt benchmark, ordered by speedup (VBench on the fixed dimension subset of App. D ; PSNR/SSIM/LPIPS on the

102

-prompt fidelity subset against the same-seed vanilla output). Configurations are described relative to FSC-Quality (

W{=}6

n{=}10

dense-to-sparse (D2S) anchors,

r{=}1.65

q_{\mathrm{scale}}{=}1.25

, LF gate on with

p_{\mathrm{LF}}{=}0.2

). The two

W{=}5

rows without tail anchors are separate runs that differ only in

r

(

1.55

vs.

1.65

); the tail-anchor point adds exact steps

44

and

47

to the

r{=}1.65

row. Adding tail anchors changes the realized repair routing because anchors reset controller state: the

W{=}5

r{=}1.65

row routes

5.7\%

of its non-anchor decisions to risk-triggered full repair,

2.9\%

C2

and

8.6\%

C3

; adding the tail anchors changes these shares to

0

0

and

3.0\%

, hence the higher speed (

C0

: whole-step replay;

C1

–

C3

: token repair of increasing fraction; Full: full repair;

W

: warmup;

n

: anchors;

r

: threshold scale). Best in bold , second best underlined .

Figure 8: Every FSC operating point has higher SSIM than MotionCache at equal or higher speed, with VBench within

0.005

. SkyReels-V2, full benchmark (VBench on the fixed dimension subset of App. D ; SSIM and LPIPS on the

102

-prompt fidelity subset); FSC-Quality, FSC-Speed, and the two other

W{=}5

points of Table 6 (line: FSC points in speed order); the dashed line marks MotionCache’s speedup.

Configuration (relative to FSC-Quality)	Anchors	$k_{\mathrm{tok}}$	Speedup $\uparrow$	$\Delta$ VBench $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$r{=}1.0$ , 12 anchors, $k_{\mathrm{tok}}{=}8$	12	8	1.393 $\times$	$-0.0029$	0.8564	0.2722
FSC-Quality (Table 1 ): dense anchors, $r{=}1.25$	15	4	1.421 $\times$	$\mathbf{+0.0004}$	0.8883	0.2297
FSC-Speed (Table 1 ): $k_{\mathrm{tok}}{=}8$	15	8	1.457 $\times$	$-0.0054$	0.8782	0.2460
MotionCache	–	–	1.357 $\times$	–	0.8468	0.2778

Appendix

Table 7: On MAGI-1, every FSC configuration is faster than MotionCache with higher SSIM and lower LPIPS. Configurations evaluated over the full benchmark prompt pool; speedup uses the vanilla-clean prompt pool after per-method latency auditing, VBench the

500

prompts of the evaluated subset, fidelity the first ten prompts of each prompt set (

110

; App. D ).

\Delta

VBench is relative to MotionCache (

0.7950

); its paired bootstrap

95\%

CI over prompts contains zero for every row (

[-0.010,+0.004]

for the twelve-anchor configuration,

[-0.007,+0.008]

for FSC-Quality,

[-0.013,+0.002]

for FSC-Speed).

k_{\mathrm{tok}}

: forced full-recomputation period;

r

: threshold scale (App. B.5 ). Best in bold , second best underlined ; the

\Delta

VBench marks rank the FSC rows only (MotionCache is the reference).

Backbone	Comparator	Speedup FSC/comp. $\uparrow$	Latency ratio	$\Delta$ PSNR (dB)	$\Delta$ SSIM	$\Delta$ LPIPS
FramePack	coupled ( $r{=}0.9$ )	2.473 $\times$ /2.513 $\times$	0.9685 $[0.9676,0.9707]$	$+5.18$ $[+4.03,+6.35]$ , 40/0	$+0.079$ $[+0.065,+0.094]$ , 40/0	$-0.092$ $[-0.105,-0.080]$ , 39/1
Matrix-Game	coupled ( $r{=}0.543$ )	2.445 $\times$ /2.535 $\times$	0.9667 $[0.9629,0.9692]$	$+2.77$ $[+2.53,+3.00]$ , 40/0	$+0.058$ $[+0.043,+0.075]$ , 36/4	$-0.057$ $[-0.067,-0.047]$ , 39/1

Appendix

Table 8: On FramePack and Matrix-Game, FSC outperforms a latency-matched coupled comparator in PSNR, SSIM, and LPIPS, with every 95% CI excluding zero. Cross-family validation on

40

held-out prompts per backbone: FSC minus comparator, mean with stratified bootstrap 95% CI and prompt wins (FSC/comparator). Latency ratio is the same-GPU median per-prompt comparator/FSC latency (each prompt’s controllers run back to back on one idle GPU; bootstrap 95% CI), and speedup is end-to-end relative to the same-seed vanilla run; speedups are means of per-prompt ratios and can differ from the median latency ratio. Pre-registered matching criterion: median ratio in

[0.90,1.10]

. Coupled: the purely reactive variant of Sec. 5.3 (the same controller without scheduled anchors, one risk signal triggering both repair and full synchronization);

r

: its threshold scale.

Method	Latency (s) $\downarrow$	Speedup $\uparrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
faster operating point
MotionCache	161	2.274 $\times$	0.7885 [0.758, 0.817]	17.55	0.7789	0.3156
FlowCache (thr $0.123$ )	159	2.305 $\times$	0.7922 [0.762, 0.821]	17.46	0.7715	0.3125
SeaCache-AR ( $\delta{=}0.28$ )	158	2.321 $\times$	0.7908 [0.761, 0.820]	17.68	0.7826	0.3222
FSC-Quality	161	2.275 $\times$	0.7883 [0.757, 0.817]	18.87	0.8277	0.2840
slower operating point

Appendix

Table 9: At both latencies, the FSC point has the highest PSNR and SSIM. Paired comparison on the

235

-prompt stratified subset (SkyReels-V2, seed

1024

; VBench on the

126

subset prompts carrying the evaluated dimensions,

124

for the slower block, with bootstrap

95\%

CIs over prompts; fidelity on the

21

prompts of the subset that have vanilla references). SeaCache-AR is our chunk-wise port of SeaCache’s scheduling rule (see text); thr,

\delta

: FlowCache and SeaCache-AR reuse thresholds;

r

: FSC threshold scale. Best in bold , second best underlined within each block.

	Speedup		VBench ( $41$ of the $78$ prompts)				FSC $-$ MotionCache, same-seed fidelity
Seed	MC	FSC	vanilla	MC	FSC	$n$	$\Delta$ PSNR (wins)	$\Delta$ SSIM (wins)	$\Delta$ LPIPS (wins)
1024	2.274	2.274	0.7802	0.7892	0.7698	13	$+1.06$ (11/13)	$+0.040$ (9/13)	$-0.028$ (9/13)
7	2.275	2.275	0.7748	0.7829	0.7736	78	$+1.71$ (71/78)	$+0.044$ (63/78)	$-0.043$ (57/78)
42	2.275	2.274	0.7875	0.7851	0.7879	78	$+0.91$ (55/78)	$+0.021$ (55/78)	$-0.016$ (52/78)
123	2.271	2.273	0.7898	0.7826	0.7835	78	$+1.38$ (65/78)	$+0.036$ (65/78)	$-0.039$ (56/78)
2026	2.276	2.275	0.7855	0.7827	0.7701	78	$+1.20$ (69/78)	$+0.033$ (67/78)	$-0.028$ (55/78)

Appendix

Table 10: FSC-Quality’s fidelity advantage over MotionCache replicates at all five seeds at identical speed. Seed robustness on the

78

-prompt subset stratified over the benchmark prompt sets (SkyReels-V2). VBench aggregate per seed for all three methods; speedup of MotionCache / FSC-Quality over the same-seed vanilla; same-seed fidelity difference FSC-Quality

-

MotionCache with the number of prompts on which FSC-Quality wins;

n

= prompts with a same-seed vanilla reference (

78

per added seed; for seed

1024

the

13

-prompt overlap with the fidelity subset, scored with the in-pipeline metrics rather than the offline metrics of the added seeds). Mean

\pm

sd rows use the unrounded per-seed values.

$\tau$	s/video $\downarrow$	Speedup $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Paths (%)
0.07	188	1.960 $\times$	17.89	0.6569	0.2806	54/33/13
0.075 s	181	2.026 $\times$	17.67	0.6399	0.3040	54/28/18
0.08	177	2.062 $\times$	17.71	0.6433	0.3073	54/26/20
0.085	175	2.100 $\times$	17.89	0.6483	0.2960	54/27/19
0.09	165	2.210 $\times$	17.49	0.6391	0.3058	62/27/11
0.1 m	162	2.267 $\times$	17.42	0.6380	0.3113	61/23/16

Appendix

Table 11: On the development prompts, MotionCache’s released

\tau{=}0.10

lies at FSC-Quality’s latency, where FSC-Quality has higher SSIM. MotionCache threshold sweep on our development set (eight prompts; seed

1024

, one idle GPU). Paths: share of group decisions that reuse / fully update / selectively update. s,m,f : frozen slow, matched and fast points. The FSC-Quality row is run 1 of App. E.12 ; Table 3 lists the stored reference run (LPIPS

0.2925

). s/video: median wall-clock. Best in bold , second best underlined .

Figure 9: On the development prompts, every perturbed FSC-Quality point has higher SSIM than the MotionCache threshold sweep at equal latency. SkyReels-V2, our development set (eight prompts); MotionCache’s

\tau

sweep of Table 11 (released

\tau{=}0.10

marked), FSC-Quality, and its local and stress perturbations of App. E.12 .

Method	s/video $\downarrow$	Speedup $\uparrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
MotionCache $\tau{=}0.075$ (slow)	180	2.035 $\times$	0.7930 $[0.761,0.823]$	17.84 (FSC $+2.22$ )	0.7869 (FSC $+0.073$ , 17/4)	0.2911 (FSC $-0.048$ , 13/8)
MotionCache $\tau{=}0.10$ (matched; released value)	161	2.278 $\times$	0.7870 $[0.756,0.816]$	17.55 (FSC $+1.32$ )	0.7784 (FSC $+0.049$ , 16/5)	0.3158 (FSC $-0.032$ , 15/6)
MotionCache $\tau{=}0.11$ (fast)	151	2.425 $\times$	0.7713 $[0.740,0.801]$	17.29 (FSC $+1.58$ )	0.7718 (FSC $+0.056$ , 19/2)	0.3389 (FSC $-0.055$ , 20/1)
FSC-Quality	161	2.275 $\times$	0.7840 $[0.752,0.814]$	18.87	0.8277	0.2840
FSC-Quality, $13$ anchors, $r{=}1.40$	182	2.015 $\times$	0.7796 $[0.746,0.810]$	20.07	0.8600	0.2435

Appendix

Table 12: FSC keeps its fidelity advantage over every frozen MotionCache point outside the sweep set. Frozen MotionCache operating points on the

235

prompts outside the threshold-sweep set (VBench on the

124

carrying the evaluated dimensions, with bootstrap 95% CI; PSNR/SSIM/LPIPS on the

21

paired fidelity prompts). FSC rows are the stored results of App. E.3 . Parentheses: fidelity margin of the nearest-latency FSC point (the

13

-anchor point for

\tau{=}0.075

, FSC-Quality otherwise), computed from unrounded means, and FSC wins / MotionCache wins on the

21

prompts; paired VBench differences:

+0.0134

[+0.0037,+0.0240]

+0.0030

[-0.0070,+0.0136]

-0.0127

[-0.0227,-0.0035]

(MotionCache

-

FSC). s/video: median wall-clock;

\tau

: MotionCache reuse threshold;

r

: threshold scale. Best in bold , second best underlined .

Runtime stack	Method	Own vanilla (s)	Latency (s) $\downarrow$	Speedup (own vanilla) $\uparrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
shared (Table 1 )	MotionCache	367	161	2.275 $\times$	0.7898	17.34	0.7899	0.3366
shared (Table 1 )	FSC-Quality (ours)	367	161	2.275 $\times$	0.7900	18.38	0.8209	0.3122
official SCOPE (upstream)	SCOPE, released configuration	589	123	4.803 $\times$	0.7748	12.10	0.5099	0.5696
official SCOPE (upstream)	SCOPE, no selective computation †	589	155	3.813 $\times$	0.7889	14.37	0.6840	0.4380

Appendix

Table 13: SCOPE is faster in seconds than FSC-Quality but lower on VBench; fidelity is not comparable across runtime stacks. SCOPE on its official stack next to the shared-stack rows of Table 1 (SkyReels-V2 1.3B, same prompts, seed, generation settings and GPU model). Speedup is normalized to each stack’s own vanilla and fidelity is measured against each stack’s own same-seed vanilla (

102

prompts) ; only latency and VBench are comparable across stacks. † official ablation of the SCOPE paper, run on the

235

-prompt stratified subset and the fidelity prompts (

316

videos); its VBench is the paired value on the subset’s

126

evaluated prompts, where the released configuration scores

0.7769

and FSC-Quality

0.7883

. Best in bold , second best underlined , only in the columns comparable across stacks (latency, VBench).

Tercile	$n$	motion	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	MC	FSC
			MotionCache			FSC			$\Delta$ DD
low	34	0.60	18.05	0.6388	0.2864	19.13	0.6563	0.2820	$+0.01$	$-0.06$
mid	32	2.26	15.51	0.6102	0.3722	16.61	0.6448	0.3475	$0.00$	$-0.02$
high	33	6.70	14.41	0.5701	0.4402	15.80	0.6192	0.3893	$-0.01$	$0.00$

Appendix

Table 14: FSC-Quality’s fidelity margin over MotionCache grows with motion. SkyReels-V2 (FSC-Quality vs. MotionCache, MC), seed

7

, the

99

retained prompts of App. E.7 split using vanilla-motion tercile boundaries defined on the original

100

-prompt pool (motion: mean top-

5\%

RAFT flow magnitude relative to VBench’s motion threshold). Fidelity against the same-seed vanilla (offline metrics of Table 10 , not on the scale of Table 1 );

\Delta

DD = mean change of the VBench dynamic-degree score (fraction of clips labelled dynamic) vs. vanilla. The better of the two methods in each fidelity metric is in bold .

Method	Latency (s) $\downarrow$	Speedup $\uparrow$	Rel. FLOPs $\downarrow$	Mem (GB) $\downarrow$	VBench $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla	696	1.000 $\times$	1.000	95.1	0.8313	–	–	–
FlowCache	437	1.595 $\times$	0.581	96.7	0.8292	16.07	0.7692	0.3189
MotionCache	389	1.792 $\times$	0.504	96.8	0.8327	15.88	0.7600	0.3289
FSC-Quality (unchanged from 1.3B)	328	2.121 $\times$	0.392	96.8	0.8406	16.18	0.7745	0.3302

Appendix

Table 15: Unchanged from 1.3B, FSC-Quality is the fastest method on SkyReels-V2 14B, with the highest PSNR and SSIM. SkyReels-V2 14B, no retuning (

47

prompts,

352\times 640

, seed

1024

; fidelity against the same-seed 14B vanilla; VBench on the

25

prompts carrying the evaluated dimensions; memory in

10^{9}

bytes on a

102.6

GB card). Best accelerated result in bold , second best underlined .

Variant	Anchors	s/video $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FSC-Quality wins (of 8)
					SSIM	LPIPS
FSC-Quality: repair controller, D2S anchors	10 D2S	161	0.6641	0.2925	–	–
uniform anchors only (no repair)	13 uniform	153	0.5766	0.3496	8	8
collapsed repair thresholds	10 D2S	165	0.6596	0.3024	5	4
no phase shaping	10 D2S	166	0.6658	0.2916	2	2

Appendix

Table 16: Further variants of the separation study: uniform anchors without repair are outperformed on all eight prompts, while removing phase shaping changes SSIM by at most

0.002

. SkyReels-V2, our development set (eight prompts), seed

1024

, median wall-clock per video on one idle GPU; rows modify FSC-Quality (D2S: dense-to-sparse); the purely reactive rows and the other factorial cells are in Table 3 . “Collapsed repair thresholds” collapses the

C1

–

C3

risk thresholds to a single level; safeguard-triggered repairs retain their native repair budget (this is not a fixed-budget controller). The last two columns count the prompts on which FSC-Quality outperforms the variant. Best in bold , second best underlined .

Figure 10: The influence of a cached step decays along the trajectory, but a local-error surrogate does not select the end-to-end best schedule. (a) Influence

g(s)

of a single cached step on the final video (final LPIPS damage per unit local reuse error; mean and range over four profiling prompts,

72

generations): influence decays about

4

–

6\times

along the trajectory (endpoint ratio

6.1\times

). (b) Local-error surrogate cost of

500

random ten-anchor schedules (histogram) and of the hand-crafted, uniform, reversed, and unweighted DP-optimal schedules (lines), each annotated with its end-to-end SSIM on our development set (Table 17 ); the random schedules are scored under the surrogate only.

Schedule	Gaps	$J\downarrow$	s/video $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FSC-Quality wins (of 8)
						SSIM	LPIPS
FSC-Quality: dense-to-sparse (Eq. 1 )	1,1,2,3,4,4,6,5,7	1.353	161	0.6641	0.2925	–	–
DP, $g$ -weighted ( $1{-}$ SSIM)	2,3,3,3,4,4,4,5,5	1.126	157	0.6558	0.3019	5	5
DP, unweighted	3,3,3,4,4,4,4,4,4	1.077	157	0.6510	0.3104	5	5
DP, $g$ -weighted (LPIPS)	2,2,3,3,4,4,5,5,5	1.146	157	0.6442	0.3197	6	6
uniform	4,3,4,4,3,4,4,3,4	1.108	157	0.6441	0.3189	5	5

Appendix

Table 17: Every alternative anchor schedule has lower mean SSIM than FSC-Quality’s dense-to-sparse schedule. Anchor schedules with the rest of FSC-Quality held fixed (SkyReels-V2, our development set of eight prompts, seed

1024

, ten anchors each; median s/video).

J

= local-error surrogate cost (lower is better under the surrogate); DP = surrogate optimum, unweighted or weighted by the measured influence

g(s)

. The last two columns count the prompts on which the hand-crafted schedule wins. The FSC-Quality, uniform, and reversed rows are the runs of Table 3 ; the DP schedules were generated separately with identical code, seed, and settings, and their win counts are paired against that FSC-Quality run. Best in bold , second best underlined .

Point	Role	s/video $\downarrow$	$\Delta$ lat. %	PSNR $\uparrow$	SSIM $\uparrow$	$\Delta$ SSIM	LPIPS $\downarrow$	$\Delta$ LPIPS	C0/C1/C2/C3/Full %	SSIM W/L	LPIPS W/L
FSC-Quality (run 1)	–	161	$+0.0$	18.92	0.6641	$+0.0002$	0.2926	$-0.0002$	70/12/3/9/6	7/1	5/3
FSC-Quality (run 2)	–	161	$+0.0$	18.90	0.6638	$-0.0002$	0.2931	$+0.0002$	70/12/3/9/6	1/7	3/5
$r$ $=1.4$	local	164	$+1.7$	18.91	0.6668	$+0.0028$	0.2905	$-0.0024$	68/10/4/12/6	7/1	5/3
$r$ $=1.9$	local	156	$-3.2$	18.76	0.6485	$-0.0155$	0.3172	$+0.0243$	71/12/4/11/3	1/7	1/7
$q_{\mathrm{scale}}$ $=1$	local	160	$-0.5$	18.83	0.6617	$-0.0023$	0.2985	$+0.0056$	70/12/3/9/6	2/6	2/6
$q_{\mathrm{scale}}$ $=1.5$	local	163	$+1.1$	18.89	0.6656	$+0.0017$	0.2887	$-0.0042$	70/12/3/9/6	7/1	6/2

Appendix

Table 18: Six of the eight local perturbations of FSC-Quality change SSIM by at most

0.0028

and latency by at most

1.8\%

. Local sensitivity around FSC-Quality (SkyReels-V2, our development set of eight prompts):

\Delta

relative to the mean of the two FSC-Quality runs, routing shares over non-anchor decisions, and W/L = prompts on which the point outperforms/underperforms FSC-Quality; local:

r\pm 15\%

q_{\mathrm{scale}}\pm 20\%

, relative weights

\pm 50\%

; stress: a wider endpoint on each axis; routing levels:

C0

whole-step replay,

C1

–

C3

token repair of increasing fraction, Full full repair (Sec. 4.3 ). s/video: median wall-clock. Best perturbed point in bold , second best underlined .

Figure 11: Perturbations toward more repair are marginally better at slightly higher latency, and the larger deviations toward less repair trade SSIM for lower latency. SSIM against median latency per video around FSC-Quality (SkyReels-V2, our development set; stars: the two FSC-Quality runs; circles: local perturbations; triangles: stress perturbations; color: perturbed parameter; values in Table 18 ).

Explore similar work

Motion-Aware Caching for Efficient Autoregressive Video Generation
May 3, 2026Jing Xu, Yuexiao Ma, Xuzhe Zheng +7Video Diffusion Models KV-Cache Management
1Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · 2ByteDance · 3Max Planck Institute for Intelligent Systems, 4ELLIS Institute Tbingen, 5Tbingen AI Center
Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation
Apr 3, 2026Hanshuai Cui, Zhiqing Tang, Zhi Yao +3Autoregressive Video Generation Diffusion Model Inference Acceleration
School of Artificial Intelligence, Beijing Normal University, Beijing, China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China · Shenzhen University of Advanced Technology, Shenzhen, China
In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion
Sep 26, 2026Yikai Wang, Xiao Han, Mengmeng Xu +9Long-Horizon Video Generation Diffusion Model Caching
Meta · S-Lab, Nanyang Technological University

May 3, 2026J/K move · Enter open · S save

Motion-Aware Caching for Efficient Autoregressive Video Generation

Jing Xu, Yuexiao Ma, Xuzhe Zheng, Xing Wang+6

1Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · 2ByteDance · 3Max Planck Institute for Intelligent Systems, 4ELLIS Institute Tbingen, 5Tbingen AI Center

Video Diffusion Models KV-Cache Management Diffusion Model Caching Autoregressive Video Generation Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Autoregressive video generation paradigms offer theoretical promise for long video synthesis, yet their practical deployment is hindered by the computational burden of sequential iterative denoising. While cache reuse strategies can accelerate generation by skipping redundant denoising steps, existing methods rely on coarse-grained chunk-level skipping that fails to capture fine-grained pixel dynamics. This oversight is critical: pixels with high motion require more denoising steps to prevent error accumulation, while static pixels tolerate aggressive skipping. We formalize this insight theoretically by linking cache errors to residual instability, and propose MotionCache, a motion-aware cache framework that exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics. MotionCache employs a coarse-to-fine strategy: an initial warm-up phase establishes semantic coherence, followed by motion-weighted cache reuse that dynamically adjusts update frequencies per token. Extensive experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups of

\textbf{6.28}\times

and

\textbf{1.64}\times

respectively, while effectively preserving generation quality (VBench:

1\%\downarrow

and

0.01\%\downarrow

respectively). The code is available at https://github.com/ywlq/MotionCache.

Abstract

Explore similar work

Motion-Aware Caching for Efficient Autoregressive Video Generation

Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation

In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion