TMPDiff: Temporal Mixed-Precision for Diffusion Models

Basile Lewandowski; Simon Kurz; Aditya Shankar; Robert Birke; Jian-Jia Chen; Lydia Y. Chen

cs.CVMar 14, 2026

TMPDiff: Temporal Mixed-Precision for Diffusion Models

Authors: Basile Lewandowski, Simon Kurz, Aditya Shankar, Robert Birke, Jian-Jia Chen, Lydia Y. Chen

Organizations: University of Neuchâtel, Switzerland · TU Dortmund University, Germany · LAMARR Institute, Germany · TU Delft, Netherlands ·

Abstract

Diffusion models are the go-to method for Text-to-Image generation, but their iterative denoising processes has high inference latency. Quantization reduces compute time by using lower bitwidths, but applies a fixed precision across all denoising timesteps, leaving an entire optimization axis unexplored. We propose TMPDiff, a temporal mixed-precision framework for diffusion models that assigns different numeric precision to different denoising timesteps. We hypothesize that quantization errors accumulate additively across timesteps, which we then validate experimentally. Based on our observations, we develop an adaptive bisectioning-based algorithm, which assigns per-step precisions with linear evaluation complexity, reducing an otherwise exponential search problem. Across four state-of-the-art diffusion models and three datasets, TMPDiff consistently outperforms uniform-precision baselines at matched speedup, achieving 10 to 20% improvement in perceptual quality. On FLUX.1-dev, TMPDiff achieves 90% SSIM relative to the full-precision model at a speedup of 2.5x over 16-bit inference.

Figures & tables

Panel of Figure 1 — Figure 1: Change in final latent error $E(Z)$ from single-timestep up- and downcasting.

Panel of Figure 2 — Figure 2: Linear fit between the measured final latent error $E(Z)$ and the schedule scoring functions $S^{\uparrow}(Z)$ and $S^{\downarrow}(Z)$ for upcasting $K$ timesteps for Sana.

Figure 3: Rank deviation per

K

over the number of sampled timesteps for calibration on the COCO dataset for the Sana model.

Figure 4: Performance metrics over different speedup factors on FLUX W4A4R32.

Model	Method	MJHQ			COCO			sDCI			Speedup
Model	Method	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	Speedup
Pixart	FP16 (8 steps)	0.58	15.3	0.41	0.61	15.4	0.40	0.53	15.1	0.41	2.5
	W4A4 (25 steps)	0.63	16.8	0.36	0.61	15.6	0.40	0.54	15.4	0.40	–
	Ours (15+5 steps)	0.69	19.4	0.27	0.69	18.4	0.30	0.61	17.8	0.31	–
Sana	BF16 (8 steps)	0.54	14.6	0.38	0.53	13.5	0.41	0.45	12.0	0.44	2.5
	W4A4 (25 steps)	0.63	17.0	0.26	0.60	15.6	0.31	0.56	15.3	0.30	2.6

Table 1: Comparison across different models and datasets. The reference point for similarity evaluation are the images generated by the unquantized (16 bits) model. To compare models with similar speedups, we scale the number of timesteps depending on the data type.

Method	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$
Small Sana	0.47	13.0	0.48
DDSM heuristic	0.80	22.7	0.09
Ours	0.82	23.3	0.08

Table 2: Result of mixing models

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Linear fits between MJHQ ground-truth final latent errors $\delta_{0}$ (128 test samples; 20 schedules per $K\!\in\!\{2,6,10,14,18\}$ ) and the scoring function $S^{\uparrow}(Z)$ . Upcasting gains $\Delta^{\uparrow}_{t}$ are computed on MJHQ with $m\in\{64,128,256,512\}$ calibration samples.

Panel of Figure 9 — Figure 9: Linear fits between DCI ground-truth final latent errors $\delta_{0}$ (128 test samples; 20 schedules per $K\!\in\!\{2,6,10,14,18\}$ ) and the scoring function $S^{\uparrow}(Z)$ . Upcasting gains $\Delta^{\uparrow}_{t}$ are computed on MJHQ with $m\in\{64,128,256,512\}$ calibration samples.

Panel of Figure 10 — Figure 10: Linear fits between COCO ground-truth final latent errors $\delta_{0}$ (128 test samples; 20 schedules per $K\!\in\!\{2,6,10,14,18\}$ ) and the scoring function $S^{\uparrow}(Z)$ . Upcasting gains $\Delta^{\uparrow}_{t}$ are computed on MJHQ with $m\in\{64,128,256,512\}$ calibration samples.

Model		IR $\uparrow$	CLIPS $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
PixArt	$r_{p}$	-0.69	-0.74	0.73	-0.59	-0.92
PixArt	$r_{s}$	-0.72	-0.78	0.82	-0.67	-0.93
Sana	$r_{p}$	-0.67	-0.42	0.84	-0.67	-0.74
Sana	$r_{s}$	-0.72	-0.41	0.90	-0.74	-0.87

Appendix

Table 3: Pearson (

r_{p}

) and Spearman (

r_{s}

) correlation coefficients between image quality metrics and final latent errors.

Figure 11: Sampling speed (seconds) for FLUX on several memory transfer strategies.

Dataset	Metric	Ours	bfloat16	w4a4r32
COCO	IR ( $\uparrow$ )	1.0434	0.9476	1.0413
	CLIP ( $\uparrow$ )	27.021	26.925	26.907
	FID ( $\downarrow$ )	71.243	71.548	71.839
MJHQ	IR ( $\uparrow$ )	1.0438	0.9239	1.0424
	CLIP ( $\uparrow$ )	27.163	26.971	27.172
	FID ( $\downarrow$ )	55.990	57.196	55.947

Appendix

Table 4: Comparison of Image Reward, CLIP Score, and FID (w.r.t. ground truth images) across datasets. The model evaluated is Sana, using identical settings to the experiment reported in Tab. 1 .

DeepCache	TMPDiff	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	Speedup
✓	✗	0.73	20.5	0.28	2.25
✗	✓	0.85	28.0	0.10	1.42
✓	✓	0.72	20.3	0.29	2.91

Appendix

Table 5: Result of combined DeepCache and TMPDiff sampling

Diffusion Model Quantization

Diffusion model quantization compresses model weights and activations to reduce inference memory and latency while preserving generated image, video, and speech quality.

See all 39 papers

Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models
Oct 22, 2026Ofek Avivi, Eliron Rahimi, Avi Mendelson +1Neural Network Quantization Mixed-Precision Quantization
Technion – Israel Institute of Technology · INSIGHT Lab, Ben-Gurion University of the Negev
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Aug 4, 2026Seokho Han, Dongwei Wang, Jinhee Kim +4Weight-Only Quantization Mixed-Precision Quantization
Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea · Department of Electrical and Computer Engineering, University of Arizona, USA · Department of Electrical and Computer Engineering, Duke University, USA +1
6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models
Mar 19, 2026Rundong Su, Jintao Zhang, Zhihang Yuan +3Diffusion Model Quantization Mixed-Precision Quantization
Fudan University · Tsinghua University · Shanghai Jiao Tong University

Oct 22, 2026J/K move · Enter open · S save

Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models

Ofek Avivi, Eliron Rahimi, Avi Mendelson, Chaim Baskin

Technion – Israel Institute of Technology · INSIGHT Lab, Ben-Gurion University of the Negev

Neural Network Quantization Mixed-Precision Quantization Adaptive Quantization Diffusion Language Models

Open paper AskPDF arXiv

Diffusion Language Models (DLMs) introduce unique quantization challenges due to their iterative denoising process. In this work, we reveal that quantization sensitivity is both taskand time-dependent, with different downstream tasks exhibiting distinct precision requirements across denoising steps. We further show that these patterns can be predicted without groundtruth labels or task-specific evaluation metrics. Building on these findings, we introduce T 2Quant, a task and time-aware mixedprecision quantization framework that uses a label-free sensitivity signal to optimally allocate activation precision across denoising steps under a fixed bit-width budget. Extensive experiments across three DLMs and six benchmarks demonstrate that T 2Quant consistently improves the average accuracy of existing quantization methods, achieving gains of up to 6.7 percentage points under the same budget.

Abstract

Diffusion Model Quantization

Explore similar work

Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models

TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models

6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models