cs.LGJan 29, 2026

Understanding and Learning Prediction Targets in Diffusion Models

Authors: Qing Jin, Chaoyang Wang

Organizations: Independent Researcher.

Abstract

Recent advances in diffusion and flow matching models have highlighted a shift in the preferred prediction target -- moving from noise (ε\varepsilon) and velocity (v) to direct data (x) prediction -- particularly in high-dimensional settings. However, a formal explanation of why the optimal target depends on the specific properties of the data remains elusive. In this work, we provide a theoretical framework based on a generalized prediction formulation u=kx−(1−k)nu=kx-(1-k)n, where xx represents clean data and nn denotes noise. This formulation accommodates arbitrary targets, of which ε\varepsilon-, v-, and x-prediction are special cases. We derive the analytical relationship between data's geometry and the optimal prediction target, offering a rigorous justification for why x-prediction becomes superior when the ambient dimension significantly exceeds the data's intrinsic dimension. Furthermore, our controlled analysis reveals how time conditioning and loss variations systematically modulate this optimal objective. While our theory identifies dimensionality as the governing factor for the optimal prediction target, the intrinsic dimension of manifold-bound data is typically intractable to estimate in practice. To bridge this gap, we propose k-Diff, a framework that employs a data-driven approach to learn the optimal prediction parameter k directly from data, bypassing the need for explicit dimension estimation. Extensive experiments in both latent-space and pixel-space image generation demonstrate that k-Diff consistently outperforms fixed-target baselines across varying architectures and data scales, providing a principled and automated approach to enhancing generative performance.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Correlation Analysis of Generative Models

    Feb 20, 2026Zhengguo Li, Chaobing Zheng, Wei WangGenerative ModelingDiffusion Models

  2. Learning Anisotropic Prediction Targets for Flow Models

    Nov 13, 2026Rayhan Zirvi, Hansheng Chen, Gordon Wetzstein +1Flow-Based Generative ModelingFlow Matching