cs.CVNov 9, 2026

What kills vv-prediction? A Patch-wise PCA Perspective on Pixel-Space Flow Matching

Authors: Tianze Luo, Haofeng Huang, Yuan Yao

Organizations: Tsinghua University · Shanghai Qi Zhi Institute

Abstract

Latent Diffusion Models dominate generative modeling but face two-stage training complexity and the inherent reconstruction limits of pre-trained autoencoders. Pixel-space Flow Matching offers a simplified end-to-end alternative yet often encounters training difficulties when using the standard vv-prediction objective. Recent research attributes this difficulty to the off-manifold nature of noised targets, motivating a shift toward xx-prediction. In this paper, we investigate this explanation through patch-wise Principal Component Analysis (PCA) and identify a bandwidth bottleneck. The velocity in minor components is largely determined by the raw noisy input, requiring deep backbones to preserve information that competes with principal-component modeling. Through controlled experiments and representation analysis, we further examine why xx-prediction remains effective: its parameterization provides a raw-input bypass, while its learned input representation can spontaneously concentrate on principal directions. Under low-rank observations, our analysis of the induced flow characterizes xx-prediction as an implicit conditional-mean decoder, reconstructing discarded components from the retained information. These observations motivate the Raw-injected Transformer (RiT), which combines a lightweight raw-conditioned decoder with an output-side shortcut to separate semantic modeling from raw-state propagation. Our method restores effective vv-prediction and achieves competitive performance on ImageNet 256×256256\times256, providing a perspective on the interaction between prediction objectives and architectural bandwidth.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Equivalent Flows, Unequal Learning: Clean-Latent Prediction in Transformers

    May 26, 2026Funing Fu, Tenghui Wang, Guanyu Zhou +2Flow MatchingDiffusion Transformer

  2. CrossFlow: One-Step Generation Across Latent and Pixel Spaces

    Jun 18, 2026Xiyuan Wang, Xiao Zhang, Yang Li +4Flow MatchingLatent Diffusion Models

  3. On the Diffusibility of High-Dimensional Latents

    Sep 23, 2026Chao Feng, Zhiyang Xu, Bowei Chen +7Flow MatchingLatent Diffusion Models