cs.CVNov 18, 2026

ViLFlow: Parallel Vision-Language Generation with Continuous Latent Flows

Authors: Kevin Zhai, Bian Sun, Zhenyi Wang, Song Wang

Organizations: University of Central Florida · Institute of Artificial Intelligence

Abstract

Vision-language generation has predominantly relied on autoregressive and discrete diffusion models. Autoregressive models generate tokens sequentially, while discrete diffusion models typically predict tokens independently within each denoising step. We propose ViLFlow (Vision-Language Flow), a continuous-flow vision-language model that combines joint modeling of output embeddings with parallel token decoding. ViLFlow adapts pretrained continuous language generation to visual question answering and image captioning through learned image guidance and prediction-aware token decoding. A visually adapted teacher provides image-guided predictions as additional denoising supervision. The denoiser learns to reproduce these predictions directly, retaining one image-conditioned prediction per generation step. We train the token decoder on denoiser predictions and paired reference text to improve the conversion of continuous outputs into words. The decoder can be adapted independently, improving answering and captioning without further denoiser training. For captioning, it also supplies text feedback during late denoising, allowing decoded words to inform subsequent refinement. ViLFlow outperforms autoregressive and masked-diffusion baselines by up to 8.46 percentage points on VQAv2 and 4.50 points on GQA without GQA training. On COCO Karpathy test, it improves CIDEr by 16.25 over masked diffusion and 17.81 over sampled autoregression.

Figures & tables

Explore similar work

CardsList
  1. TextLDM: Language Modeling with Continuous Latent Diffusion

    May 8, 2026Jiaxiu Jiang, Jingjing Ren, Wenbo Li +10Diffusion TransformerLatent Diffusion Models