Diffusion VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

18 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model
Nov 5, 2026Chengcheng Wang, Wei He, Wenhao Li +4Vision-Language-Action Models Vision-Language Models
University of Sydney · TokenRhythm Technologies · City University of Hong Kong
PATTS: Parallel-Aware Test-Time Scaling for Thought-Image Generation in Multimodal Discrete Diffusion LLMs
Nov 3, 2026Yasong Dai, Zeeshan Hayder, David Ahmedt-Aristizabal +1Test-Time Scaling for VLMs Diffusion VLMs
Australian National University · CSIRO Data61 · Amazon
Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs
Oct 27, 2026Donghyeon Cho, Joowon Kim, Seungho Shin +1Efficient VLM Inference Diffusion VLMs
KAIST · AITRICS
Nemotron-Labs-Diffusion-Uni: Elastic Self-Refining Unified Diffusion LLM with Cross-Paradigm On-Policy Distillation
Oct 27, 2026Shufan Li, Hanrong Ye, Greg Heinrich +5Diffusion VLMs Diffusion Language Models
Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models
Oct 23, 2026Dianye Huang, Jiamei Guo, Yue Zhou +5Medical Image Analysis Medical VLMs
Medical Intelligence and Robotic Cognition (MIRoC) Lab, Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR, China · Chair for Computer-Aided and Medical Procedures (CAMP), Tehcnical University of Munich, Munich, Germany · Department of Medical Ultrasonics, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China +2
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2VLM Evaluation Vision-Language Models
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
CT-OPD: Counterfactual Trace On-Policy Distillation for Diffusion Vision-Language Models
Sep 26, 2026Long Qian, Bingke Zhu, Jiaqi Wei +3Diffusion VLMs Vision-Language Models
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · School of Future Technology, University of Chinese Academy of Sciences · School of Engineering, Cardiff University +2
Routing by Reasoning Need: Trajectory-Aware Decoding Control for Diffusion Vision-Language Models
Sep 10, 2026Yixiang Liu, Zhongxing Xu, Zhonghua Wang +1Diffusion Language Model Inference VLM Reasoning
Southern University of Science and Technology · Monash University
LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes
Sep 3, 2026Chuyan Chen, Haoxing Chen, Kun Chen +27Diffusion Transformer T2I Diffusion Models
AGI Research Center, Inclusion AI
DIVA: Exploiting Cross-Step Conditional Propagation for Visual Jailbreaks in Discrete Diffusion Vision-Language Models
Sep 1, 2026Guorui Song, Runqing Tang, Jingye Zhang +11Diffusion VLMs Adversarial Attacks on VLMs
Tsinghua University · Beijing University of Posts and Telecommunications
Reliability Challenges in Diffusion Vision-Language Models
Sep 1, 2026Md. Atabuzzaman, Chris ThomasVLM Evaluation VLM Robustness
Department of Computer Science Virginia Tech
DAVET: Denoising-Aware Visual Evidence Trajectory Allocation for Diffusion Vision-Language Models
Aug 3, 2026Yongkang Zhou, Xiang Xia, Cheng Yan +2Vision-Language Models Diffusion VLMs
University of Science and Technology of China Hefei, China
ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
Jul 20, 2026Keuntae Kim, Beomseok Lee, Hyunwoo Kim +1VLM Reasoning Diffusion Language Model Decoding
Hanyang University · LG Electronics · KT Corporation
Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models
May 14, 2026Sujung Hong, Chanyong Yoon, Seong Jae HwangVision-Language Models Diffusion VLMs
Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
Apr 15, 2026Baoyou Chen, Hanchen Xia, Peng Tu +5Vision-Language Models Diffusion VLMs
Shanghai Academy of AI for Science · Fudan University · Shanghai Innovation Institute +1
Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM
Apr 8, 2026Chengyue Wu, Shiyi Lan, Yonggan Fu +9Vision-Language Models Diffusion VLMs
The University of Hong Kong · NVIDIA · MBZUAI +1
Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
Mar 27, 2026Shrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju +1Diffusion VLMs GUI Grounding
Arizona State University · AWS Agentic AI
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Multimodal Understanding Vision-Language Models
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1

Nov 5, 2026J/K move · Enter open · S save

Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

Chengcheng Wang, Wei He, Wenhao Li, Hongguang Li+3

University of Sydney · TokenRhythm Technologies · City University of Hong Kong

Vision-Language-Action Models Vision-Language Models Diffusion VLMs

Open paper AskPDF arXiv

Most vision-language-action (VLA) policies decouple manipulation from cognition by bolting an external action mechanism onto a vision-language model (VLM)--either quantizing actions into discrete tokens whose errors bear no stable correspondence to physical displacements, or delegating continuous trajectory generation to a separate action expert whose gradients compete with or are blocked from updating the VLM backbone. We argue that manipulation and cognition should instead be embodied in the same model, shifting from a decoupled VLA to an Embodied Vision-Language Model (eVLM). We introduce Concertina, an eVLM that places continuous trajectory representation, blockwise generation, and physical-coordinate supervision directly on the computation and parameter path of a shared diffusion VLM backbone. Each control timestep is encoded as one continuous latent action token in the backbone hidden space, and the shared Transformer reconstructs temporally ordered action blocks via the same corrupt-and-reconstruct operator used for multimodal pretraining. Active action projections account for only 0.21% of parameters in the 4.46B model with zero separate action-generator layers, compared with 16.5%-35.1% dedicated action-expert capacity in representative continuous VLAs. Using our proposed Action Interface Audit, we confirm that cross-time trajectory coupling emerges in the middle and late layers of the shared backbone rather than at the action interfaces. On standard benchmarks, Concertina achieves 98.15% average success on regular LIBERO, 85.9% on the 10,030-task LIBERO-Plus benchmark, and 4.142 average completed chain length on CALVIN ABC-D. Code: https://github.com/lose4578/Concertina Hugging Face: https://huggingface.co/collections/Traly/concertina