Diffusion Model Adaptation

Latest papers

40 papers10 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models
Nov 3, 2026Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar +2Unified Multimodal Models Multimodal Diffusion Models
University of Illinois Urbana-Champaign
One Image Is Enough: Self-Generated Supervision for Personalizing Human Video Diffusion at Test Time
Nov 2, 2026Sadman Rakib Pinon, Adamo Orsini, Yanis Falaki +4Identity-Preserving Video Generation Video Generation
Concordia University · Mila – Quebec AI Institute
Structure-retained low-rank adapters for weather synthesis
Oct 28, 2026Shunxin Wang, Alexandros Stergiou, Luuk Spreeuwers +2Image Generation Synthetic Data Augmentation
work done while at the University of Twente
Variational Optimal Control for Diffusion Guidance
Oct 27, 2026Ruoyu Wang, Xinyi Ye, Yunshuo Shi +3Diffusion Model Guidance Inference-Time Guidance
AGI Lab, Westlake University · Shandong University · Shanghai Jiao Tong University +1
Domain-Adapted Latent Diffusion Models for CT-to-PET Image Translation
Oct 26, 2026Yann Bourdé, Lea Bogensperger, Arnela Hadzic +1Medical Image Modality Translation Medical Image Synthesis
Department of Quantitative Biomedicine, University of Zurich · Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz
Fast and Useful: Adapting Distilled Generators for Visual Learning
Oct 23, 2026Patryk Bartkowiak, Dominik Michels, Sören Pirk +1Diffusion Model Adaptation Synthetic Data Augmentation
Adam Mickiewicz University · GreenMatterAI · KAUST +1
WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models
Oct 22, 2026Chenxi Song, Yanming Yang, Tong Zhao +3Camera-Controlled Video Generation Trajectory-Guided Video Generation
Westlake University, Hangzhou, China · Beihang University, Beijing, China · Singapore Management University, Singapore
Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion
Oct 22, 2026Jinjian Jiang, Jing Peng, Rehan Deen +4Diffusion Model Adaptation Few-Shot Domain Adaptation
University of Warwick · Columbia University · Independent Researcher +1
To Route or to Adapt? Multi-Expert Diffusion Planners in Unseen Regions
Oct 22, 2026Yun Li, Ehsan Javanmardi, Simon Thompson +7Autonomous Driving Planning Diffusion-Based Planning
The University of Tokyo · TIER IV, Inc. · Huawei
HimmBO: High-Dimensional Interactive Model-Merging Bayesian Optimization for Image Generation
Oct 22, 2026Jinyuan Mao, Zhihao Shen, Chenxi LiuModel Merging Bayesian Optimization
Zhejiang University · University of Victoria
Policy-Mask: Budget-Aware Adaptation of Diffusion Features to Discriminative Tasks
Oct 20, 2026Ivan Listopadov, Maksim Nakhodnov, Aibek AlanovDiffusion Model Adaptation Feature Selection
HSE University · MSU · Constructor University +1
Continual Concept Personalization in Diffusion Models via Residual-Space Adaptation
Oct 20, 2026Yuzhi Liu, Enis Simsar, Giulia Lanzillotta +1Diffusion Model Adaptation T2I Diffusion Models
ETH Zürich
D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
Oct 4, 2026Jieke Wu, Qijie Zhu, Weimin Wu +3Diffusion Model Adaptation Diffusion Model Guidance
Department of Computer Science, King Abdullah University of Science and Technology · Department of Statistics and Data Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University Department of Computer Science, Northwestern University +1
Diffu-LoRA: A Novel Low-Rank Adaptation for Personalized Diffusion Models
Sep 24, 2026Tianjing Li, Wei ZhuNonlinear LoRA Diffusion Model Adaptation
Zhangjiang Lab of Artificial Intelligence, Shanghai, China
dQwen3.5: Hybrid-Attention Diffusion Language Models
Sep 17, 2026Anton Xue, Litu Rout, Aditya Akella +3Diffusion Language Model Decoding Diffusion Language Models
University of Texas at Austin
STAR: Structure-aware Test-time Adaptation for diffusion-based light field Reconstruction
Sep 17, 2026Wontae Choi, Ki Ryum Moon, Jae Young Lee +2Diffusion Model Adaptation Image Reconstruction
Department of Artificial Intelligence (AI), Sungkyunkwan University (SKKU), Suwon, South Korea · Department of Electrical and Computer Engineering, SKKU, Suwon, South Korea
Continuous Adversarial MeanFlow for Few-Step Generative Adaptation from Any Parameterization
Aug 20, 2026Yara Bahram, Zahra Dehghani, Mélodie Desbos +3Diffusion Model Adaptation Flow Matching
LIVIA · ILLS · ÉTS Montreal +2
In-Loop Model Adaptation with Coupled Latent-Noise Guidance for High-Fidelity Subject-Driven Text-to-Image Generation
Aug 10, 2026Yushun Tang, Weiming Chen, Siyi Liu +3Diffusion Model Guidance Diffusion Model Adaptation
Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen 518055, China · School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China · Pengcheng Lab, Shenzhen 518066, China
HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models
Aug 7, 2026Yu Xue, Haoxuan Qu, Zhuoling Li +5Diffusion Model Adaptation Diffusion Transformer
Lancaster University, Lancaster, UK · South China University of Technology, Guangzhou, China · NVIDIA AI Tech Centre, Singapore
UNIFUSION: Adapting Autoregressive Language Models into Discrete Diffusion under a Unified Reverse-Rate Objective
Jul 27, 2026Xiaoyi Jiang, Jingyuan Li, Yixuan Jiang +4Diffusion Language Models Diffusion Models
Tsinghua University · MathonAI · Beijing Institute of Mathematical Sciences and Applications +1
Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning
Jul 23, 2026Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming FuFine-Tuning Diffusion Model Fine-Tuning
University of Science and Technology of China, China
FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration
Jul 7, 2026Jin Jiang, Jia Wang, Panwen Hu +2Diffusion Model Adaptation Video Diffusion Models
College of Information Technology (CIT), United Arab Emirates University (UAEU), Al Ain, Abu Dhabi, United Arab Emirates · State Key Laboratory of Physical Oceanography and the Faculty of Information Science and Engineering, Ocean University of China (OUC), Shandong, 266100, China · Computer Vision Department, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates
LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models
Jul 6, 2026Marian Lupascu, Sebastian Ripa, Mihai Trascau +2Subject-Driven Image Generation Compositional Image Generation
Adobe Research, Romania · International Computer High School of Bucharest, Romania
DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation
Jun 30, 2026Héctor Laria, Yiping Han, Julian D. Santamaria +4Diffusion Model Adaptation Diffusion Model Unlearning
Computer Vision Center, Barcelona, Spain · Universitat Autonoma de Barcelona, Barcelona, Spain · Program of Computer Science, City University of Hong Kong (Dongguan), China +1
UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion
Jun 18, 2026Aryan Das, Koushik Biswas, Moloud Abdar +1Compositional Image Generation Conditional Diffusion Models
VIT Bhopal, India · IIIT Delhi, India · The University of Queensland, Australia +1
A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decoding
Jun 11, 2026Sophia Tang, Yuchen Zhu, Molei Tao +1Diffusion Model Fine-Tuning Diffusion Language Model Decoding
Department of Computer and Information Science, University of Pennsylvania · School of Mathematics, Georgia Institute of Technology · Department of Bioengineering, University of Pennsylvania
Hierarchical Variational Policies for Reward-Guided Diffusion
May 20, 2026Kushagra Pandey, Farrin Marouf Sofian, Jan Niklas Groeneveld +2Few-Step Diffusion Sampling Diffusion Model Guidance
Department of Computer Science University of California Irvine
Generation of Heterogeneous PET Images from Uniform Organ Activity Maps Using a Pretrained Domain-Adapted Diffusion Model
May 18, 2026Suya Li, Kaushik Dutta, Debojyoti Pal +2Diffusion Model Adaptation PET/CT Imaging
Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, USA · Imaging Science Program, McKelvey School of Engineering, Washington University in St Louis, St. Louis, USA · Department of Surgery, Washington University School of Medicine, St. Louis, USA +1
Efficient Adjoint Matching for Fine-tuning Diffusion Models
May 12, 2026Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu +5Diffusion Model Fine-Tuning RL for Diffusion Models
Seoul National University · Georgia Institute of Technology · Sungkyunkwan University
AD-Relight: Training-Free Banner Relighting via Illumination Translation with Diffusion Priors
Apr 27, 2026Rameshwar Mishra, Bishshoy Das, A V Subramanyam +1Diffusion Model Adaptation Image Relighting
Dolby Labs, Bangalore

Nov 3, 2026J/K move · Enter open · S save

From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models

Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar, Halil Kilicoglu+1

University of Illinois Urbana-Champaign

Unified Multimodal Models Multimodal Diffusion Models Diffusion Model Adaptation Diffusion Model Distillation Rectified Flow

Open paper AskPDF arXiv

While discrete tokenization in Unified Multimodal Models (UMMs) establishes strong semantic reasoning, it introduces a strict quantization bottleneck that degrades fine-grained visual details, often resulting in structurally accurate but artifact-heavy images. We introduce FlowShift, a parameter-efficient post-training framework that bypasses this bottleneck by converting pre-trained discrete UMMs into continuous rectified-flow generators. Instead of discarding the backbone's semantic priors, FlowShift replaces discrete visual interfaces with continuous latent projections and applies LoRA to the attention layers. To efficiently boost visual quality without semantic drift, we propose Dual-Objective Velocity Vector Distillation (DOVD), which aligns the model with a high-fidelity continuous teacher and ground-truth continuous trajectories. This approach drastically improves perceptual quality while preserving prompt adherence and reasoning. When applied to leading discrete UMMs, FlowShift significantly improves COCO-30K FID and aesthetic scores, while retaining over 99.5% of multimodal understanding capabilities. Ablations confirm DOVD drives generation quality. Ultimately, FlowShift achieves high-fidelity continuous generation without sacrificing pre-trained semantic grounding.