Diffusion Model Fine-Tuning

Follow this topic Ask about this topic

Diffusion model fine-tuning studies how to adapt pretrained image, video, audio, molecular, and language generators without rebuilding them from scratch. Parameter-efficient methods such as LoRA, prompt tuning, and adapters support personalization and domain adaptation, while supervised training and self-generated trajectories target tasks such as image editing, restoration, and masked-language-model refinement. For preference alignment and constrained generation, reinforcement learning methods such as GRPO, reward matching, and distillation optimize outputs using reward signals, often on benchmarks for text-image alignment, generation quality, or downstream utility. A central tension is improving the requested behavior while preserving the model’s general capabilities and diversity, with training cost, limited data, and reward hacking complicating that balance.

Latest papers

181 papers32 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning
Mar 13, 2026Yamin Arefeen, Sidharth Kumar, Steven Warach +2Accelerated MRI Reconstruction Magnetic Resonance Imaging
Chandra Family Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX, USA · Department of Imaging Physics MD Anderson Cancer Center Houston, TX, USA · Dell Medical School Department of Neurology The University of Texas at Austin Austin, TX, USA +3
Continuous Diffusion Transformers for Designing Synthetic Regulatory Elements
Mar 11, 2026Jonathan Liu, Kia GhodsMolecular Generation Diffusion Transformer
Department of Computer Science Princeton University
Visually-Guided Controllable Medical Image Generation via Fine-Grained Semantic Disentanglement
Mar 11, 2026Xin Huang, Junjie Liang, Qingshan Hou +4Medical Image Synthesis Controllable Image Generation
Computer Science and Engineering, Northeastern University, Shenyang, China · Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China · National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China +2
BLK-Assist: A Methodological Framework for Artist-Led Co-Creation with Generative AI Models
Mar 10, 2026Daniel Grimes, Rachel M. HarrisonImage Generation Human-AI Co-Creation
Visual-Synthesizer.com Grass Valley, CA 95945 · Ophiuchus LLC Dover, DE 19904
Diffusion-Based Data Augmentation for Image Recognition: A Systematic Analysis and Evaluation
Mar 9, 2026Zekun Li, Yinghuan Shi, Yang Gao +1Diffusion-Based Data Augmentation Data Augmentation
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China. · School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China.
PureCC: Pure Learning for Text-to-Image Concept Customization
Mar 8, 2026Zhichao Liao, Xiaole Xian, Qingyu Li +7Subject-Driven Image Generation Diffusion Model Fine-Tuning
Tsinghua University · School of Computer Science & Software Engineering, Shenzhen University · Kling Team, Kuaishou Technology +3
Diffusion Controller: Framework, Algorithms and Parameterization
Mar 7, 2026Tong Yang, Moonkyung Ryu, Chih-Wei Hsu +4Diffusion Model Adaptation Diffusion Model Fine-Tuning
Carnegie Mellon University; Intern at Google Research · Google · Yale University
Optimizing 3D Diffusion Models for Medical Imaging via Multi-Scale Reward Learning
Mar 6, 2026Yueying Tian, Xudong Han, Meng Zhou +3Medical Image Synthesis 3D Diffusion Models
University of Sussex · University of Toronto
Tuning Just Enough: Lightweight Backdoor Attacks on Multi-Encoder Diffusion Models
Mar 4, 2026Ziyuan Chen, Yujin Jeong, Tobias Braun +1Backdoor Attacks T2I Diffusion Models
Computer Science Department, TU Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany
CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language
Mar 2, 2026Roy Uziel, Omer Belhasin, Itay Levy +4Diffusion Language Models Continuous Diffusion Language Models
NVIDIA
ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets
Feb 23, 2026Hoyoung Kim, Minwoo Jang, Jabin Koo +2Synthetic Data Augmentation Diffusion Model Fine-Tuning
Graduate School of AI, POSTECH · Dept. of CSE, POSTECH · NAVER AI Lab
DGPO: RL-Steered Graph Diffusion for Neural Architecture Generation
Feb 22, 2026Aleksei Liuliakov, Luca Hermes, Barbara HammerGraph Generation Graph Diffusion
Machine Learning Group, Bielefeld University, Germany
RegionRoute: Regional Style Transfer with Diffusion Model
Feb 22, 2026Bowen Chen, Jake Zuena, Alan C. Bovik +1Image Style Transfer Diffusion-Based Image Editing
The University of Texas at Austin · Dolby Laboratories, Inc.
Efficient Tail-Aware Generative Optimization via Flow Model Fine-Tuning
Feb 18, 2026Zifan Wang, Riccardo De Santi, Xiaoyu Mo +3Molecule Design Fine-Tuning
KTH Royal Institute of Technology · ETH Zurich · ETH AI Center +2
Efficient Text-Guided Convolutional Adapter for the Diffusion Model
Feb 16, 2026Aryan Das, Koushik Biswas, Swalpa Kumar Roy +2Diffusion Model Adaptation Conditional Image Generation
VIT Bhopal · IIIT Delhi · Tezpur University Assam +1
SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning
Feb 13, 2026Jintao Zhang, Kai Jiang, Chendong Xiang +5Sparse Attention Diffusion Model Acceleration
Tsinghua University · UC Berkeley
Selective Fine-Tuning for Targeted and Robust Concept Unlearning
Feb 8, 2026Mansi, Avinash Kori, Francesca Toni +1Fine-Tuning Adversarial Robustness
Department of Computing Imperial College London, United Kingdom
Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
Feb 4, 2026Jaemoo Choi, Yuchen Zhu, Wei Guo +6T2I Generation Diffusion Model Fine-Tuning
Georgia Institute of Technology · National University of Singapore · Nanjing university.
Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models
Jan 21, 2026Injin Kong, Hyoungjoon Lee, Yohan JoMasked Diffusion Models Autoregressive Diffusion
Graduate School of Data Science, Seoul National University · Department of Biosystems & Biomaterials Science and Engineering, Seoul National University
Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation
Jan 18, 2026Jinmei Liu, Haoru Li, Zhenhong Sun +6RL for Image Generation Behavioral Diversity in RL
Nanjing University · Australian National University · Wuhan University +2
Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
Dec 29, 2025Dohyun Kim, Seungwoo Lyu, Seung Wook Kim +1Diffusion Model Alignment Diffusion Model Fine-Tuning
Dept. of CSE, Korea University · 2NVIDIA
MixFlow Training: Alleviating Exposure Bias with Slowed Interpolation Mixture
Dec 22, 2025Hui Li, Fu-Yun Wang, Haoyuan Xia +4Diffusion Model Fine-Tuning Diffusion-Based Image Generation
Shanghai Innovation Institute · Fudan University · Shanghai Academy of AI for Science +2
MatLat: Material Latent Space for PBR Texture Generation
Dec 19, 2025Kyeongmin Yeo, Yunhong Min, Jaihoon Kim +1Texture Synthesis Latent Diffusion Models
KAIST
Anchor to Expand: Semantic Anchoring for Personalized Text-to-Image Diffusion Models
Nov 27, 2025Seoyun Yang, Gihoon Kim, Taesup KimSubject-Driven Image Generation T2I Generation
Graduate School of Data Science, Seoul National University
Rethinking Garment Conditioning in Diffusion-based Virtual Try-On: Decouple, Don't Denoise
Nov 24, 2025Kihyun Na, Jinyoung Choi, Injung KimVirtual Try-On Diffusion Model Fine-Tuning
Advanced AI Talent Education and Research Group for Industrial Innovation, Handong Global University, Pohang 37554, Republic of Korea · Department of Artificial Intelligence, Computer and Electrical Engineering, Handong Global University, Pohang 37554, Republic of Korea
FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning
Nov 22, 2025Bo Yin, Xiaobin Hu, Xingyu Zhou +7Fine-Tuning Adaptive Model Routing
National University of Singapore · University of Electronic Science and Technology of China · Nanyang Technological University +4
DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space
Sep 29, 2025Wenkun He, Yuchao Gu, Junyu Chen +11Diffusion Model Fine-Tuning Diffusion Model Inference Acceleration
Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models
Jun 12, 2025Konstantinos Vilouras, Ilias Stogiannidis, Junyu Yan +2Diffusion Model Alignment Prompt Tuning
School of Engineering, University of Edinburgh · Canon Medical Research Europe Ltd.
From Uncertain to Safe: Conformal Adaptation of Diffusion Models for Safe PDE Control
Feb 4, 2025Peiyan Hu, Xiaowei Qian, Wenhao Deng +8Diffusion Model Guidance Safety-Critical Control
Work done as an intern at Westlake University · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · School of Engineering, Westlake University, Hangzhou, China +2
A LoRA is Worth a Thousand Pictures
Dec 16, 2024Chenxi Liu, Towaki Takikawa, Alec JacobsonWeight-Space Analysis Low-Rank Adaptation
University of Toronto · Outerport · Adobe Research

Mar 13, 2026J/K move · Enter open · S save

Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning

Yamin Arefeen, Sidharth Kumar, Steven Warach, Hamidreza Saber+1

Chandra Family Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX, USA · Department of Imaging Physics MD Anderson Cancer Center Houston, TX, USA · Dell Medical School Department of Neurology The University of Texas at Austin Austin, TX, USA+3

Accelerated MRI Reconstruction Magnetic Resonance Imaging Diffusion Model Fine-Tuning Diffusion Models Neuroimaging

Open paper AskPDF arXiv

Purpose: To develop a data-efficient strategy for accelerated MRI reconstruction with Diffusion Probabilistic Generative Models (DPMs) that enables faster scan times in clinical stroke MRI when only limited fully-sampled data samples are available. Methods: Our simple training strategy, inspired by the foundation model paradigm, first trains a DPM on a large, diverse collection of publicly available brain MRI data in fastMRI and then fine-tunes on a small dataset from the target application using carefully selected learning rates and fine-tuning durations. The approach is evaluated on controlled fastMRI experiments and on clinical stroke MRI data with a blinded clinical reader study. Results: DPMs pre-trained on approximately 4000 subjects with non-FLAIR contrasts and fine-tuned on FLAIR data from only 20 target subjects achieve reconstruction performance comparable to models trained with substantially more target-domain FLAIR data across multiple acceleration factors. Experiments reveal that moderate fine-tuning with a reduced learning rate yields improved performance, while insufficient or excessive fine-tuning degrades reconstruction quality. When applied to clinical stroke MRI, a blinded reader study involving two neuroradiologists indicates that images reconstructed using the proposed approach from

2 \times

accelerated data are non-inferior to standard-of-care in terms of image quality and structural delineation. Conclusion: Large-scale pre-training combined with targeted fine-tuning enables DPM-based MRI reconstruction in data-constrained, accelerated clinical stroke MRI. The proposed approach substantially reduces the need for large application-specific datasets while maintaining clinically acceptable image quality, supporting the use of foundation-inspired diffusion models for accelerated MRI in targeted applications.