Diffusion Model

Diffusion models are generative models that create data by reversing a noise-diffusion process, aiming to generate high-quality samples from complex distributions. Current research focuses on improving efficiency through techniques like stochastic Runge-Kutta methods and dynamic model architectures (e.g., Dynamic Diffusion Transformer), as well as enhancing controllability and safety via methods such as classifier-free guidance and reinforcement learning from human feedback. These advancements are significantly impacting various fields, including medical imaging, robotics, and artistic creation, by enabling novel applications in image generation, inverse problem solving, and multi-modal data synthesis.

Papers

September 15, 2024

GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion
Vitor Guizilini, Pavel Tokmakov, Achal Dave, Rares Ambrus
Diffusion Model Zero Shot Depth Prediction Geometric Prior Happy Image Metric Depth Metric Depth Estimation
E-Commerce Inpainting with Mask Guidance in Controlnet for Reducing Overcompletion
Guandong Li
Diffusion Model E Commerce Mask Guided Mask Refinement Mask Guidance
HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models
Tingwei Meng, Zongren Zou, Jérôme Darbon, George Em Karniadakis
Diffusion Model Inverse Problem Score Based Generative Stochastic Sampling Riccati Equation Reverse Sampler
DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motion
Liao Shen, Tianqi Liu, Huiqiang Sun, Xinyi Ye, Baopu Li, Jianming Zhang, Zhiguo Cao
Diffusion Model Image Pair Semantic Correspondence Prior Correction Continuous Motion Image Interpolation Pre Trained Image Diffusion Model
Bias Begets Bias: The Impact of Biased Embeddings on Diffusion Models
Sahil Kuchlous, Marvin Li, Jeffrey G. Wang
Diffusion Model Global Impact Text to Image Bias Mitigation Multimodal Embeddings Bias Challenge

September 14, 2024

Real-world Adversarial Defense against Patch Attacks based on Diffusion Model
Xingxing Wei, Caixin Kang, Yinpeng Dong, Zhengyi Wang, Shouwei Ruan, Yubo Chen, Hang Su
Diffusion Model Anomaly Detection Adversarial Patch Adversarial Patch Attack Patch Attack
Towards Diverse and Efficient Audio Captioning via Diffusion Models
Manjie Xu, Chenxing Li, Xinyi Tu, Yong Ren, Ruibo Fu, Wei Liang, Dong Yu
Diffusion Model Diverse Set Audio Captioning
Enhancing EEG Signal Generation through a Hybrid Approach Integrating Reinforcement Learning and Diffusion Models
Yang An, Yuhao Tong, Weikai Wang, Steven W. Su
Diffusion Model Brain Computer Interface Hybrid Reinforcement Learning

September 13, 2024

Adaptive Multi-Modal Control of Digital Human Hand Synthesis Using a Region-Aware Cycle Loss
Qifan Fu, Xiaohang Yang, Muhammad Asad, Changjae Oh, Shanxin Yuan, Gregory Slabaugh
Diffusion Model 3D Hand Human Hand Hand Pose Multimodal Control Hand Keypoints
Neural Message Passing Induced by Energy-Constrained Diffusion
Qitian Wu, David Wipf, Junchi Yan
Diffusion Model Diffusion Transformer
Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Sampling
Nebiyou Yismaw, Ulugbek S. Kamilov, M. Salman Asif
Diffusion Model Unified Framework Inverse Problem RSD Difference of Gaussian Image Prior Reverse Diffusion Diffusion Based Method Diffusion Posterior Sampling
DX2CT: Diffusion Model for 3D CT Reconstruction from Bi or Mono-planar 2D X-ray(s)
Yun Su Jeong, Hye Bin Yoo, Il Yong Chun
Diffusion Model Conditional Diffusion Model Computed Tomography Reconstruction Tomographic Reconstruction Multiple Level 2 Dimensional X Ray
Think Twice Before You Act: Improving Inverse Problem Solving With MCMC
Yaxuan Zhu, Zehao Dou, Haoxin Zheng, Yasi Zhang, Ying Nian Wu, Ruiqi Gao
Diffusion Model Inverse Problem Markov Chain Monte Carlo Posterior Inference Posterior Approximation Diffusion Posterior Sampling
Cross-conditioned Diffusion Model for Medical Image to Image Translation
Zhaohu Xing, Sicheng Yang, Sixiang Chen, Tian Ye, Yijun Yang, Jing Qin, Lei Zhu
Diffusion Model Generative Adversarial Network Medical Image Magnetic Resonance Imaging Image Translation Multi Modal Magnetic Resonance
Risks When Sharing LoRA Fine-Tuned Diffusion Model Weights
Dixi Yao
Diffusion Model Generative Model Supervised Autoencoder Low Rank Adaptation Risk Sensitive Fine Tuned Model Variational AutoEncoder
Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling
Vivek Oommen, Aniruddha Bora, Zhen Zhang, George Em Karniadakis
Diffusion Model Neural Operator Surrogate Modeling Turbulent Flow Spectral Representation Turbulence Model

September 12, 2024

Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
Hanyang Zhao, Haoxian Chen, Ji Zhang, David D. Yao, Wenpin Tang
Reinforcement Learning Diffusion Model Generative Model New Framework Policy OpTimization Performance Score Past Action
DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
Zhenyuan Dong, Sai Qian Zhang
Diffusion Model Latent Variable Diffusion Transformer Multiplier Free Quantization

September 11, 2024

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models
Fengzhe Zhang, Jiajun He, Laurence I. Midgley, Javier Antorán, José Miguel Hernández-Lobato
Diffusion Model High Efficiency Denoising Diffusion Probabilistic Model Importance Sampling Consistency Model Boltzmann Distribution Equivariant Filter Unbiased Sampling Boltzmann Generator
Exploring User-level Gradient Inversion with a Diffusion Prior
Zhuohang Li, Andrew Lowy, Jing Liu, Toshiaki Koike-Akino, Bradley Malin, Kieran Parsons, Ye Wang
Diffusion Model Diffusion Prior Gradient Inversion Attack Realistic Face Gradient Inversion Data Reconstruction

Diffusion Model

Papers

GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion

E-Commerce Inpainting with Mask Guidance in Controlnet for Reducing Overcompletion

HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models

DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motion

Bias Begets Bias: The Impact of Biased Embeddings on Diffusion Models

Real-world Adversarial Defense against Patch Attacks based on Diffusion Model

Towards Diverse and Efficient Audio Captioning via Diffusion Models

Enhancing EEG Signal Generation through a Hybrid Approach Integrating Reinforcement Learning and Diffusion Models

Adaptive Multi-Modal Control of Digital Human Hand Synthesis Using a Region-Aware Cycle Loss

Neural Message Passing Induced by Energy-Constrained Diffusion

Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Sampling

DX2CT: Diffusion Model for 3D CT Reconstruction from Bi or Mono-planar 2D X-ray(s)

Think Twice Before You Act: Improving Inverse Problem Solving With MCMC

Cross-conditioned Diffusion Model for Medical Image to Image Translation

Risks When Sharing LoRA Fine-Tuned Diffusion Model Weights

Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling

Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning

DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models

Exploring User-level Gradient Inversion with a Diffusion Prior