Diffusion Policy

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 450% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

Sample-Efficient Diffusion-based Reinforcement Learning with Critic Guidance
May 28, 2026Shutong Ding, Zejia Zhong, Zhongyi Wang +4Diffusion Guidance Robotic RL
1ShanghaiTech University.
Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control
May 28, 2026Hao Ren, Zetong Bi, Yiming Zeng +5Diffusion Model Guidance Diffusion-Based Planning
Sun Yat-sen University, Guangzhou, China · Insta360 Research, Shenzhen, China
Frequency-Guided Action Diffusion via Sub-Frequency Manifold Traversal
May 27, 2026Junlin WangDiffusion Model Guidance Diffusion Guidance
School of Engineering and Applied Science University of Pennsylvania
Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization
May 25, 2026Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu +5World Model Learning Diffusion Policy
Dynamic Systems Lab, University College London · College of Computing and Data Science, Nanyang Technological University · School of Intelligence Science and Technology, Peking University +3
Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learning
May 24, 2026Ziheng Cheng, Yixiao Huang, Hanlin Zhu +5Diffusion Policy Semi-Supervised Learning
University of California, Berkeley.
X-DiffVLA: X-Embodied Diffusion Action Heads for Vision-Language-Action Models
May 24, 2026Boyu Li, Chaoyi Xu, Haoqi Yuan +5Cross-Embodiment Robot Learning Diffusion Policy
1SKL-MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Beijing Academy of Artificial Intelligence +2
Dynamic Neural Koopman Distillation for Fast Robot Control Using Diffusion Models
May 24, 2026Lei Zheng, Peiqi Yu, Zengqi Peng +2Robotic Control Koopman Operator Learning
Department of Electrical and Computer Engineering, National University of Singapore, Singapore · Department of Electrical and Computer Engineering, Carnegie Mellon University, USA · Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology, Hong Kong SAR, China
Semantically Structured Mixture-of-Experts for Compositional Robotic Manipulation
May 22, 2026Chengyu Deng, Guanqi Chen, Yizhou Chen +4Robot Policy Learning Robot Skill Learning
The University of Hong Kong · Southern University of Science and Technology
Factored Diffusion Policies:Compositionally Generalized Robot Control with a Single Score Network
May 21, 2026Sayan Mitra, Ege Yuceel, Noah Giles +1Robotic Control Diffusion Policy
University of Illinois Urbana-Champaign
SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control
May 21, 2026Jingyan Zhang, Han Liang, Ruichi Zhang +6RL for Robotics Humanoid Robot Control
ShanghaiTech University, China · Bytedance Seed, China · University of Pennsylvania, USA +2
HITL-D: Human In The Loop Diffusion Assisted Shared Control
May 20, 2026Riley Zilka, Sergey Khlynovskiy, Allie Wang +1Robot Teleoperation Diffusion Policy
Department of Computing Science, University of Alberta, Edmonton Alberta, Canada, T6G 2E8
Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation
May 20, 2026Haoran Huang, Haonan Dong, Huixu DongLong-Horizon Robotic Manipulation Mobile Manipulation
Zhejiang University, Hangzhou 310027, China
SADP: Subgoal-Aware Diffusion Policy for Explainable Robots Learned from Foundation Model Generated Demonstrations
May 16, 2026Site Hu, Takato HoriiRobot Policy Learning Long-Horizon Robotic Manipulation
Department of Systems Innovation, Graduate School of Engineering Science, Osaka University, Osaka, Japan
Ada-Diffuser: Latent-Aware Adaptive Diffusion for Decision-Making
May 15, 2026Fan Feng, Selena Ge, Minghao Fu +6Latent Action Learning Latent Dynamics Modeling
University of California San Diego · MBZUAI · Carnegie Mellon University +2
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
May 15, 2026Xiaoxuan He, Siming Fu, Zeyue Xue +9Diffusion Model Alignment Video Diffusion Models
Zhejiang University · Joy Future Academy · Independent Researcher +1
Feedback World Model Enables Precise Guidance of Diffusion Policy
May 15, 2026Tuo An, Jindou Jia, Gen Li +8World Model Learning World Models for Robotics
1MARS Lab, Nanyang Technological University
DSSP: Diffusion State Space Policy with Full-History Encoding
May 14, 2026Zhiyuan Guan, Jianshu Hu, Han Fang +5Long-Horizon Robotic Manipulation Diffusion Policy
Shanghai Jiao Tong University
Before the Body Moves: Learning Anticipatory Joint Intent for Language-Conditioned Humanoid Control
May 14, 2026Haozhe Jia, Honglei Jin, Yuan Zhang +9Robotic Control Latent Action Learning
The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. +3
WarmPrior: Straightening Flow-Matching Policies with Temporal Priors
May 13, 2026Sinjae Kang, Chanyoung Kim, Kaixin Wang +2Flow Matching Visuomotor Policy Learning
KAIST · Microsoft Research
Test-time Sparsity for Extreme Fast Action Diffusion
May 13, 2026Kangye Ji, Yuan Meng, Jianbo Zhou +3Diffusion Transformer Inference-Time Optimization
Tsinghua University · The Chinese University of Hong Kong
SI-Diff: A Framework for Learning Search and High-Precision Insertion with a Force-Domain Diffusion Policy
May 12, 2026Yibo Liu, Stanko Oparnica, Simon Shewchun-Jakaitis +5Contact-Rich Robotic Manipulation Diffusion Policy
Epson Canada, Markham, Ontario L3R 6G3, Canada · Queen’s University, Kingston, Ontario K7L 3N6, Canada
TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
May 12, 2026Matthew M. Hong, Jesse Zhang, Anusha Nagabandi +1Robot Policy Adaptation Robot Skill Learning
University of Washington · Amazon FAR
Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations
May 12, 2026Lasse Peters, Laura Ferranti, Andrea Bajcsy +1Multi-Agent Coordination Diffusion Policy
University of California, Berkeley · Delft University of Technology · Carnegie Mellon University
HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions
May 11, 2026Zhenhao Shen, Zeming Yang, Yue Chen +5Robotic Grasping Robot Foundation Models
1Peking University · 2Tianjin University
ElasticFlow: One-Step Physics-Consistent Policy with Elastic Time Horizons for Language-Guided Manipulation
May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Flow Matching Long-Horizon Robotic Manipulation
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences · Chongqing School, University of Chinese Academy of Sciences · Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
Geometry Guided Self-Consistency for Physical AI
May 9, 2026Yinwei Dai, Zhuofu Chen, Lijie Yang +1Diffusion Policy Vision-Language-Action Models
Princeton University
Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models
May 8, 2026Meng Zheng, Samhita Marri, Anwesa Choudhuri +6Diffusion Policy Robotic Manipulation
United Imaging Intelligence, Boston, MA, USA · University of Illinois Urbana-Champaign, Urbana, IL, USA
Decentralized Diffusion Policy Learning for Enhanced Exploration in Cooperative Multi-agent Reinforcement Learning
May 8, 2026Yuyang Zhang, Haldun Balim, Na LiDiffusion Policy Decentralized MARL
SEAS, Harvard University
ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving
May 6, 2026Huimin Wang, Yue Wang, Bihao Cui +7Diffusion Policy Autonomous Driving Planning
LiAuto
Hyper-DP3: Frequency-Aware Right-Sizing of 3D Diffusion Policies for Visuomotor Control
May 2, 2026Jinhao Zhang, Zhexuan Zhou, Huizhe Li +5Visuomotor Policy Learning Diffusion Policy
Harbin Institute of Technology, Shenzhen · Shanghai Jiao Tong University

May 28, 2026, cs.ROJ/K move · Enter open · S save

Sample-Efficient Diffusion-based Reinforcement Learning with Critic Guidance

Shutong Ding, Zejia Zhong, Zhongyi Wang, Ke Hu+3

1ShanghaiTech University.

Diffusion Guidance Robotic RL Diffusion Policy RL for Diffusion Models Sample-Efficient RL

Open paper AskPDF arXiv

Recent advances in reinforcement learning (RL) have achieved great successes by leveraging the multimodality and exploration capability of diffusion policies. Among these approaches, one representative branch focuses on the sampling-based policy optimization. This design enables better exploration capability of the diffusion model, particularly at the beginning of training, but suffer from low exploitation in Q-value information, resulting in a slow policy convergence. Another branch pays attention to gradient-based policy optimization, which sufficiently exploits the gradient of the Q function yet tends to collapse into a unimodal policy with low diversity. To address this issue, we propose CGPO, \textbf{C}ritic-\textbf{G}uided diffusion \textbf{P}olicy \textbf{O}ptimization, which effectively balances exploration and exploitation with the training-free guidance technique integrated into the denoising process of diffusion policy. Concretely, CGPO steers action generation toward high-value regions defined by the critic network and uses the guided actions as regression objectives. In this manner, CGPO reduces the time required to obtain high-quality actions and improves final performance with better balance between the exploration-exploitation tradeoff. We validate the effectiveness of CGPO on 5 MuJoCo locomotion tasks, and CGPO achieves state-of-the-art performance compared with existing diffusion-based RL methods. Notably, CGPO is the first success to incorporate diffusion policy into real-world RL, with its superior performance on Franka robot arm grasping tasks. Our official page is released at https://dingsht.tech/cgpo-webpage.

Momentum

Related topics

Latest papers 134