Diffusion policies effectively model multimodal action distributions for robotic manipulation, but their iterative denoising requires tens to hundreds of network function evaluations (NFEs) for each control prediction, limiting their applicability to high-frequency closed-loop control and online reinforcement learning (RL). We present a two-stage framework for native one-step generative policies that transfers iterative refinement from inference to training. First, Drift-Based Policy (DBP) uses a fixed-point drifting objective to internalize corrective dynamics into the model parameters, producing multimodal action chunks with a single network evaluation by design. Second, Drift-Based Policy Optimization (DBPO) augments the pretrained backbone with a stochastic interface that provides exact conditional rollout likelihoods for PPO-style on-policy updates while preserving 1-NFE deployment. On the 12-task Diffusion Policy suite, DBP improves the average success rate from 0.79 to 0.83 while reducing inference from 100 NFEs to 1. Across 37 point-cloud manipulation tasks, DBP achieves an average success rate of 88.4%, surpassing the leading 1-NFE baseline OMP at 82.3%. DBPO further improves pretrained one-step policies through stable online fine-tuning on RoboMimic and D4RL. On a physical dual-arm UR5 platform, DBP achieves 123/150 successes (82%) with an average end-to-end latency of 9.5 ms, compared with MP1's 89/150 successes (59%) under the same setup. Code is available at https://github.com/YuxuanGao0822/DBPO.
Figures & tables
Fig. 1 : Generative policy paradigms for robot control. (a) Multi-step diffusion policies rely on iterative denoising at inference. (b) One-step mean-flow policies generate actions in one pass with auxiliary corrections. (c) Drift-Based Policy internalizes attraction-repulsion refinement during training, yielding a native one-step generator. (d) Our method achieves the best average success rate and control frequency on Adroit and MetaWorld against established generative baselines.
Fig. 2 : Two-stage Drift-Based Policy framework. Stage 1 learns a native one-step generator offline via attraction-repulsion refinement during training. Stage 2 fine-tunes a stochastic actor initialized from the pretrained backbone with on-policy PPO and anchor regularization, while deployment remains one-step (1-NFE).
Fig. 3 : Evolution of the internalized drift manifold. The policy action distribution (blue) progressively aligns with expert modes (peach) during training.
Task
Diffusion Policy (100 NFE)
Ours (1 NFE)
Push-T (Image)
0.91
0.89
Push-T (Low-Dim)
0.85
0.87
BlockPush (P1/P2)
0.24
0.43
RoboMimic (Low-Dim)
0.80
0.92
RoboMimic (Image)
0.91
0.87
Kitchen (P1/P2/P3/P4)
1.00
1.00
TABLE I : Comparison between Diffusion Policy and Ours on the Diffusion Policy suite. BlockPush/Kitchen are phase-averaged. Entries report success rate. Best results are in bold .
Adroit
Meta-World
Method
NFE
Hammer
Door
Pen
Easy (21)
Medium (4)
Hard (4)
Very Hard (5)
Average
DP
10
16.0 ± 10.0
34.0 ± 11.0
13.0 ± 2.0
50.7 ± 6.1
11.0 ± 2.5
5.25 ± 2.5
22.0 ± 5.0
35.2 ± 5.3
DP3
10
100.0 ± 0.0
56.0 ± 5.0
46.0 ± 10.0
87.3 ± 2.2
44.5 ± 8.7
32.7 ± 7.7
39.4 ± 9.0
68.7 ± 4.7
Simple DP3
10
98.0 ± 2.0
40.0 ± 17.0
36.0 ± 4.0
86.8 ± 2.3
42.0 ± 6.5
38.7 ± 7.5
35.0 ± 11.6
67.4 ± 5.0
Adaflow
—
45.0 ± 11.0
27.0 ± 6.0
18.0 ± 6.0
49.4 ± 6.8
12.0 ± 5.0
5.75 ± 4.0
24.0 ± 4.8
35.6 ± 6.1
CP
1
45.0 ± 4.0
31.0 ± 10.0
13.0 ± 6.0
69.3 ± 4.2
21.2 ± 6.0
17.5 ± 3.9
30.0 ± 4.9
50.1 ± 4.7
TABLE II : Point-cloud imitation comparison on Adroit and Meta-World under the MP1/OMP protocol. Ours refers to Drift-Based Policy. Entries report success rate (%, mean ± std over 3 seeds). Best results are in bold ; second-best are underlined .
Fig. 4 : Online PPO fine-tuning results on RoboMimic and D4RL with anchor ablation (DBPO vs. DBPO w/o anchor). Solid bars denote offline initialization, and hatched bars denote gains after fine-tuning. DBPO achieves the strongest post-fine-tuning performance, while removing the anchor reduces gains over pretrained baselines.
Fig. 5 : Real-world bimanual deployment on the physical UR5 testbed. Drift-Based Policy executes precision Lift, Can, and synchronized bimanual Transport using raw trilateral camera inputs.
Task
DBP (Ours)
MP1 (one-step baseline)
Lift (single-arm)
47/50 (94%)
38/50 (76%)
Can (single-arm)
43/50 (86%)
32/50 (64%)
Transport (dual-arm)
33/50 (66%)
19/50 (38%)
Overall
123/150 (82%)
89/150 (59%)
TABLE III : Additional real-robot dual-arm deployment results on the physical UR5 setup (50 trials per task).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9Figure 10
Configuration
Mean Score
Std Score
single_T0p02 ( R={0.02} )
0.758
0.031
single_T0p05 ( R={0.05} )
0.864
0.038
single_T0p20 ( R={0.2} )
0.873
0.012
multi_default ( R={0.02,0.05,0.2} )
0.858
0.057
multi_wide ( R={0.01,0.05,0.2,0.5} )
0.849
0.007
multi_dense ( R={0.01,0.02,0.05,0.1,0.2,0.4} )
0.817
0.030
Appendix
TABLE IV : Temperature configurations and performance results. Bold indicates the best-performing configuration.
Fig. 10 : Test performance across temperature configurations (mean ± std over 3 seeds, 50 rollouts each). Single temperature T=0.2 achieves the best performance with the lowest variance.
Mode
Mean Score
Std Score
Chunk mode ( S=Hda , per_timestep_loss=False )
0.890
0.015
Step-wise mode ( S=da , per_timestep_loss=True )
0.845
0.021
Appendix
TABLE V : Loss computation mode comparison. Chunk mode achieves 5.3% higher performance with lower cross-seed variance.
Method
Training Epochs
Inference NFE
Diffusion Policy
4,500
100
Drift-Based Policy (Ours)
100
1
Appendix
TABLE VI : Training and inference efficiency comparison on the PushT low-dimensional task.
Hyperparameter
Value
Tuned?
Tuning Range
Drift-Field Method
Hypothesis count G
4
Yes
{1,2,4,8,16}
Temperature set R
{0.2}
Yes
Single: {0.02,0.05,0.2}
Multi: various combinations
Negative reference count Cn
0
No
–
Positive reference count Cp
1
No
–
Appendix
TABLE VII : DBP hyperparameters. All values are held constant across experiments unless explicitly varied in the corresponding analysis.
Component
Configuration
Generator (Conditional U-Net 1D)
Latent dimension
256
Down-sampling channels
[512, 1024, 2048]
Up-sampling channels
[2048, 1024, 512]
Kernel size
5
Normalization
GroupNorm (8 groups)
Appendix
TABLE VIII : Network architecture hyperparameters. The generator uses a conditional U-Net architecture with 1D convolutions.
Fig. 11 : Drift-Based Policy execution on Adroit and Meta-World tasks (Part 1). Frames show temporally sampled states from successful rollouts under 1-NFE inference, demonstrating smooth manipulation and precise object control.
Fig. 12 : Drift-Based Policy execution on Adroit and Meta-World tasks (Part 2). The temporal sequences demonstrate stable execution and coherent action progression without iterative correction at inference time.
Generative flow and diffusion models provide the continuous, multimodal action distributions needed for high-precision robotic policies. However, their reliance on iterative sampling introduces severe inference latency, degrading control frequency and harming performance in time-sensitive manipulation. To address this problem, we propose the One-Step Flow Policy (OFP), a from-scratch self-distillation framework for high-fidelity, single-step action generation without a pre-trained teacher. OFP unifies a self-consistency loss to enforce coherent transport across time intervals, and a self-guided regularization to sharpen predictions toward high-density expert modes. In addition, a warm-start mechanism leverages temporal action correlations to minimize the generative transport distance. Evaluations across 56 diverse simulated manipulation tasks demonstrate that a one-step OFP achieves state-of-the-art results, outperforming 100-step diffusion and flow policies while accelerating action generation by over 100×. We further integrate OFP into the π0.5 model on RoboTwin 2.0, where one-step OFP surpasses the original 10-step policy. These results establish OFP as a practical, scalable solution for highly accurate and low-latency robot control.
Shaolong Li, Lichao Sun, Yongchao Chen
CSE Department, University of Michigan · Lehigh University · College of AI, Tsinghua University
Diffusion-based visuomotor policies model complex action distributions through iterative denoising, but repeated inference adds latency to robotic control. One-step generators reduce this cost, motivating training objectives that retain useful action structure with few demonstrations. We present Ada3Drift, a point-cloud-conditioned policy that builds on Drifting Models to perform distribution refinement during training and generate action chunks in one forward pass. Our central design is a regression-to-drifting curriculum: paired action regression first emphasizes observation--action correspondence, while a sigmoid schedule progressively increases a batch-level action-distribution regularizer. The drifting term combines attraction to demonstrated actions and repulsion among generated samples using inherited multi-temperature aggregation. A timestep-free generator preserves single-step inference throughout. On Adroit, Meta-World, RoboTwin, and five real-world tasks, Ada3Drift achieves the highest reported average success rates among the evaluated baselines with 1 NFE, compared with 10 NFE for the diffusion baselines. Controlled ablations favor sigmoid scheduling at matched demonstration budgets. We will release our code and pretrained model weights.
Chongyang Xu, Yixian Zou, Tianyu Yang +4
College of Computer Science, Sichuan University, Chengdu, China · School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China
Real-time robot control demands fast action generation. Diffusion and flow matching policies for robot control require multi-step sampling, limiting their deployment in real-time scenarios. Natively reducing the sampling steps to one sacrifices representation quality and task performance, creating a trilemma among speed, fidelity, and performance. We present One-Step Generative Policy Optimization (OGPO), a systematic framework to resolve this trilemma. OGPO first pairs a lightweight architecture with the interval velocity principle for distillation-free one-step inference, while representation spreading prevents representation quality degradation. It then performs on-policy reinforcement learning (RL) fine-tuning on this fast, stable policy to break the imitation learning ceiling. Experiments on RoboMimic and OpenAI Gym benchmarks show that OGPO matches or exceeds multi-step baselines while achieving a 5-20 times inference speedup and over 120Hz control frequency. Physical deployment on a Franka-Emika-Panda robot validates real-world applicability. Project page: https://ogpo-project.github.io/
Guowei Zou, Haitao Wang, Hejun Wu +3
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China