Target-Consistent Action Guidance via Semi-Unbalanced Optimal Transport for Drifting World Models
Organizations: Bangladesh University of Engineering and Technology · KAIST
Abstract
Action-conditioned world models let robots imagine the outcomes of their actions for planning and offline policy evaluation, but these uses need many rollouts, which multi-step diffusion makes expensive. Drifting models move the iteration from inference to training and generate samples in one network pass. DriftWorld brought this paradigm to action-conditioned world models; to strengthen action following, it repels predictions from the current frame, a construction borrowed from classifier-free guidance. We show that this accentuation has a target bias: the push persists even when a prediction matches the recorded future, in kernel, balanced, and unbalanced Wasserstein drifting fields alike. We introduce Mass-Share Accentuation (MSA) to remove this bias. MSA treats the current frame as a counterfactual destination competing with the recorded future in a semi-relaxed optimal transport problem; subtracting the closed-form share of the correct future makes the guidance vanish at the target, and the remainder is shared by all predictions. We prove that the recorded future is the unique fixed point along the motion direction for collapsed predictions, and that the guidance fades with prediction error. On Push-T, Robomimic, and BridgeV2, where trained models under-react rather than overshoot, MSA improves action ranking, response slope, and moving-region error over latent-space baselines at no inference cost, though these benchmarks do not isolate bias removal as the cause.
Figures & tables
| Action | Motion | Standard | ||||||||||||
| 1 px / 2 mm | 3 px / 5 mm | |||||||||||||
| Dataset | Method | Slope | Corr. | Noise | Mov. | Drift | Static | Late | Late | MSE | SSIM | PSNR | LPIPS | |
| Kernel | 0.682 | 0.699 | 0.826 | 3.43 | 0.1806 | 0.0218 | 0.0192 | 0.42 | 0.45 | 0.0247 | 0.951 | 24.33 | 0.0607 | |
| Kernel + acc. | 0.683 | 0.698 | 0.843 | 3.33 | 0.1846 | 0.0221 | 0.0195 | 0.39 | 0.42 | 0.0248 | 0.951 | 24.31 | 0.0602 | |
| Push-T | W-Flow | 0.693 | 0.713 | 0.830 | 1.49 | 0.1720 | 0.0208 | 0.0183 | 0.41 | 0.46 | 0.0244 | 0.952 | 24.48 | 0.0608 |
| W-Flow + acc. | 0.678 | 0.698 | 0.852 | 1.32 | 0.1761 | 0.0220 | 0.0194 | 0.42 | 0.42 | 0.0270 | 0.948 | 23.93 | 0.0663 | |
| Policy 1 | Policy 2 | |||
|---|---|---|---|---|
| Method | IoU | s/plan | IoU | s/plan |
| Base policy | – | – | ||
| Kernel | 1.92 | 1.96 | ||
| Kernel + acc. | 1.84 | 1.94 | ||
| W-Flow | 1.88 | 1.87 | ||
| W-Flow + acc. | 1.79 | 1.78 | ||
| at | Endpoint err. | ||||
|---|---|---|---|---|---|
| Method | 1.75 | 2.5 | 3.25 | 5 | ( ) |
| W-Flow | 1.001 | 1.001 | 1.001 | 1.001 | .003 |
| Kernel + acc. | 1.019 | 1.031 | 1.049 | 1.100 | .021 |
| UW-Flow + acc. | 1.001 | 1.001 | 1.001 | 1.001 | .004 |
| W-Flow + acc. | 1.754 | 2.504 | 3.253 | 5.004 | 1.050 |
| MSA w/o debiasing | 1.069 | 1.106 | 1.135 | 1.187 | .072 |
| Controllability | Motion probe | Visual and temporal quality | |||||||||
| Method | PSNR | PSNR | Prog. | Prog. | Off-mot. | Spread | SSIM | PSNR | LPIPS | FID | FVD |
| Copy-last-frame | 0 | 19.99 | 0 | 0 | – | – | .832 | 20.07 | .111 | 4.17 | 251.7 |
| Unrelated real video | – | – | – | – | – | – | .211 | 10.07 | .656 | 23.60 | 115.1 |
| Kernel | 2.108 | 23.03 | 0.427 | 0.464 | 0.339 | 0.025 | .818 | 23.28 | .159 | 55.49 | 326.1 |
| Kernel + acc. | 2.111 | 23.00 | 0.436 | 0.450 | 0.347 | 0.025 | .831 | 23.23 | .148 | 55.44 | 304.4 |
| W-Flow | 2.185 | 22.99 | 0.426 | 0.440 | 0.326 | 0.031 | .827 | 23.18 | .148 | 56.40 | 323.3 |
| Method | Agent err (px) | Block xy (px) | Block angle ( ∘ ) | EPE mov. | Mov. MSE | Static drift |
|---|---|---|---|---|---|---|
| Kernel | 2.56 | 44.3 | 44.1 | 1.263 | .209 | .037 |
| Kernel + acc. | 2.20 | 44.7 | 44.6 | 1.238 | .207 | .036 |
| W-Flow | 1.56 | 44.1 | 44.3 | 1.231 | .198 | .036 |
| W-Flow + acc. | 1.54 | 45.4 | 44.1 | 1.268 | .206 | .036 |
| MSA (ours) | 1.44 | 42.1 | 43.6 | 1.181 | .182 | .037 |
| 15k | 40k | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | Extra hyperparameters | Sinkhorn solves/step | Zero push at target | Slope | Noise | Mov. MSE | EPE mov. | Slope | Noise | Tail share | ||
| W-Flow | – | 2 | n/a | .301 | .349 | 3.30 | .303 | 1.695 | .538 | .579 | 2.33 | .000 |
| W-Flow + acc. | (train + inference) | 3 | .266 | .279 | 2.74 | .312 | 1.696 | .471 | .446 | 1.81 | .000 | |
| UW-Flow | 2 | n/a | .266 | .341 | 7.80 | .314 | 1.724 | .471 | .546 | 5.15 | .000 | |
| UW-Flow + acc. | (matched ), | 3 | .320 | .337 | 3.67 | .286 | 1.659 | .177 | .364 | 39.8 | .542 | |
| MSA (ours) | fixed (no knob) | 2 + gate | .315 | .397 | 3.63 | .286 | 1.610 | .553 | .612 | 2.23 | .000 | |
| Variant | P / Z / S | Slope | Noise | Bias | Mov. MSE | |
|---|---|---|---|---|---|---|
| W-Flow | – / – / – | 0.693 | 0.713 | 1.49 | – | 0.1720 |
| + constant push | / / | 0.678 | 0.698 | 1.32 | 0.484 | 0.1761 |
| + joint softmax | / / | 0.694 | 0.718 | 1.44 | 0.492 | 0.1743 |
| MSA (ours) | / / | 0.712 | 0.746 | 1.55 | 0.462 | 0.1699 |
Action-Conditioned World Models
Action-conditioned world models predict how environments change under actions to support planning, control, and simulation.
See all 378 papers