KoopmanFlow: Spectrally Decoupled Generative Control Policy via Koopman Structural Bias

Chengsi Yao; Ge Wang; Kai Kang; Shenhao Yan; Jiahao Yang; Fan Feng; Honghao Cai; Xianxian Zeng

cs.ROMar 14, 2026

KoopmanFlow: Spectrally Decoupled Generative Control Policy via Koopman Structural Bias

Authors: Chengsi Yao, Ge Wang, Kai Kang, Shenhao Yan, Jiahao Yang, Fan Feng, Honghao Cai, Xianxian Zeng, +4 more

Organizations: ISing AI · Future Network of Intelligence Institution, The Chinese University of Hong Kong, Shenzhen · Guangdong Polytechnic Normal University · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen

Abstract

Generative Control Policies (GCPs) show immense promise in robotic manipulation but struggle to simultaneously model stable global motions and high-frequency local corrections. While modern architectures extract multi-scale spatial features, their underlying Probability Flow ODEs apply a uniform temporal integration schedule. Compressed to a single step for real-time Receding Horizon Control (RHC), uniform ODE solvers mathematically smooth over sparse, high-frequency transients entangled within low-frequency steady states. To decouple these dynamics without accumulating pipelined errors, we introduce KoopmanFlow, a parameter-efficient generative policy guided by a Koopman-inspired structural inductive bias. Operating in a unified multimodal latent space with visual context, KoopmanFlow bifurcates generation at the terminal stage. Because visual conditioning occurs before spectral decomposition, both branches are visually guided yet temporally specialized. A macroscopic branch anchors slow-varying trajectories via single-step Consistency Training, while a transient branch uses Flow Matching to isolate high-frequency residuals stimulated by sudden visual cues (e.g., contacts or occlusions). Guided by an explicit spectral prior and optimized via a novel asymmetric consistency objective, KoopmanFlow establishes a fused co-training mechanism. This allows the variant branch to absorb localized dynamics without multi-stage error accumulation. Extensive experiments show KoopmanFlow significantly outperforms state-of-the-art baselines in contact-rich tasks requiring agile disturbance rejection. By trading a surplus latency buffer for a richer structural prior, KoopmanFlow achieves superior control fidelity and parameter efficiency within real-time deployment limits.

Figures & tables

Fig. 1: Latent Space Topography of Multi-Dataset Action Features. We visualize the action manifolds across multiple manipulation tasks, revealing severe topological overlap of distinct kinematic frequencies. Time-variant critical states (colored dots, representing grasping or contact) are deeply intertwined within the continuous trajectories of massive time-invariant steady states (grey dots). This severe spatial overlap highlights why standard continuous ODEs struggle: a single vector field cannot simultaneously resolve smooth inertia and high-frequency reactive corrections within the same topological neighborhood.

Fig. 2: Architectural comparison of generative backbones. Left (DiT): The standard formulation using flattened observation conditioning. Middle (DiTx [ 27 ] ): An architecture utilizing cascading cross-attention layers. Right (Ours): The proposed hierarchical condition injection, where the terminal Hybrid Koopman FFN decouples the velocity field into time-invariant ( $\mathbf{v}_{inv}$ ) and time-variant ( $\mathbf{v}_{var}$ ) dynamic components.

Fig. 3 : KoopmanFlow architecture. Multi-modal features are processed via DiT and spectrally decoupled by a terminal HKFFN into time-invariant ( $\mathbf{v}_{inv}$ ) and time-variant ( $\mathbf{v}_{var}$ ) velocity fields. Fused Co-Training ensures exact spectral decomposition and single-step generation (NFE = 1) for real-time control.

(Image-based)
Algorithm	Obs.	Adroit				DexArt
Algorithm	Obs.	Hammer	Door	Pen	Avg.	Laptop	Faucet	Bucket	Toilet	Avg.
Diffusion Policy	Img	$54.0\pm 3.6$	$41.8\pm 2.7$	$18.5\pm 2.5$	$38.1\pm 2.9$	$81.7\pm 2.1$	$29.3\pm 2.1$	$26.0\pm 2.4$	$77.3\pm 1.9$	$53.6\pm 2.1$
Flow Matching	Img	$55.7\pm 4.2$	$40.0\pm 1.6$	$21.2\pm 0.8$	$39.0\pm 2.2$	$81.7\pm 2.5$	$31.3\pm 3.7$	$24.0\pm 2.2$	$76.3\pm 1.2$	$53.3\pm 2.4$
2D ManiFlow	Img	$100.0\pm 0.0$	$67.0\pm 2.2$	$56.0\pm 3.6$	$74.3\pm 1.9$	$85.7\pm 2.1$	$32.3\pm 0.5$	$29.7\pm 3.4$	$77.7\pm 3.3$	$56.3\pm 2.3$
KoopmanFlow (Ours)	Img	$\mathbf{100.0\pm 0.0}$	$\mathbf{68.3\pm 1.8}$	$\mathbf{62.7\pm 2.4}$	$\mathbf{77.0\pm 1.4}$	$\mathbf{87.7\pm 1.5}$	$\mathbf{41.3\pm 1.2}$	$\mathbf{36.7\pm 2.0}$	$\mathbf{81.3\pm 1.8}$	$\mathbf{61.8\pm 1.6}$

TABLE I : Algorithm Performance Comparison on Adroit and DexArt Tasks.

Panel of Figure 5 — Fig. 4 : Experimental Analysis. (Left) MetaWorld success rates across diverse tasks. (Right) Sensitivity analysis of the cumulative energy threshold $\alpha$ . KoopmanFlow exhibits a robust performance plateau when $\alpha\in[80\%,90\%]$ , gracefully avoiding loss of global structure and high-frequency noise absorption.

Fig. 6 : Quantitative success rate comparison between ManiFlow and KoopmanFlow across eight real-world manipulation tasks. To maintain fairness, both models utilize a frozen R3M (ResNet-18) visual encoder from a single-camera view and are trained on 65 episodes per task. Results are averaged over 30 real-world trials under varied environmental conditions using 1 and 10 inference steps. KoopmanFlow exhibits superior performance, notably in long-horizon tasks requiring continuous TPU gripper coordination.

Fig. 7 : Qualitative visualization of the decomposed latent dynamics in the Hybrid Koopman FFN. (Top) The time-invariant operator ( $\mathbf{v}_{inv}$ ) maintains consistently stable absolute energy across the trajectory, capturing physical inertia unperturbed by sudden visual events. (Middle) The visually-conditioned time-variant operator ( $\mathbf{v}_{var}$ ) exhibits sharp energy spikes that are highly correlated with semantic visual events (dashed lines), acting as a vision-driven reactive module to environmental disruptions. (Bottom) Naive high-frequency energy extracted via RFFT on raw action data lacks visual context and fails to align with these critical semantic events. This contrast demonstrates that KoopmanFlow achieves true semantic spectral decoupling rather than blind mathematical filtering.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 8 : Visualization of representative simulation tasks from Adroit, DexArt, and RoboTwin 2.0.

Benchmark	Demonstrations (Per Task)
Adroit	10
DexArt	100
MetaWorld	10
RoboTwin 2.0	50

Appendix

TABLE VIII : Number of expert demonstrations used in each simulation benchmark.

Fig. 9 : Mean episode length and relative performance improvement of KoopmanFlow on 3D DexArt. Tasks with longer effective horizons generally show larger gains, supporting the claim that KoopmanFlow is particularly beneficial for long-horizon and contact-rich manipulation.

Fig. 10 : Visualization of Real-World Tasks. The tasks span from complex bimanual coordination to high-precision single-arm interactions.

Fig. 11 : Representative failure cases in real-world unseen-object handover tasks. The top row shows Stick Handover, where the manipulated object is substantially larger and heavier than those seen during training, making the contact-transfer dynamics harder to handle and leading to unsuccessful execution. The bottom row shows Pen Handover, where the small object size together with finger-object occlusion increases the difficulty of precise grasp localization, causing failure at an earlier grasping stage.

Fig. 12 : Water Pouring under instruction-object mismatch. Each row shows a four-step sequence. When the bottle identity matches the instruction (Coke), the policy completes the pouring task successfully. When the instruction is kept as Coke but the manipulated bottle is mismatched (Water or Sprite), the evolving monocular observation becomes inconsistent with the language-conditioned target identity during manipulation, and the policy suppresses the lift-and-pour transient and transitions to a conservative termination routine.

	Meta-World (Easy)
Alg $\backslash$ Task	Button Press	Button Press Topdown	Button Press Topdown Wall	Button Press Wall	Coffee Button	Dial Turn	Door Close
3D Diffusion Policy	$62\pm 15$	$100\pm 0$	$100\pm 0$	$72\pm 25$	$73\pm 34$	$53\pm 15$	$100\pm 0$
3D Flow Matching	$0\pm 0$	$100\pm 0$	$100\pm 0$	$67\pm 31$	$97\pm 5$	$70\pm 7$	$100\pm 0$
3D ManiFlow	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$67\pm 13$	$100\pm 0$
KoopmanFlow	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{96\pm 2}$	$\mathbf{100\pm 0}$

Appendix

TABLE IX : Language-conditioned multi-task results on 48 Meta-World simulation tasks.

Explore similar work

Jul 4, 2026

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

Generative models such as diffusion and flow matching have advanced robotic visuomotor policies by modeling multimodal action distributions, but their multi-step sampling or ODE solving introduces inference latency. Existing one-step acceleration methods often compress the whole generation process into a single large update, leading to spatial deviation, frequency distortion, and mode averaging. This paper proposes a high-fidelity one-step generative visuomotor policy framework that addresses these issues with three complementary mechanisms. Recursive Consistent Action Flow (RCAF) uses recursive correction to compensate for spatial truncation errors and align one-step predictions with refined flow trajectories. Dual-Timestep Frequency Consistency (DTFC) preserves high-frequency manipulation details through adaptive spectral consistency across flow timesteps. Contrastive Flow Matching (CFM) separates entangled action flows with a margin-based repulsive objective, reducing ambiguous actions in multimodal manipulation. Experiments on RoboTwin, RoboTwin 2.0, Adroit, DexArt, and real-world robot platforms show that the proposed method achieves competitive or superior performance compared with strong 10-step generative policy baselines while requiring only one forward pass (1 NFE), enabling low-latency visuomotor control.

Yuran Chen, Xinye Cai, Zhonglin Gong +1

School of Safety Science and Engineering, Anhui University of Science and Technology, Huainan 232001, China · State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science and Technology, Huainan 232001, China · School of Artificial Intelligence, Anhui University of Science and Technology, Hefei 231131, China

Flow Matching Visuomotor Policy Learning One-Step Generative Modeling Robotic Manipulation

Ask PDF

Mar 18, 2026

Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control

Diffusion models and flow matching have become a cornerstone of robotic imitation learning, yet they suffer from a structural inefficiency where inference is often bound to a fixed integration schedule that is agnostic to state complexity. This paradigm forces the policy to expend the same computational budget on trivial motions as it does on complex tasks. We introduce Generative Control as Optimization (GeCO), a time-unconditional framework that transforms action synthesis from trajectory integration into iterative optimization. GeCO learns a stationary velocity field in the action-sequence space where expert behaviors form stable attractors. Consequently, test-time inference becomes an adaptive process that allocates computation based on convergence--exiting early for simple states while refining longer for difficult ones. Furthermore, this stationary geometry yields an intrinsic, training-free safety signal, as the field norm at the optimized action serves as a robust out-of-distribution (OOD) detector, remaining low for in-distribution states while significantly increasing for anomalies. We validate GeCO on standard simulation benchmarks and demonstrate seamless scaling to pi0-series Vision-Language-Action (VLA) models. As a plug-and-play replacement for standard flow-matching heads, GeCO improves success rates and efficiency with an optimization-native mechanism for safe deployment. Video and code can be found at https://hrh6666.github.io/GeCO/

Zunzhe Zhang, Runhan Huang, Yicheng Liu +3

Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China · Galaxea Inc., Beijing, China · Department of Computer Science, Princeton University, New Jersey, US

Robotic Control Robot Imitation Learning

Ask PDF

Mar 12, 2026

One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies

Generative flow and diffusion models provide the continuous, multimodal action distributions needed for high-precision robotic policies. However, their reliance on iterative sampling introduces severe inference latency, degrading control frequency and harming performance in time-sensitive manipulation. To address this problem, we propose the One-Step Flow Policy (OFP), a from-scratch self-distillation framework for high-fidelity, single-step action generation without a pre-trained teacher. OFP unifies a self-consistency loss to enforce coherent transport across time intervals, and a self-guided regularization to sharpen predictions toward high-density expert modes. In addition, a warm-start mechanism leverages temporal action correlations to minimize the generative transport distance. Evaluations across 56 diverse simulated manipulation tasks demonstrate that a one-step OFP achieves state-of-the-art results, outperforming 100-step diffusion and flow policies while accelerating action generation by over

100\times

. We further integrate OFP into the

π_{0.5}

model on RoboTwin 2.0, where one-step OFP surpasses the original 10-step policy. These results establish OFP as a practical, scalable solution for highly accurate and low-latency robot control.

Shaolong Li, Lichao Sun, Yongchao Chen

CSE Department, University of Michigan · Lehigh University · College of AI, Tsinghua University

Visuomotor Policy Learning One-Step Generative Modeling Flow-Based Generative Modeling Robotic Manipulation Self-Distillation

Ask PDF