KoopmanFlow: Spectrally Decoupled Generative Control Policy via Koopman Structural Bias

Chengsi Yao; Ge Wang; Kai Kang; Shenhao Yan; Jiahao Yang; Fan Feng; Honghao Cai; Xianxian Zeng

cs.ROMar 14, 2026

KoopmanFlow: Spectrally Decoupled Generative Control Policy via Koopman Structural Bias

Authors: Chengsi Yao, Ge Wang, Kai Kang, Shenhao Yan, Jiahao Yang, Fan Feng, Honghao Cai, Xianxian Zeng, +4 more

Organizations: ISing AI · Future Network of Intelligence Institution, The Chinese University of Hong Kong, Shenzhen · Guangdong Polytechnic Normal University · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen

Abstract

Generative Control Policies (GCPs) show immense promise in robotic manipulation but struggle to simultaneously model stable global motions and high-frequency local corrections. While modern architectures extract multi-scale spatial features, their underlying Probability Flow ODEs apply a uniform temporal integration schedule. Compressed to a single step for real-time Receding Horizon Control (RHC), uniform ODE solvers mathematically smooth over sparse, high-frequency transients entangled within low-frequency steady states. To decouple these dynamics without accumulating pipelined errors, we introduce KoopmanFlow, a parameter-efficient generative policy guided by a Koopman-inspired structural inductive bias. Operating in a unified multimodal latent space with visual context, KoopmanFlow bifurcates generation at the terminal stage. Because visual conditioning occurs before spectral decomposition, both branches are visually guided yet temporally specialized. A macroscopic branch anchors slow-varying trajectories via single-step Consistency Training, while a transient branch uses Flow Matching to isolate high-frequency residuals stimulated by sudden visual cues (e.g., contacts or occlusions). Guided by an explicit spectral prior and optimized via a novel asymmetric consistency objective, KoopmanFlow establishes a fused co-training mechanism. This allows the variant branch to absorb localized dynamics without multi-stage error accumulation. Extensive experiments show KoopmanFlow significantly outperforms state-of-the-art baselines in contact-rich tasks requiring agile disturbance rejection. By trading a surplus latency buffer for a richer structural prior, KoopmanFlow achieves superior control fidelity and parameter efficiency within real-time deployment limits.

Figures & tables

Fig. 1: Latent Space Topography of Multi-Dataset Action Features. We visualize the action manifolds across multiple manipulation tasks, revealing severe topological overlap of distinct kinematic frequencies. Time-variant critical states (colored dots, representing grasping or contact) are deeply intertwined within the continuous trajectories of massive time-invariant steady states (grey dots). This severe spatial overlap highlights why standard continuous ODEs struggle: a single vector field cannot simultaneously resolve smooth inertia and high-frequency reactive corrections within the same topological neighborhood.

Fig. 2: Architectural comparison of generative backbones. Left (DiT): The standard formulation using flattened observation conditioning. Middle (DiTx [ 27 ] ): An architecture utilizing cascading cross-attention layers. Right (Ours): The proposed hierarchical condition injection, where the terminal Hybrid Koopman FFN decouples the velocity field into time-invariant ( $\mathbf{v}_{inv}$ ) and time-variant ( $\mathbf{v}_{var}$ ) dynamic components.

Fig. 3 : KoopmanFlow architecture. Multi-modal features are processed via DiT and spectrally decoupled by a terminal HKFFN into time-invariant ( $\mathbf{v}_{inv}$ ) and time-variant ( $\mathbf{v}_{var}$ ) velocity fields. Fused Co-Training ensures exact spectral decomposition and single-step generation (NFE = 1) for real-time control.

(Image-based)
Algorithm	Obs.	Adroit				DexArt
Algorithm	Obs.	Hammer	Door	Pen	Avg.	Laptop	Faucet	Bucket	Toilet	Avg.
Diffusion Policy	Img	$54.0\pm 3.6$	$41.8\pm 2.7$	$18.5\pm 2.5$	$38.1\pm 2.9$	$81.7\pm 2.1$	$29.3\pm 2.1$	$26.0\pm 2.4$	$77.3\pm 1.9$	$53.6\pm 2.1$
Flow Matching	Img	$55.7\pm 4.2$	$40.0\pm 1.6$	$21.2\pm 0.8$	$39.0\pm 2.2$	$81.7\pm 2.5$	$31.3\pm 3.7$	$24.0\pm 2.2$	$76.3\pm 1.2$	$53.3\pm 2.4$
2D ManiFlow	Img	$100.0\pm 0.0$	$67.0\pm 2.2$	$56.0\pm 3.6$	$74.3\pm 1.9$	$85.7\pm 2.1$	$32.3\pm 0.5$	$29.7\pm 3.4$	$77.7\pm 3.3$	$56.3\pm 2.3$
KoopmanFlow (Ours)	Img	$\mathbf{100.0\pm 0.0}$	$\mathbf{68.3\pm 1.8}$	$\mathbf{62.7\pm 2.4}$	$\mathbf{77.0\pm 1.4}$	$\mathbf{87.7\pm 1.5}$	$\mathbf{41.3\pm 1.2}$	$\mathbf{36.7\pm 2.0}$	$\mathbf{81.3\pm 1.8}$	$\mathbf{61.8\pm 1.6}$

TABLE I : Algorithm Performance Comparison on Adroit and DexArt Tasks.

Panel of Figure 5 — Fig. 4 : Experimental Analysis. (Left) MetaWorld success rates across diverse tasks. (Right) Sensitivity analysis of the cumulative energy threshold $\alpha$ . KoopmanFlow exhibits a robust performance plateau when $\alpha\in[80\%,90\%]$ , gracefully avoiding loss of global structure and high-frequency noise absorption.

Fig. 6 : Quantitative success rate comparison between ManiFlow and KoopmanFlow across eight real-world manipulation tasks. To maintain fairness, both models utilize a frozen R3M (ResNet-18) visual encoder from a single-camera view and are trained on 65 episodes per task. Results are averaged over 30 real-world trials under varied environmental conditions using 1 and 10 inference steps. KoopmanFlow exhibits superior performance, notably in long-horizon tasks requiring continuous TPU gripper coordination.

Fig. 7 : Qualitative visualization of the decomposed latent dynamics in the Hybrid Koopman FFN. (Top) The time-invariant operator ( $\mathbf{v}_{inv}$ ) maintains consistently stable absolute energy across the trajectory, capturing physical inertia unperturbed by sudden visual events. (Middle) The visually-conditioned time-variant operator ( $\mathbf{v}_{var}$ ) exhibits sharp energy spikes that are highly correlated with semantic visual events (dashed lines), acting as a vision-driven reactive module to environmental disruptions. (Bottom) Naive high-frequency energy extracted via RFFT on raw action data lacks visual context and fails to align with these critical semantic events. This contrast demonstrates that KoopmanFlow achieves true semantic spectral decoupling rather than blind mathematical filtering.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 8 : Visualization of representative simulation tasks from Adroit, DexArt, and RoboTwin 2.0.

Benchmark	Demonstrations (Per Task)
Adroit	10
DexArt	100
MetaWorld	10
RoboTwin 2.0	50

Appendix

TABLE VIII : Number of expert demonstrations used in each simulation benchmark.

Fig. 9 : Mean episode length and relative performance improvement of KoopmanFlow on 3D DexArt. Tasks with longer effective horizons generally show larger gains, supporting the claim that KoopmanFlow is particularly beneficial for long-horizon and contact-rich manipulation.

Fig. 10 : Visualization of Real-World Tasks. The tasks span from complex bimanual coordination to high-precision single-arm interactions.

Fig. 11 : Representative failure cases in real-world unseen-object handover tasks. The top row shows Stick Handover, where the manipulated object is substantially larger and heavier than those seen during training, making the contact-transfer dynamics harder to handle and leading to unsuccessful execution. The bottom row shows Pen Handover, where the small object size together with finger-object occlusion increases the difficulty of precise grasp localization, causing failure at an earlier grasping stage.

Fig. 12 : Water Pouring under instruction-object mismatch. Each row shows a four-step sequence. When the bottle identity matches the instruction (Coke), the policy completes the pouring task successfully. When the instruction is kept as Coke but the manipulated bottle is mismatched (Water or Sprite), the evolving monocular observation becomes inconsistent with the language-conditioned target identity during manipulation, and the policy suppresses the lift-and-pour transient and transitions to a conservative termination routine.

	Meta-World (Easy)
Alg $\backslash$ Task	Button Press	Button Press Topdown	Button Press Topdown Wall	Button Press Wall	Coffee Button	Dial Turn	Door Close
3D Diffusion Policy	$62\pm 15$	$100\pm 0$	$100\pm 0$	$72\pm 25$	$73\pm 34$	$53\pm 15$	$100\pm 0$
3D Flow Matching	$0\pm 0$	$100\pm 0$	$100\pm 0$	$67\pm 31$	$97\pm 5$	$70\pm 7$	$100\pm 0$
3D ManiFlow	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$100\pm 0$	$67\pm 13$	$100\pm 0$
KoopmanFlow	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{100\pm 0}$	$\mathbf{96\pm 2}$	$\mathbf{100\pm 0}$

Appendix

TABLE IX : Language-conditioned multi-task results on 48 Meta-World simulation tasks.

Explore similar work

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching
Jul 4, 2026Yuran Chen, Xinye Cai, Zhonglin Gong +1Flow Matching Visuomotor Policy Learning
School of Safety Science and Engineering, Anhui University of Science and Technology, Huainan 232001, China · State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science and Technology, Huainan 232001, China · School of Artificial Intelligence, Anhui University of Science and Technology, Hefei 231131, China
Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control
Mar 18, 2026Zunzhe Zhang, Runhan Huang, Yicheng Liu +3Robotic Control Robot Imitation Learning
Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China · Galaxea Inc., Beijing, China · Department of Computer Science, Princeton University, New Jersey, US
One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies
Mar 12, 2026Shaolong Li, Lichao Sun, Yongchao ChenVisuomotor Policy Learning One-Step Generative Modeling
CSE Department, University of Michigan · Lehigh University · College of AI, Tsinghua University

Jul 4, 2026J/K move · Enter open · S save

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

Yuran Chen, Xinye Cai, Zhonglin Gong, Yang Huang

School of Safety Science and Engineering, Anhui University of Science and Technology, Huainan 232001, China · State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science and Technology, Huainan 232001, China · School of Artificial Intelligence, Anhui University of Science and Technology, Hefei 231131, China

Flow Matching Visuomotor Policy Learning One-Step Generative Modeling Robotic Manipulation

Open paper AskPDF arXiv

Generative models such as diffusion and flow matching have advanced robotic visuomotor policies by modeling multimodal action distributions, but their multi-step sampling or ODE solving introduces inference latency. Existing one-step acceleration methods often compress the whole generation process into a single large update, leading to spatial deviation, frequency distortion, and mode averaging. This paper proposes a high-fidelity one-step generative visuomotor policy framework that addresses these issues with three complementary mechanisms. Recursive Consistent Action Flow (RCAF) uses recursive correction to compensate for spatial truncation errors and align one-step predictions with refined flow trajectories. Dual-Timestep Frequency Consistency (DTFC) preserves high-frequency manipulation details through adaptive spectral consistency across flow timesteps. Contrastive Flow Matching (CFM) separates entangled action flows with a margin-based repulsive objective, reducing ambiguous actions in multimodal manipulation. Experiments on RoboTwin, RoboTwin 2.0, Adroit, DexArt, and real-world robot platforms show that the proposed method achieves competitive or superior performance compared with strong 10-step generative policy baselines while requiring only one forward pass (1 NFE), enabling low-latency visuomotor control.

Abstract

Explore similar work

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control

One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies