cs.ROSep 9, 2026
SaveErgodic Control and Controlled Diffusion for Robot Learning: Review and Tutorial
Organizations: Northwestern University · Idiap Research Institute · Carnegie Mellon University · University of Rijeka · National University of Singapore · Pennsylvania State University · University of Sydney/Yale University
Abstract
Diffusion learning leverages the statistical mechanism of diffusion processes for learning, reasoning, and inferring complex distributions from data. Recent advances in diffusion learning have been transformative, with robot learning emerging as a key opportunity area, with applications spanning perception, control, and decision-making. At the same time, the statistical mechanism of diffusion processes can be controlled to shape the temporal evolution of the state distribution underlying robot trajectories, inducing ergodic behavior in robotic systems. The frameworks of controlled diffusion and ergodic control were developed around the same time as diffusion learning, and their theories and algorithms have increasingly converged. Ergodicity induced by controlled diffusion has several significant implications for robot learning, distinct from applying diffusion learning to robotics problems: it formally enforces statistical properties required to ensure optimality of robot learning, enables non-myopic search over uncertain information landscapes for data collection, and enables behavior specification based on spatial rather than temporal characteristics of trajectories. This survey introduces the intuition behind controlled diffusion for robot learning, explores its connection to diffusion learning, presents theoretical foundations and numerical tutorials for solving controlled diffusion and ergodic control problems, and reviews applications across robotics. Finally, we discuss key challenges and future opportunities in leveraging controlled diffusion for robot learning.
Figures & tables
Figure 1.1 : Illustrations of ergodic trajectories and ergodic control in action.
Figure 2.1 : Illustration of the “rectangle game”. We assume the binary measurements are sampled from an evenly distributed grid within an arbitrary but fixed squared area. The learned rectangle is computed as the smallest rectangle that contains all the positive signals and none of the negative signals.
Figure 2.3 : There exists a computable lower bound of the error between the ground truth rectangle and the learned rectangle from a finite number of samples. Furthermore, this bound decreases as the number of samples increases.
Figure 2.5 : Illustration of the BoxGPT example. Training and inference of the generative model are straightforward; what is not straightforward is data collection.
Figure 2.6 : Exhaustive sampling is an effective but inefficient data collection strategy.
Figure 2.7 : Based on the model’s current predictions, sampling measurements from different regions—regardless of their exact values—has varying impacts on model improvement. Identifying the optimal data distribution to sample from is the key to optimal data collection for learning.
Figure 2.9 : Increasing the sampling frequency along a trajectory where the time-averaged statistics fail to converge to a stationary distribution may not yield a better dataset.
Figure 2.11 : Temporal vs. spatial specifications for motion synthesis. (a) Conventional methods focus on the temporal sequence of robot actions. (b) Controlled diffusion focuses on the spatial statistics of the robot trajectory.
Figure 3.1 : For generative models, the diffusion process transforms a simple distribution into a complex one. Controlled diffusion follows the same principle, but applies the diffusion process to motion synthesis, generating trajectories that match desired time-averaged statistics. There are two perspectives on controlled diffusion: in the first, trajectories are generated incrementally over time (incremental generation), while in the second, they are iteratively optimized across the entire horizon (trajectory optimization).
Figure 4.1 : Diagrams for specifying an ergodic control implementation.
| Fourier | Kernelized | MMD | KL | Sinkhorn | |
|---|---|---|---|---|---|
| Target representation | Grid / Samples | Normalized density | Samples | Score function | Grid / samples |
| Time complexity | |||||
| Space complexity | |||||
| Dimension scaling | Exponential | Linear | Linear | Linear | Linear |
| Domain | Bounded rectangular Euclidean domain | Any domain with a suitable kernel | Any domain with a suitable kernel | Euclidean; specialized extensions for manifolds | Any domain with a suitable kernel |
| Advantages | Numerically stable and robust | Scalability and flexibility across domains | Works directly with samples | Supports unnormalized target densities | Handles irregular or disjoint supports |
Table 4.1 : Comparison of ergodic metrics.
Figure 4.2 : The number of Fourier coefficients determines the reconstruction accuracy of the target distribution and the resulting ergodic trajectory.
Figure 4.3 : Kernel ergodic control effectively combined information maximization behavior with uniform coverage behavior to synthesize optimal ergodic trajectories.
Figure 4.4 : The iterative LQR process is shown to be equivalent to the flow matching [ 113 ] framework in generative models subject to dynamics constraints [ 176 ] .
Figure 4.5 : (a) Example of ergodic exploration using HEDAC FEM method presented in [ 81 ] . The triangular discretization allows for curved outer and inner boundaries of the domain. (b) 3D trajectories filling a torus with HEDAC FEM implementation from [ 79 ] .
Figure 5.1 : Filtered Ergodic Control. Shown is an example ergodic trajectory (orange) for a planar drone-like system covering the darker areas in the background. Obstacles to be avoided are shown in green where the non-occupied space defines the safe set for the drone. The red arrow illustrates the magnitude of the ergodic control vector while the blue arrow is the closest resulting safe controller that avoids the obstacle.
Figure 5.2 : Integrated Safety Ergodic Control. Here, the optimized ergodic trajectory (orange) has the safety barrier constraint satisfied throughout the planning horizon. The blue vector is the commanded ergodic control with integrated safety is that follows the optimized path. Performance of the ergodic controller is preserved while maintaining safety throughout.
Figure 5.3 : Time-Evolution of a 2D drone following a robust ergodic controller. Top row: the time-averaged spatial distribution and the information distribution. As time increases, the time-averaged trajectory statistics approach the information distribution, which is the goal of ergodic exploration. Bottom: evolution of the trajectory through phase space, superimposed on a cross-section of the value function with respect to position and velocity, where are fixed to their value at that instant in the trajectory.
Figure 5.4 : SVGD Prevents Mode Collapse and Promotes Diverse Ergodic Controllers. Here, we demonstrate mode collapse of inferring ergodic trajectories, i.e., when there is no repulsive force to diversify solutions. (Left) Ergodic trajectories randomly initialized converge to similar solutions. (Right) With the repulsive force from SVGD, the trajectory solutions leverage the multimodal nature of the ergodic metric, yielding diverse solutions, and ultimately solution robustness to uncertainty. Mode collapse can occur with increased dimensionality of the optimization variables which can be mitigated with the right choice of kernel [ 115 ] .
Figure 5.5 : GPU Acceleration of SVGD and the Ergodic metric for Robust Control. (Left) Example SVGD for robust ergodic control in an online exploration problem (cover dark areas) with dynamics obstacles (green). A total of ergodic trajectories are calculated (shown in black). The red line indicates the best possible solution according to a heuristic ( [ 104 ] ), and blue line indicates visited areas. (Right) Computational scaling across devices. The Stein variational ergodic controller can optimize several paths that can be used as redundancies to attain robustness in uncertain and dynamic environments.
Figure 5.6 : Time Optimal Ergodic Trajectory Evolution [ 44 ] in clutter. As one lower the upper bound , satisfying the ergodic metric requires the solver to search for larger which allows the robot trajectory to cover more of the space. Here, is given as a uniform distribution.
Figure 5.7 : Energy-Aware Ergodic Coverage . State of charge awareness can maintain a persistent level of ergodicity. Each agent shown trades off uniform coverage over the space and battery charge. High density areas are defined via whereas charging stations are denoted by .
Figure 5.8 : Ergodicity as a function of state-of-charge . The top plot shows the evolution of the ergodicity for a set of agents. The bottom shows the average ergodicity. Initially, the agents are at their charging stations. As agents begin to lose battery charge, they they start moving to the charging stations – ergodicity increases while other agents maintain the average ergodicity to be under the value .
Figure 5.9 : The Laplacian eigenbasis extends Fourier analysis to curved manifolds, including surfaces reconstructed from point clouds. This basis enables efficient closed-loop ergodic control on such surfaces, supporting operations such as cleaning or surface inspection.
Figure 5.10 : Example spectral decomposition of an information prior with two heterogeneous agents: A given information prior is decomposed directly in the spectral domain, and the spectral coefficients assigned to the different agents based on human intuition. Here, the UAV with omnidirectional motion capabilities and high-range, low-fidelity sensing (e.g., onboard camera) is assigned low-order coefficients, allowing it to quickly cover the lowest-frequency portions of the information space. Conversely, a ground vehicle with curve-constrained motion constraints, equipped with a low-range, high-fidelity sensor (e.g., LiDAR) is assigned higher-frequency coefficients, allowing it to easily chain peaks of information.
Figure 5.11 : Learning spectral allocations [ 149 ] : Relying on the A3C algorithm with three different distribution schemes, agents learn their allocation to search subtasks. Each distribution method has different initial conditions. In all three methods, the output is a weighted set of spectral coefficients of the information distribution being searched for each agent. The learning process seeks to minimize the joint ergodic metric of the team’s trajectories.
Figure 6.1 : Examples of ergodic control applications in information gathering and active perception. From top to bottom: (1) Information gathering with UAVs over uneven terrains; (2) Information gathering with human-swarm systems; (3) Information gathering over disaster sites; (4) Active perception with binary sensors; (5) Active perception with generative models.
Figure 6.2 : Examples of ergodic control applications in manipulation. From left to right: (1) Learning insertion policies from human demonstrations; (2) Reactive erasing in dynamic environments; (3) Erasing over curved surfaces with tactile feedback.
Figure 6.3 : Examples of ergodic control applications in human-robot interaction. From left to right: (1) Ergodicity as a measure of task-relevant information in patients’ motion during rehabilitation; (2) Ergodic control ensures safety while maintaining adaptivity in physical therapy; (3) Ergodic control for learning from physical demonstration.
Figure 6.4 : Examples of ergodic control applications in manufacturing. From left to right: (1) Micro-patterned surface manufacturing with swarm robots; (2) Learning assembly from demonstrations; (3) Autonomous inspection of manufactured components.
Figure 6.5 : Examples of ergodic control applications in robot art.