Notes on Generative Modeling for Feedback Control and Planning

Karthik Elamvazhuthi

math.OCSep 27, 2026

Notes on Generative Modeling for Feedback Control and Planning

Authors: Karthik Elamvazhuthi

Abstract

In these notes, we view control as a dynamically constrained sampling problem on the state-space of a control system. With this viewpoint, we extend methods from generative modeling, such as flow matching, normalizing flows and denoising diffusions to control problems. Concepts such as controllability, optimal control and trajectory planning play an important role in guiding the extension and understanding well-posedeness of the corresponding algorithms, with application to steering systems to target states or distributions and sampling from reachable sets. The notes are intended as an accessible introduction for readers with a background in control theory and robotics.

Figures & tables

Figure 2.1: From a single trajectory to an evolving density. Left: A single solution

x(t)

\dot{x}=f(x)

from a known initial condition

x_{0}

. Center: An ensemble of trajectories from initial conditions drawn from

\rho_{0}

. Right: The probability density

\rho_{t}

at successive times; as trajectories compress, the density narrows and its peak grows.

Figure 2.2: Controlled transport from the left gyre-core distribution toward a point target near $(1.5,0.5)$ . The terminal density is concentrated in a Dirac target.

Figure 2.3: Controlled transport between the two gyre-core distributions.

Panel of Figure 3 — Figure 3.1: Steering the system $\dot{x}_{1}=u_{1}$ , $\dot{x}_{2}=u_{2}$ , $\dot{x}_{3}=x_{2}u_{1}$ toward the origin. Top: open-loop trajectories used to generate data for the flow matching algorithm. Bottom: closed-loop trajectories under the learned feedback. Each row shows the initial, intermediate, and terminal states from left to right. Blue points represent the current states, and the red star marks the target.

Figure 3.2: Noising and denoising for an underactuated system on $SO(3)$ , visualized through the first column $Re_{1}\in S^{2}$ . Top: noising from the identity under bounded Brownian-motion controls. Bottom: learned denoising from Haar measure. Each row progresses from left to right. Dots indicate current column vectors, curves show their paths, and the star marks $e_{1}$ .

Figure 3.3: RRT*-based noising and learned denoising. Top: tree growth at iterations $0$ , $600$ , $1200$ , and $2000$ . Bottom: learned return trajectories from fresh initial states sampled uniformly in free space, shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ . Curves show accumulated paths, dots mark current states, and the star marks the goal.

Figure 3.4: Dijkstra-based flow matching. Top: outward propagation of the shortest-path tree from the goal configuration at successive stages of the Dijkstra algorithm. Bottom: trajectories generated by the learned continuous feedback controller from fresh initial conditions sampled independently in the free space. Curves show accumulated trajectories, dots indicate the current states, and the star marks the goal.

Approach	Procedure
Classical optimal control	Fix initial and final conditions, then solve the PMP-based boundary value problem or the HJB equation.
Denoising-based optimal control	Fix the final condition, propagate states through the backward-reachable set using the PMP system, and denoise to synthesize a feedback controller.

Table 3.1: Classical and denoising-based routes to feedback synthesis.

Figure 3.5: PMP-based noising and learned denoising for the unicycle. Top: trajectories spreading from the goal under sampled adjoint conditions. Bottom: learned feedback trajectories from uniformly sampled initial configurations. Curves show paths, while arrows indicate the heading $x_{3}$ , and the star marks the target position.

Figure 4.1: Independent bounded control sampling for the Van der Pol oscillator with $\Delta t=0.2$ , $0.05$ , and $0.01$ . Blue curves show sampled trajectories, and blue dots mark their terminal states. The red curve shows the averaged dynamics with $u=0$ ; the star marks the common initial state.

Figure 4.2: White-noise scaling for the controlled Van der Pol oscillator. On each interval of length $\Delta t$ , the control is $u_{k}=\sigma\xi_{k}/\sqrt{\Delta t}$ , where $\xi_{k}\sim\mathcal{N}(0,1)$ are independent and $\sigma=2$ . Blue curves show sampled trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same horizon $T=6$ and axis limits.

Figure 4.3: Projected Brownian controls for the Van der Pol oscillator. The control is $u(t)=P_{[-4,4]}(2W_{t})$ , approximated by piecewise-constant values sampled at intervals of length $\Delta t$ . Blue curves show sampled state trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same Brownian realizations, horizon $T=6$ , and axis limits.

Figure 4.4: Random-switching controls for the Van der Pol oscillator. Switching times follow a Poisson process with rate $\lambda=2$ , and the initial control and each subsequent control value are sampled independently and uniformly from $[-4,4]$ . Blue curves show sampled state trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same switching times and control values, horizon $T=6$ , and axis limits. Here, $\Delta t$ specifies the time grid discretization.

Figure 4.5: Random-switching controls experiment repeated with $N=1000$ samples.

Figure 4.6: Kernel smoothing of an empirical measure. Left: point masses at the sampled terminal states. Center: Gaussian bumps centered at these states, each with integral $1/N$ . Right: the resulting density $\rho_{T,\delta}^{N}=k_{\delta}*\mu_{T}^{N}$ (red), obtained by summing the weighted bumps (blue).

Figure 4.7: Optimal-transport-based reachable-set sampling for the Van der Pol oscillator with $\mu=1$ and $|u|\leq 4$ . Panels show the states at $t/T=0$ , $0.3$ , $0.6$ , and $1$ . Blue points indicate current states, and faint blue curves show their accumulated paths. The red curve shows the zero-control trajectory up to the displayed time, and the star marks the common initial state $(1.5,0)$ . The terminal interaction cost promotes dispersion without requiring prior knowledge of the reachable set.

Figure 5.1: Continuous normalizing flow for the fully actuated planar system. Blue points show the evolving model samples, initialized from a Gaussian distribution, while faint red points show samples from the noisy two-moons target distribution. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with identical axis limits. At the terminal time, the learned flow approximately reproduces the two-moons distribution.

Figure 5.2: Controlled normalizing flow for a four-dimensional kinematic vehicle model. Blue points show current positions, and faint red points show samples from the two-moons target position distribution. Arrows indicate the heading of a subset of vehicles, rather than their velocity. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with $T=4$ and identical axis limits. Only the position coordinates are displayed

Figure 5.3: Entropy-maximizing controlled normalizing flow for the Van der Pol oscillator. Initial states are sampled uniformly from a disk of radius $0.25$ centered at $(1.5,0)$ . Blue points show the current states under the learned feedback $u^{\theta}\in[-4,4]$ , and blue curves show accumulated paths for a subset of particles. The red curve shows the zero-control trajectory from the initial center, marked by the star. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with $T=6$ and identical axis limits.

Figure 6.1: Evolution of reflected diffusion particles toward a four-component Gaussian-mixture target on $[0,1]^{2}$ . Blue points represent the particle states, red contours represent the target density $\rho_{\infty}$ , and the crosses mark the centers of the four component. The particles are initialized near $(0.5,0.5)$ and separate into the four target modes over time.

Figure 6.2: Stochastic gradient flow for the unicycle model. Blue points represent the unicycle positions, and the arrows indicate the headings of a subset of the particles. Red contours show the desired four-component Gaussian-mixture density, while the crosses mark its component centers. The positions are initialized near $(0.5,0.5)$ with uniformly distributed headings.

Figure 6.3: Convergence to Gaussian stationary densities. Top: fully actuated damped single integrator. Bottom: damped oscillator with noise in its second coordinate. Blue points show current states, and faint blue curves trace selected particles. The star marks the equilibrium.

Figure 6.4: Unicycle horizontal-gradient descent with fixed potential strength $K=10$ and increasing diffusion strength. Rows correspond to $D=0$ , $0.1$ , $1$ , and $4$ ; columns show successive times. Blue points indicate positions, arrows show headings, and faint curves trace selected particles. For $D>0$ , red circles enclose $50\%$ and $90\%$ of the stationary position probability.

Figure 7.1: Diffusion-based feedback planning around circular obstacles. Top: forward noising from the target. Bottom: deterministic denoising under the learned score feedback, initialized uniformly in free space. Points show current states, and curves trace selected particles up to the displayed time. Gray disks represent obstacles, and the star marks the target.

Figure 7.2: Noising and denoising for the unicycle. Top: forward diffusion initialized at the target. Bottom: deterministic denoising using the score network selected by the lowest validation loss. Points show positions, arrows indicate headings, and faint curves trace selected trajectories. The star marks the target position; gray disks represent obstacles.

Figure 7.3: Noising and denoising for a four-dimensional controllable linear system. The plot shows eterministic denoising from Gaussian samples. Points show the position coordinates $(x_{1},x_{3})$ , faint curves trace selected trajectories, and stars mark the target positions.

Figure 7.4: Noising and denoising for the inverted pendulum. Top: stochastic noising initialized at the upright equilibrium. Bottom: deterministic feedback using the learned horizontal score. Rods show orientations, arrows represent $\omega$ . The star marks the upright target.

Figure 8.1: Nonholonomic Moser transport for the Brockett integrator using linear density interpolation.

Figure 8.2: Schematic of Perron-Frobenius argument. The target ball in red is kept fixed. The parameter $\eta$ controls the volume of points transported to the ball. Dark blue shows the assigned endpoint inside ball. Light blue shows the assigned endpoint outside ball Target ball.

Feedback Control

Feedback control uses ongoing measurements to adjust actions, stabilize dynamical systems, track goals, and respond to disturbances and constraints.

See all 255 papers

May 14, 2026

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

This work reformulates language generation as a stochastic optimal control problem, providing a unified theoretical perspective to analyze autoregressive and diffusion models and explain their limitations (Efficiency-Fidelity Paradox, Irreversibility Error Propagation, Optimization Tractability and Fidelity) in terms of combination of trajectory singularity, adjoint state vanishing, and gradient absence. To address these issues, we approximate the solution to the Hamilton-Jacobi-Bellman (HJB) equation, yielding an optimal policy that acts as a closed-loop controller. To bypass the intractability of directly solving the HJB PDE, we employ Flow Matching as the optimal trajectory solver within the rectified latent control space. This allows our Manta-LM with Global Integral Operator to approximate the global vector field, effectively realizing a model that simultaneously achieves high-fidelity text generation and efficient, low-cost parallel sampling. Empirically, our method achieves strong performance on language modeling and conditional generation tasks, while exhibiting improved stability, efficiency, and controllability.

ZiYi Dong, Yuliang Huang, Weijian Deng +3

Sun Yat-sen University, Guangzhou, Guangdong, China · Australian National University, Canberra, ACT, Australia · Tsinghua University, Beijing, China

Language Model-Based Control Stochastic Optimal Control Diffusion Sampling Diffusion Language Models Optimal Control

Ask PDF

Jun 19, 2026

BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling

Robots must generate trajectories that remain faithful to learned expert behavior while satisfying safety constraints and task-specific objectives specified only at inference time. We formulate constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling, with the learned demonstration distribution as a prior and an inference-time, cost-derived likelihood tilting it toward feasible, optimal trajectories. To sample from this posterior without any retraining of the base policy, we leverage the Feynman--Kac corrector framework, originally formulated for diffusion models, and extend it to deterministic flow-matching policies. The result is a unified, inference-time, retraining-free sampler for diffusion and flow policies. We validate the approach on pretrained Diffusion Policy, GR00T-N1.6, and

π_{0.5}

checkpoints across simulated and real-world manipulation tasks, including planning around non-convex obstacles introduced at inference time, and show improvements over the base

π_{0.5}

on zero-shot tasks.

Sreevardhan Sirigiri, Weiming Zhi, Fabio Ramos

School of Computer Science, The University of Sydney, Australia · Australian Centre For Robotics, The University of Sydney, Australia · College of Connected Computing, Vanderbilt University, TN, USA. +1

Diffusion Policy Trajectory Generation Flow-Based Generative Modeling Diffusion Posterior Sampling Robot Policy Learning Constrained Sampling+1

Ask PDF

Oct 3, 2026

LyapuFlow: Controlling Generative Flows with Lyapunov Feedback for Inverse Problems

Pretrained flow models are now widely used as generative priors in science and vision, where inference-time guidance enables test-time constraints without retraining. Existing methods use projection, posterior sampling, or iterative optimization of the generative trajectory. We propose LyapuFlow, an alternative based on Lyapunov feedback control. At each sampling step, LyapuFlow predicts the terminal sample towards which the current flow is evolving, and evaluates the constraint violation on this prediction. Then, we compute the minimum-norm control that satisfies a prescribed Lyapunov decrease condition. The resulting control remains inactive when the uncontrolled dynamics already reduce the constraint violation at the prescribed rate. Otherwise, it provides a corrective update within a feedback trust region that prevents the control from dominating the pretrained dynamics. We demonstrate LyapuFlow in both data and latent spaces, outperforming alternatives spanning different mechanisms for test-time constraint enforcement in scientific machine learning and image inverse problems.

Minseon Gwak, Hans Hao-Hsun Hsu, Danielle C. Maddix +1

Lawrence Berkeley National Lab · International Computer Science Institute · Georgia Institute of Technology +1

Constrained Generative Modeling Inference-Time Guidance Inverse Problems Flow-Based Generative Modeling Lyapunov-Based Control

Ask PDF

Abstract

Feedback Control

Explore similar work

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling

LyapuFlow: Controlling Generative Flows with Lyapunov Feedback for Inverse Problems