Notes on Generative Modeling for Feedback Control and Planning

Karthik Elamvazhuthi

math.OCSep 27, 2026

Notes on Generative Modeling for Feedback Control and Planning

Authors: Karthik Elamvazhuthi

Abstract

In these notes, we view control as a dynamically constrained sampling problem on the state-space of a control system. With this viewpoint, we extend methods from generative modeling, such as flow matching, normalizing flows and denoising diffusions to control problems. Concepts such as controllability, optimal control and trajectory planning play an important role in guiding the extension and understanding well-posedeness of the corresponding algorithms, with application to steering systems to target states or distributions and sampling from reachable sets. The notes are intended as an accessible introduction for readers with a background in control theory and robotics.

Figures & tables

Figure 2.1: From a single trajectory to an evolving density. Left: A single solution

x(t)

\dot{x}=f(x)

from a known initial condition

x_{0}

. Center: An ensemble of trajectories from initial conditions drawn from

\rho_{0}

. Right: The probability density

\rho_{t}

at successive times; as trajectories compress, the density narrows and its peak grows.

Figure 2.2: Controlled transport from the left gyre-core distribution toward a point target near $(1.5,0.5)$ . The terminal density is concentrated in a Dirac target.

Figure 2.3: Controlled transport between the two gyre-core distributions.

Panel of Figure 3 — Figure 3.1: Steering the system $\dot{x}_{1}=u_{1}$ , $\dot{x}_{2}=u_{2}$ , $\dot{x}_{3}=x_{2}u_{1}$ toward the origin. Top: open-loop trajectories used to generate data for the flow matching algorithm. Bottom: closed-loop trajectories under the learned feedback. Each row shows the initial, intermediate, and terminal states from left to right. Blue points represent the current states, and the red star marks the target.

Figure 3.2: Noising and denoising for an underactuated system on $SO(3)$ , visualized through the first column $Re_{1}\in S^{2}$ . Top: noising from the identity under bounded Brownian-motion controls. Bottom: learned denoising from Haar measure. Each row progresses from left to right. Dots indicate current column vectors, curves show their paths, and the star marks $e_{1}$ .

Figure 3.3: RRT*-based noising and learned denoising. Top: tree growth at iterations $0$ , $600$ , $1200$ , and $2000$ . Bottom: learned return trajectories from fresh initial states sampled uniformly in free space, shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ . Curves show accumulated paths, dots mark current states, and the star marks the goal.

Figure 3.4: Dijkstra-based flow matching. Top: outward propagation of the shortest-path tree from the goal configuration at successive stages of the Dijkstra algorithm. Bottom: trajectories generated by the learned continuous feedback controller from fresh initial conditions sampled independently in the free space. Curves show accumulated trajectories, dots indicate the current states, and the star marks the goal.

Approach	Procedure
Classical optimal control	Fix initial and final conditions, then solve the PMP-based boundary value problem or the HJB equation.
Denoising-based optimal control	Fix the final condition, propagate states through the backward-reachable set using the PMP system, and denoise to synthesize a feedback controller.

Table 3.1: Classical and denoising-based routes to feedback synthesis.

Figure 3.5: PMP-based noising and learned denoising for the unicycle. Top: trajectories spreading from the goal under sampled adjoint conditions. Bottom: learned feedback trajectories from uniformly sampled initial configurations. Curves show paths, while arrows indicate the heading $x_{3}$ , and the star marks the target position.

Figure 4.1: Independent bounded control sampling for the Van der Pol oscillator with $\Delta t=0.2$ , $0.05$ , and $0.01$ . Blue curves show sampled trajectories, and blue dots mark their terminal states. The red curve shows the averaged dynamics with $u=0$ ; the star marks the common initial state.

Figure 4.2: White-noise scaling for the controlled Van der Pol oscillator. On each interval of length $\Delta t$ , the control is $u_{k}=\sigma\xi_{k}/\sqrt{\Delta t}$ , where $\xi_{k}\sim\mathcal{N}(0,1)$ are independent and $\sigma=2$ . Blue curves show sampled trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same horizon $T=6$ and axis limits.

Figure 4.3: Projected Brownian controls for the Van der Pol oscillator. The control is $u(t)=P_{[-4,4]}(2W_{t})$ , approximated by piecewise-constant values sampled at intervals of length $\Delta t$ . Blue curves show sampled state trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same Brownian realizations, horizon $T=6$ , and axis limits.

Figure 4.4: Random-switching controls for the Van der Pol oscillator. Switching times follow a Poisson process with rate $\lambda=2$ , and the initial control and each subsequent control value are sampled independently and uniformly from $[-4,4]$ . Blue curves show sampled state trajectories, and blue dots mark their terminal states. The red curve shows the zero-control dynamics; the star marks the initial state $(1.5,0)$ . All panels use the same switching times and control values, horizon $T=6$ , and axis limits. Here, $\Delta t$ specifies the time grid discretization.

Figure 4.5: Random-switching controls experiment repeated with $N=1000$ samples.

Figure 4.6: Kernel smoothing of an empirical measure. Left: point masses at the sampled terminal states. Center: Gaussian bumps centered at these states, each with integral $1/N$ . Right: the resulting density $\rho_{T,\delta}^{N}=k_{\delta}*\mu_{T}^{N}$ (red), obtained by summing the weighted bumps (blue).

Figure 4.7: Optimal-transport-based reachable-set sampling for the Van der Pol oscillator with $\mu=1$ and $|u|\leq 4$ . Panels show the states at $t/T=0$ , $0.3$ , $0.6$ , and $1$ . Blue points indicate current states, and faint blue curves show their accumulated paths. The red curve shows the zero-control trajectory up to the displayed time, and the star marks the common initial state $(1.5,0)$ . The terminal interaction cost promotes dispersion without requiring prior knowledge of the reachable set.

Figure 5.1: Continuous normalizing flow for the fully actuated planar system. Blue points show the evolving model samples, initialized from a Gaussian distribution, while faint red points show samples from the noisy two-moons target distribution. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with identical axis limits. At the terminal time, the learned flow approximately reproduces the two-moons distribution.

Figure 5.2: Controlled normalizing flow for a four-dimensional kinematic vehicle model. Blue points show current positions, and faint red points show samples from the two-moons target position distribution. Arrows indicate the heading of a subset of vehicles, rather than their velocity. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with $T=4$ and identical axis limits. Only the position coordinates are displayed

Figure 5.3: Entropy-maximizing controlled normalizing flow for the Van der Pol oscillator. Initial states are sampled uniformly from a disk of radius $0.25$ centered at $(1.5,0)$ . Blue points show the current states under the learned feedback $u^{\theta}\in[-4,4]$ , and blue curves show accumulated paths for a subset of particles. The red curve shows the zero-control trajectory from the initial center, marked by the star. Snapshots are shown at $t/T=0$ , $0.3$ , $0.6$ , and $1$ , with $T=6$ and identical axis limits.

Figure 6.1: Evolution of reflected diffusion particles toward a four-component Gaussian-mixture target on $[0,1]^{2}$ . Blue points represent the particle states, red contours represent the target density $\rho_{\infty}$ , and the crosses mark the centers of the four component. The particles are initialized near $(0.5,0.5)$ and separate into the four target modes over time.

Figure 6.2: Stochastic gradient flow for the unicycle model. Blue points represent the unicycle positions, and the arrows indicate the headings of a subset of the particles. Red contours show the desired four-component Gaussian-mixture density, while the crosses mark its component centers. The positions are initialized near $(0.5,0.5)$ with uniformly distributed headings.

Figure 6.3: Convergence to Gaussian stationary densities. Top: fully actuated damped single integrator. Bottom: damped oscillator with noise in its second coordinate. Blue points show current states, and faint blue curves trace selected particles. The star marks the equilibrium.

Figure 6.4: Unicycle horizontal-gradient descent with fixed potential strength $K=10$ and increasing diffusion strength. Rows correspond to $D=0$ , $0.1$ , $1$ , and $4$ ; columns show successive times. Blue points indicate positions, arrows show headings, and faint curves trace selected particles. For $D>0$ , red circles enclose $50\%$ and $90\%$ of the stationary position probability.

Figure 7.1: Diffusion-based feedback planning around circular obstacles. Top: forward noising from the target. Bottom: deterministic denoising under the learned score feedback, initialized uniformly in free space. Points show current states, and curves trace selected particles up to the displayed time. Gray disks represent obstacles, and the star marks the target.

Figure 7.2: Noising and denoising for the unicycle. Top: forward diffusion initialized at the target. Bottom: deterministic denoising using the score network selected by the lowest validation loss. Points show positions, arrows indicate headings, and faint curves trace selected trajectories. The star marks the target position; gray disks represent obstacles.

Figure 7.3: Noising and denoising for a four-dimensional controllable linear system. The plot shows eterministic denoising from Gaussian samples. Points show the position coordinates $(x_{1},x_{3})$ , faint curves trace selected trajectories, and stars mark the target positions.

Figure 7.4: Noising and denoising for the inverted pendulum. Top: stochastic noising initialized at the upright equilibrium. Bottom: deterministic feedback using the learned horizontal score. Rods show orientations, arrows represent $\omega$ . The star marks the upright target.

Figure 8.1: Nonholonomic Moser transport for the Brockett integrator using linear density interpolation.

Figure 8.2: Schematic of Perron-Frobenius argument. The target ball in red is kept fixed. The parameter $\eta$ controls the volume of points transported to the ball. Dark blue shows the assigned endpoint inside ball. Light blue shows the assigned endpoint outside ball Target ball.

Feedback Control

Feedback control uses ongoing measurements to adjust actions, stabilize dynamical systems, track goals, and respond to disturbances and constraints.

See all 255 papers

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
May 14, 2026ZiYi Dong, Yuliang Huang, Weijian Deng +3Language Model-Based Control Stochastic Optimal Control
Sun Yat-sen University, Guangzhou, Guangdong, China · Australian National University, Canberra, ACT, Australia · Tsinghua University, Beijing, China
BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling
Jun 19, 2026Sreevardhan Sirigiri, Weiming Zhi, Fabio RamosDiffusion Policy Trajectory Generation
School of Computer Science, The University of Sydney, Australia · Australian Centre For Robotics, The University of Sydney, Australia · College of Connected Computing, Vanderbilt University, TN, USA. +1
LyapuFlow: Controlling Generative Flows with Lyapunov Feedback for Inverse Problems
Oct 3, 2026Minseon Gwak, Hans Hao-Hsun Hsu, Danielle C. Maddix +1Constrained Generative Modeling Inference-Time Guidance
Lawrence Berkeley National Lab · International Computer Science Institute · Georgia Institute of Technology +1

May 14, 2026J/K move · Enter open · S save

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

ZiYi Dong, Yuliang Huang, Weijian Deng, Xiangyang Ji+2

Sun Yat-sen University, Guangzhou, Guangdong, China · Australian National University, Canberra, ACT, Australia · Tsinghua University, Beijing, China

Language Model-Based Control Stochastic Optimal Control Diffusion Sampling Diffusion Language Models Optimal Control

Open paper AskPDF arXiv

This work reformulates language generation as a stochastic optimal control problem, providing a unified theoretical perspective to analyze autoregressive and diffusion models and explain their limitations (Efficiency-Fidelity Paradox, Irreversibility Error Propagation, Optimization Tractability and Fidelity) in terms of combination of trajectory singularity, adjoint state vanishing, and gradient absence. To address these issues, we approximate the solution to the Hamilton-Jacobi-Bellman (HJB) equation, yielding an optimal policy that acts as a closed-loop controller. To bypass the intractability of directly solving the HJB PDE, we employ Flow Matching as the optimal trajectory solver within the rectified latent control space. This allows our Manta-LM with Global Integral Operator to approximate the global vector field, effectively realizing a model that simultaneously achieves high-fidelity text generation and efficient, low-cost parallel sampling. Empirically, our method achieves strong performance on language modeling and conditional generation tasks, while exhibiting improved stability, efficiency, and controllability.

Abstract

Feedback Control

Explore similar work

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling

LyapuFlow: Controlling Generative Flows with Lyapunov Feedback for Inverse Problems