Learning Stabilizing Latent Feedback Controllers via Multi-Step Backward Losses
Authors: Paul Lutkus, Lars Lindemann, Stephen Tu
Organizations: Thomas Lord Department of Computer Science, University of Southern California · Automatic Control Laboratory, ETH Zürich · Ming-Hsieh Department of Electrical and Computer Engineering, University of Southern California
Can learned latent dynamics models support the design of feedback controllers that provably stabilize the original system? For linear time-invariant systems, we show that learning linear autoencoders and latent dynamics with multi-step prediction-reconstruction losses \unicodex2014 which we term "backward losses" as they encourage latent trajectories to reconstruct full state trajectories \unicodex2014 provably recovers the structure needed for stabilization. Specifically, a sufficiently long reconstruction horizon ensures that a minimum H∞-norm controller designed using the learned latent model stabilizes the original system through output-feedback via the corresponding encoder, provided the latent dimension is large enough to capture all non-stable modes of the underlying system. We also show why both reconstruction and a multi-step horizon are essential: forward prediction losses that require only low prediction error along latent trajectories admit global loss minimizers that discard unstable modes regardless of horizon length, while single-step reconstruction losses can yield latent models whose stabilizing controllers fail to stabilize the original system. Finally, experiments on a non-linear flexible-arm system illustrate the practical implications of our results for learning non-linear autoencoders and latent dynamics.
Figures & tables
Figure 1: Stabilization of a flexible arm using linear latent output feedback. Only Lbwd , the multi-step prediction-reconstruction loss, leads to a stabilizing latent output-feedback controller. In (A), (B), the light gray arm denotes the shared initial state, and opacity denotes the position of the arm in time (darker denotes later). In (B), the × denotes the arm impacting the ground. Panels (C) and (D) are linearizations about the upright equilibrium.
Figure 2: For the single-step backward loss Lbwd(1) , the set of stabilizing gains for the learned latent system separates from the set of output-feedback stabilizing gains for the LTI system ( 5.10 ) as non-normality grows. In the right panel, the spectral radius ρ(Aη+BKEˉ1) of the resulting output-feedback closed-loop matrix is shown.
Figure 3: For the multi-step backward loss Lbwd(T) , the set of stabilizing gains for the learned latent system converges to the set of output-feedback stabilizing gains for LTI system ( 5.10 ) as T grows (here the non-normality parameter η is held fixed). In the right panel, the spectral radius ρ(Aη+BKEˉT) of the resulting output-feedback closed-loop matrix is shown, indicating stable closed-loop behavior for T⩾2 .
Figure 4: Local geometry of the flexible arm described in ( 6.1 ). Section i , of length ℓ , connects joints i and i+1 , and the gravitation force mˉg acts at its center of mass. The solid dots indicate torsional springs of stiffness ki:=κ⋅(K0)ii . The torque τi combines the input, spring, and damping contributions at joint i , with Cd:=c0M+c1Keff .
Figure 5: Linearized flexible arm controlled by linear latent models ( m=4 ). Panels (a, b) show results from training on the population losses Lbwd(T) (blue, λreg=1 ) and Lfwd(T) (orange, most favorable λreg ). (a) Subspace distance Δ of the minimizer with the lowest objective among 8 random initializations (error bars span the 25 %- 75 % interquartile range over the 8 initializations). (b) Number of the 8 initializations whose latent H∞ gain K makes A+BKE Schur stable. (c) The ratio ∥xt∥/∥x0∥ for both the closed-loop of the latent LQR gain (cost J ) learned from Lbwd(40) with Rorth , and of full state LQR (cost J⋆ ). The lines and shaded area illustrate the median and 25 %- 75 % interquartile range over 32 initial states x0∼N(0,In) . The panels (d, e) are as (a, b), but for the empirical losses of N trajectories at T=40 . The black dash-dotted line in (b, e) signifies Algorithm 1 with known (A,B) , which stabilizes at every T and N . Note that series are slightly offset horizontally for legibility.
Table 1: A comparison of stabilization performance for latent H∞ versus latent LQR control design, for both population losses (top, with T varying) and finite-sample losses (bottom, with T=40 fixed, N varying). The subspace distance Δ shown is for the lowest-objective model over 8 initialization (same as Figure 5 (a)) for the population loss, and the median over 5 datasets (each the better of 2 initializations) for the finite-sample loss. Metrics for Lfwd(T) and Lfwd(T) are shown for the value of λreg that produces the largest number of stabilizing models over all T .
Figure 6: MLP (green) and linear (blue) latent models trained on data from the non-linear arm. Solid lines are medians, and error bars are the 25% to 75% interquartile range. All models are trained using the empirical backwards objective Lbwd(T) with latent dimension m=4 (with the modifications to Jfwd(1) discussed in the beginning of Section 6 ). Panels (a)–(d) compare the linearization of the MLP model about the unstable equilibrium q=0 to the linear model. In (a), we compare the subspace distance of row(JE(0)) , where JE(x):=dxdE(x) . Panels (b) and (c) compare relative errors of the unstable eigenvalue of Jaz(0) and induced input gain, against L0 and H0 (shown only for models that capture the unstable mode, i.e. Δ<0.5 ). For panel (c), let w be the left eigenvector of Jaz(0) for the unstable eigenvalue, scaled so that wTJE(0)D0=1 . Thus, h^:=wTgz(0) is invariant to the learned latent representation, and equals H0 when the unstable mode is recovered exactly (see Appendix I for more details). Panel (d) displays ρ(A+BKJE(0)) for the latent LQR gain K , and Algorithm 1 with known (A,B) (dash-dotted). At T=1 , the linear model is degenerate ( E≈0 ) and has no stabilizing latent LQR gain. MLP values for T⩽7 lie above the axis. For the models selected at T=40 , (e) displays for each data set (dot) the number of the 8 initial states from which its controller stabilizes the non-linear arm, under u=KE(x) (filled) and u=KJE(0)x (hollow; the same law for the linear model). Data sets with the same count are stacked vertically. Each initial state releases the arm from rest with its tip displaced along the buckling mode (random sign), plus a random bend that leaves the tip in place. The same 8 initial states are scaled to each tip displacement.
Figure 7: Training on empirical Lbwd(40) with three unstable modes, from the standard initialization (random orthogonal L , grey) and from L=ρ^Im held for the first 500 Adam steps (blue), for each of the 16 random initializations of one data set. The dashed line marks the switch from Adam to L-BFGS. (a) Objective relative to the lowest one found, f⋆ . (b) Real parts of the latent eigenvalues (dotted lines are σ+(A)={1.15,1.12,1.09} ). (c) ∥Prow(E)W+∥F2 , where the columns of W+ are the unit left eigenvectors of the unstable modes (equals 3 when all three modes are captured).
initialization
T=10
T=20
T=30
T=40
random orthogonal L
0/80
0/80
0/80
0/80
L=1.2Im
10/80
71/80
78/80
72/80
L=1.2Im , held for 500 steps
80/80
80/80
80/80
80/80
L=ρ^Im , held for 500 steps
80/80
80/80
80/80
80/80
Table 2: Flexible arm with three unstable modes ( m=m+=3 ): We count runs (over 5 data sets with 16 random initializations each) that capture all three modes i.e., Δ(G+R(AT);row(E))<0.5 (in every run, Δ⩽0.19 or Δ⩾0.82 ).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5
Figure 6
Figure 7 , Table 2
population
finite sample
plant
linearized arm
linearized arm
non-linear arm, coordinates x/ε
three-mode linearized arm, mass coordinates
data
none (exact expectations)
5 data sets, N∈{5,20,100,500,1000} , nested
5 data sets, N=2000 , ε=0.01 ( 0.03 and 0.1 in Table 5 )
5 data sets, N=2000
initializations
8
2 per data set and N
MLP: 8 per data set, one continued; linear: 8 per data set
16 per data set and strategy, all counted
Appendix
Table 3: Protocols of the experiments of Section 6 . The data are regenerated at every horizon with the same initial states.
loss
regularizer
N
λmin
rA
ru
rh
H∞ , non-linear arm, tip [cm]
2
10
20
30
40
60
80
Lbwd(T)
Rorth
pop.
1
0.034
2.2⋅10−4
5.5⋅10−5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
100
1
0.057
2.3⋅10−4
6.8⋅10−5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
500
1
0.039
2.3⋅10−4
6.8⋅10−5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1000
1
0.035
2.2⋅10−4
6.2⋅10−5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
Rvr , N(0,I)
pop.
0.99
0.034
2.2⋅10−4
8.8⋅10−5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
Appendix
Table 4: Linear models at T=40 ( λreg as in Figure 5 ): for the population losses (pop.), the lowest-objective initialization; for the empirical losses, the reported models of the 5 data sets with N trajectories ( Δ and stabilization of A+BKE : Table 1 ). λmin is the smallest eigenvalue of EET , and rA:=∥E^A−L^E^∥F , where (E^,L^,H^):=(S−1E,S−1LS,S−1H) with S:=(EET)1/2 is the same model with orthonormal encoder rows. ru:=∥eTA−λ^ueT∥ and rh:=∣eTB−wTH∣ , where λ^u is the eigenvalue of L nearest λu=1.129 , w is its left eigenvector, and e:=ETw , scaled so that ∥e∥=1 ; both residuals are invariant under any invertible change of latent coordinates. We report ru and rh for models with Δ<0.5 . For the empirical losses, λmin , rA , ru and rh are medians over the data sets. Last columns: fraction of the 8 initial states (averaged over the data sets) from which the H∞ gain stabilizes the non-linear arm, per tip displacement.
ε
T
dev. [%]
fallen [%]
model
captured
Δ
λmin
rA
ru
rh
stable
0.01
10
0.17
0
linear
0/5
1
1
0.026
–
–
0/5
MLP
4/5
0.094
0.99
2
0.11
0.012
4/5
20
2.3
0
linear
5/5
0.015
1
0.037
5.9⋅10−3
1.1⋅10−3
5/5
MLP
5/5
0.019
0.99
2
0.059
2.9⋅10−3
5/5
40
54
47
linear
5/5
0.013
1.4⋅10−4
4.6
0.17
0.07
1/5
MLP
5/5
0.011
2.3⋅10−3
2.6
0.03
2.2⋅10−3
5/5
Appendix
Table 5: Non-linear arm data at scale ε , and the linearization (E,L,H):=(JE(0),Jaz(0),gz(0)) of the models selected at horizon T (for the linear model, its parameters). Data: deviation from linearized rollouts, and fraction of trajectories in which a section passes 90∘ . Models: data sets (of 5 ) that capture the unstable mode, medians over the data sets of Δ , λmin and rA , medians over the capturing data sets of ru and rh (all as in Table 4 ), and data sets whose latent LQR gain stabilizes A+BKE (stabilization of the non-linear arm: Table 6 ).
ε
T
controller
tip displacement [cm]
2
10
20
30
40
60
80
0.01
10
MLP, u=KE(x)
0.65
0.60
0.50
0.40
0.38
0.20
0.20
MLP, u=KJE(0)x
0.70
0.70
0.70
0.75
0.78
0.80
0.80
linear, u=KEx
0.00
0.00
0.00
0.00
0.00
0.00
0.00
20
MLP, u=KE(x)
1.00
1.00
1.00
1.00
0.78
0.25
0.25
MLP, u=KJE(0)x
1.00
1.00
1.00
1.00
1.00
1.00
1.00
Appendix
Table 6: Non-linear arm data at scale ε (cf. Table 5 ): fraction of the 8 initial states, averaged over the 5 data sets, from which the controllers of the models selected at horizon T stabilize the non-linear arm, per tip displacement (panel (e) of Figure 6 shows ε=0.01 , T=40 per data set).
The paper proposes a novel data-driven framework for designing and training a feedback linearizing controller by explicitly incorporating relative degree based conditions into the learning process. This enables the conventional feedback controller components to be replaced by neural Lie derivatives, thereby facilitating a fully data-driven feedback linearization framework. Furthermore, practical closed-loop stability is established by deriving sufficient conditions under which bounded identification errors lead to bounded tracking errors. The derived theoretical results are validated through their application to an armature controlled DC motor.
Lakshmi Priya P. K., Andreas Schwung
Department of Automation Technology and Learning Systems, South Westphalia University of Applied Sciences,Lübecker Ring 2, Soest, 59494, Germany
Learning Koopman operators with autoencoders enables linear prediction in a latent space, but long-horizon rollouts often drift off the learned manifold, leading to phase and amplitude errors on systems with switching, continuous spectra, or strong transients. We introduce two complementary components that make Koopman predictors more robust. First, we add an attention-free latent memory (AFT) block that aggregates a short window of past latents to produce a corrected latent before each Koopman update. Unlike multi-head attention, AFT operates in linear time and adds only ≈30k parameters (3d2+T2, fewer than matched multi-head attention), yet captures the local temporal context needed to suppress error divergence. Second, we propose dynamic re-encoding: lightweight, online change-point triggers (EWMA, CUSUM, and sequential two-sample tests) that detect latent drift and project predictions back onto the autoencoder manifold. Across three benchmark systems -- Duffing oscillator, Repressilator, IRMA -- our model consistently reduces error accumulation compared to a Koopman autoencoder and matched-capacity multi-head attention. We also compare against GRU and Transformer autoencoders, evaluated both from initial conditions and with a 50-step context, and find that Koopman+AFT (with optional re-encoding) attains markedly lower long-horizon error while maintaining lower inference latency. We report improvements over horizons up to 1000 steps, together with ablations over trigger policies. The result is a fast, compact predictor that stays on the learned manifold over long horizons.
Mohammed Nagdi, Evangelos-Marios Nikolados, Alexey Yermakov +3
Institute for Bioengineering, School of Engineering and Centre for Engineering Biology, University of Edinburgh, Edinburgh, United Kingdom · Electrical and Computer Engineering and Applied Mathematics, University of Washington, Seattle, WA, USA · Electrical and Computer Engineering and Computer Science & Engineering, University of Washington, Seattle, WA, USA
Robotic systems often exhibit unstable modes, along which small perturbations and disturbances can cause unbounded growth unless corrected through feedback. Controlling such systems from high-dimensional visual observations requires representations that preserve these modes. Joint-embedding predictive architectures (JEPAs) provide a natural framework for learning such representations and their dynamics from visual data. However, we demonstrate that next step prediction combined with anti-collapse regularization does not guarantee that controllable unstable modes are preserved: the training loss can be minimized while these modes are collapsed, making stabilization from the learned representation impossible. To address this, we augment world-model training with an action reconstruction objective (i.e., an inverse dynamics loss) that encourages control-aware representations, namely, visual representations that preserve crucial features for control. We prove that exact action reconstruction makes the encoder injective on the finite-horizon reachable subspace. Thus, the encoder cannot discard any state direction reachable by an action sequence within H steps. Moreover, we show that, as H grows, the dominant eigenspace of the finite-horizon controllability Gramian converges to the controllable unstable subspace. We establish our theoretical results for linear systems and demonstrate empirically that our findings extend to nonlinear visual control tasks (CartPole, Walker2D, and PointMaze), highlighting the benefits of control-aware representation learning.
Leonardo F. Toso, Yann LeCun, James Anderson +1
Columbia University · New York University, AMI Labs · New York University