Interpreting Control Latents for System Identification via Conditional Flow Matching

Dingqi Zhang; Ruiqi Zhang; Mark W. Mueller

cs.ROAug 24, 2026

Interpreting Control Latents for System Identification via Conditional Flow Matching

Authors: Dingqi Zhang, Ruiqi Zhang, Mark W. Mueller

Organizations: High Performance Robotics Lab, Dept. of Mechanical Engineering, UC Berkeley

Abstract

Latent-conditioned adaptive policies can control robots across changing dynamics, but their learned latents remain internal representations of the policy rather than physical models that can be inspected, rolled out, or used by other control modules. This limits closed-loop analysis, diagnosis, and further improvement of a fixed policy. A direct mapping from latent to physical parameters is also under-specified, because multiple systems can induce similar closed-loop behavior. We therefore decode each operational latent into a distribution of quadrotor models using conditional flow matching. The decoded distribution enables two downstream uses without modifying the policy: online predictive tuning of a high-level controller around the fixed low-level policy, and robustness analysis under specified disturbances. Under perturbed actuator dynamics, decoded-model predictive tuning reduces position tracking RMSE by $23\%$ and heading RMSE by $45\%$ relative to fixed gains. Under Gaussian force disturbances, decoded-model ensembles closely predict the lateral tracking-error evolution. Together, these results show that control latents can be converted into physical model ensembles for tuning, robustness analysis, and diagnosis of frozen adaptive policies.

Figures & tables

Fig. 1: Overview of the proposed interpretation framework. Observation history is encoded into a compact latent, which a conditional flow-matching decoder then maps to a distribution of physically consistent system models (distributional system identification). Each decoded model is interpretable and approximates the true system’s closed-loop behavior under the deployed policy: they are run in parallel simulations (parametrized by the decoded models) and predictive rollouts are compared against real responses, enabling downstream tasks such as controller tuning and robustness analysis without modifying the policy.

Fig. 2: Overview of the training pipeline. (a) Unified training aligns the privileged latent with the history-based latent so the policy can adapt across randomized dynamics. (b) Conditional flow matching decodes the latent into a distribution of dynamically consistent physical models.

Fig. 3: Quadrotor convention used in this work. The body-fixed reference frame consists of the orthogonal basis $\bm{x}_{B},\bm{y}_{B},\bm{z}_{B}$ . Four motor-propeller pairs are arranged symmetrically about the vehicle center of mass. Motors 1 and 3 rotate in the opposite direction from motors 2 and 4. The nominal hardware parameters, measured directly from the vehicle and its components, are listed in Table I .

Parameter	Value	Unit
$m$	$0.925$	$\mathrm{kg}$
$l$	$0.184$	$\mathrm{m}$
$\bm{J}$	$\mathrm{diag}(8.5,7.5,15.5)$	$\mathrm{g\cdot m^{2}}$
$\bm{D}$	$\mathrm{diag}(0.20,0.20,0.85)$	$\mathrm{kg/s}$
$k_{t}$	$7.64e^{-6}$	$\mathrm{N/(rad/s)^{2}}$
$k_{\tau}$	$1.07e^{-7}$	$\mathrm{N\cdot m/(rad/s)^{2}}$

TABLE I: Quadcopter Parameters

Metric	Naive	Baseline	CFM-tuned
Pos. RMSE (m)	$0.330\pm 0.147$	$\mathbf{0.198\pm 0.103}$	$0.218\pm 0.102$
Max Pos. Err. (m)	$0.624\pm 0.291$	$\mathbf{0.470\pm 0.298}$	$0.510\pm 0.268$
Heading RMSE (deg)	$\mathbf{2.676\pm 1.883}$	$4.696\pm 2.724$	$4.894\pm 3.368$

TABLE II: Predictive high-level controller tuning over

100

online random trajectories with randomized quadrotor dynamics. Values are mean

\pm

standard deviation across rollouts.

Metric	Baseline	Nominal +CEM	CFM-tuned
Pos. RMSE (m)	$0.705\pm 1.256$	$0.584\pm 0.725$	$\mathbf{0.546\pm 0.560}$
Max Pos. Err. (m)	$1.671\pm 3.144$	$1.332\pm 2.078$	$\mathbf{1.155\pm 1.363}$
Heading RMSE (deg)	$14.251\pm 15.581$	$13.807\pm 13.441$	$\mathbf{7.853\pm 6.581}$

TABLE III: Predictive tuning with increased motor response time constants. Values are mean

\pm

standard deviation over

100

online random trajectories.

Speed	Ctrl.	Pos. RMSE (m)	Max pos. err. (m)	Heading RMSE (deg)
Slow	Baseline	$0.046\pm 0.012$	$0.079$	$\mathbf{1.197\pm 0.177}$
Slow	CFM-tuned	$\mathbf{0.025\pm 0.009}$	$\mathbf{0.048}$	$1.303\pm 0.302$
Medium	Baseline	$0.060\pm 0.015$	$0.090$	$1.316\pm 0.291$
Medium	CFM-tuned	$\mathbf{0.038\pm 0.010}$	$\mathbf{0.062}$	$\mathbf{1.258\pm 0.268}$
Fast	Baseline	$\mathbf{0.110\pm 0.045}$	$0.252$	$\mathbf{1.354\pm 0.341}$
Fast	CFM-tuned	$0.129\pm 0.035$	$\mathbf{0.187}$	$1.386\pm 0.365$

TABLE IV: Hardware gain tuning for 3D lemniscate tracking at slow, medium, and fast reference speeds (three complete cycles each). Position and heading columns report RMSE

\pm

std. of per-sample error; max position error is the peak over the window. Bold marks the lower value for each speed and metric.

Fig. 4: Hardware tracking of the 3D lemniscate at medium reference speed. CFM-tuned achieves tracking comparable to Baseline while reducing the spread of the position error, consistent with Table IV .

Fig. 5: Single-trajectory response to deterministic constant force steps on the nominal hardware-scale system for horizontal force $[1,1,0]\,\mathrm{N}$ and vertical force $[0,0,1]\,\mathrm{N}$ . The top row reports lateral error $\sqrt{\Delta X^{2}+\Delta Y^{2}}$ and the bottom row reports vertical error $|\Delta Z|$ . In each panel, we compare the true closed-loop rollout, decoded-model rollouts sampled from the latent posterior, and a true-system uncertainty baseline (uniform $\pm 20\%$ parameter perturbation) under matched initial conditions and matched disturbance inputs; shaded regions indicate the 5–95% confidence interval. The decoded-model envelope brackets the true lateral response at the correct scale and is far tighter than the $\pm 20\%$ baseline, while the vertical response is less well captured, localizing the decoder’s reliability to the lateral channel.

Fig. 6: Temporal tracking-error envelopes under i.i.d. Gaussian force disturbances with per-axis standard deviation $\sigma=1\,\mathrm{N}$ , $3\,\mathrm{N}$ , and $5\,\mathrm{N}$ . The top row reports lateral error $\sqrt{\Delta X^{2}+\Delta Y^{2}}$ and the bottom row reports vertical error $|\Delta Z|$ ; shaded regions indicate the 5–95% confidence interval. The decoded envelopes track the growth of the true lateral error as $\sigma$ increases, providing a calibrated robustness envelope in the lateral channel.

Fig. 7: Final tracking-error distributions at the end of the Gaussian disturbance rollouts. Columns correspond to $\sigma=1\,\mathrm{N}$ , $3\,\mathrm{N}$ , and $5\,\mathrm{N}$ ; rows show lateral and vertical error. The decoded final-error distributions closely match the true lateral distribution across all $\sigma$ , while overestimating the vertical spread, consistent with the temporal envelopes.

Fig. 8: Lemniscate tracking and open-loop action prediction on hardware. The quadrotor completes one lemniscate cycle in $5\,\mathrm{s}$ with position tracking RMSE $0.121\,\mathrm{m}$ . The right panels show the latent evolution and normalized motor commands from the neural policy, the CFM-instantiated parametric controller, and the regression baseline. The policy runs at $500\,\mathrm{Hz}$ , while CFM and regression decode the latent at $10\,\mathrm{Hz}$ . The CFM-decoded controller reproduces the policy’s motor commands in both magnitude and temporal pattern, whereas the regression baseline oscillates around them, indicating that the decoded parameters capture the policy’s latent-to-action mapping even under aggressive flight.

Task	Method	RMSE $\downarrow$	PCC $\uparrow$
Tracking ( $5\,\mathrm{s}$ )	Random	0.141	0.202
	Regression	0.059	0.077
	CFM	0.032	0.723
Tracking ( $10\,\mathrm{s}$ )	Random	0.151	0.080
	Regression	0.064	0.145
	CFM	0.034	0.710

TABLE V: Open-loop action prediction performance on hardware rollouts. RMSE is computed on raw normalized motor commands; PCC is computed after low-pass filtering the action histories. Arrows indicate the preferred direction:

\downarrow

lower is better (RMSE),

\uparrow

higher is better (PCC). Best value per row in bold.

Fig. 9: Off-center payload experiment. A $100\,\mathrm{g}$ payload is attached during hover at position $[0.1,\,0.0,\,0.0]\,\mathrm{m}$ in the body frame; the shaded interval marks the attachment event. The upper panel shows latent evolution, and the lower panels compare normalized motor commands from the neural policy, the CFM-instantiated parametric controller, and the regression baseline. When the payload attaches, the latent shifts abruptly and the CFM-decoded commands continue to match the policy through the transient while the regression baseline does not, showing that the decoder follows a sudden physical change and stays consistent with the policy.

System Identification

System identification infers mathematical models and parameters of dynamical systems from input-output, trajectory, sensor, or video data.

See all 157 papers

What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response
Sep 26, 2026Enzo Nicolás Spotorno, Josafat Leal Filho, Antônio Augusto FröhlichLatent Dynamics Modeling State Representation Learning
Department of Informatics and Statistics Federal University of Santa Catarina
Modeling Latent Disturbances for Robust Decision-Making in World Models
Oct 6, 2026Junwon Seo, Andrea BajcsyRobot Policy Learning World Model-Based Planning
The authors are with the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
End-to-end QP-based policies: A unified perspective on robust control and robot learning
Sep 25, 2026Fausto Vega, Priyanka Supraja Balaji, Chase Dunaway +3Robot Policy Learning Feedback Control
Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Mechanical Engineering, California Institute of Technology, Pasadena, CA, USA

Sep 26, 2026J/K move · Enter open · S save

What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response

Enzo Nicolás Spotorno, Josafat Leal Filho, Antônio Augusto Fröhlich

Department of Informatics and Statistics Federal University of Santa Catarina

Latent Dynamics Modeling State Representation Learning World Models for Autonomous Driving Joint-Embedding Predictive Architecture

Open paper AskPDF arXiv

Models of vehicle dynamics learned from logged states and commands complement physics-based models, and latent world models, which predict in a learned representation, are used to plan and train controllers in other domains. Vehicle controllers are usually specified in physical terms: costs, limits, and references depend on position, yaw angle, speed, and yaw rate, and the optimizer compares or differentiates predicted outcomes across nearby commands. A latent model placed in such a controller must therefore let these quantities be recovered and must change its predictions with commands as the vehicle does, and prediction error on its own latent targets measures neither. We contribute a measurement protocol for action-conditioned latent predictors with a physical readout that separately tests retention, physical-neighborhood organization, forecasting, and local response to command perturbations, using an untrained-encoder reference and three matched response paths that locate errors in the representation or the predictor. In a case study of a temporal joint-embedding predictive model trained on signals logged in IPG CarMaker, the representations retain the measured planar outputs, though an untrained encoder of the same architecture retains them slightly better; future-command input improves one-second forecasts with retention nearly unchanged; and responses to small command pulses diverge from the simulator already in latent coordinates, raising regret when choosing among nearby commands in all comparisons. Updating the predictor on responses corrects them locally at a cost in forecast accuracy. Measuring retention, forecasting, and local response separately is thus what qualifies a predictive latent as a candidate model for control, and the protocol provides the basis for its closed-loop evaluation.

Abstract

System Identification

Explore similar work

What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response

Modeling Latent Disturbances for Robust Decision-Making in World Models

End-to-end QP-based policies: A unified perspective on robust control and robot learning