Generalizable Output Feedback Optimal Control via Transformers Without Explicit State Estimation
Organizations: Department of Electrical & Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor, MI, USA · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
Abstract
Classical optimal control is used to design a separate controller for each plant. This paper asks a different question. Can one learned policy serve a whole family of partially observed, noisy linear systems? We show that one transformer can. The policy is trained to imitate optimal LQG control on a heterogeneous family of Linear Time-Invariant (LTI) systems with Multiple Inputs and Multiple Outputs (MIMO), whose dimensions, cost matrices, and noise levels all vary. A shared representation with system-wise standardization, zero-padding, masking, and an encoding of the cost and noise covariances lets one set of weights control every member of the family. At run time, a short window of noisy measurements is all the policy needs to produce a control action, with no plant matrices and no state estimator. We characterize the relation between the optimal LQG policy (represented as a dynamic state-space model) and a static transformer acting on a finite window of measurements. We then express the closed-loop excess cost of any causal policy as the energy of its deviation from the optimal action. Finally, we evaluate the approach on 28 simulated systems. The trained policy reaches the optimal cost to within a median gap of for seen systems. For unseen systems, a light fine-tuning step brings the median gap to .
Figures & tables
| Seen Systems: 1. Simple Pendulum , 2. Two Link Arm , 3. Spring Damper , 4. Suspension System , 5. DC Motor , 6. Three Link Manipulator , 7. Differential Drive Robot , 8. SCARA Robot , 9. Omnidirectional Robot , 10. Cable Driven Robot , 11. Flexible Joint Robot , 12. Six DOF Manipulator , 13. Dual Arm Robot , 14. Double Integrator , 15. Lotka Volterra . |
| Unseen Systems: 16. Inverted Pendulum , 17. Segway Robot , 18. Asymmetric Oscillator , 19. Active Mass Damper , 20. Coupled Oscillators , 21. Damped Oscillator , 22. Triple Mass Spring , 23. Electromechanical Actuator , 24. Thermal System , 25. Fluid Tank , 26. Vibrating Beam , 27. Motor Generator , 28. Mechanical Linkage . |