Deep Learning Warm Starts for Trajectory Optimization on the International Space Station
Authors: Somrita Banerjee, Abhishek Cauligi, Marco Pavone
Organizations: Somrita Banerjee is a Machine Learning Researcher at Apple. Work completed during PhD at Stanford University, Stanford, CA 94305, USA. · Abhishek Cauligi is an Assistant Professor in the Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA. · Marco Pavone is an Associate Professor in the Department of Aeronautics and Astronautics, Stanford University, Stanford, CA 94305, USA.
Trajectory optimization is a cornerstone of modern robot autonomy, enabling systems to compute trajectories and controls in real-time while respecting safety and physical constraints. However, it has seen limited usage in spaceflight applications due to its heavy computational demands that exceed the capability of most flight computers. In this work, we provide results on the first in-space demonstration of using machine learning-based warm starts for accelerating trajectory optimization for the Astrobee free-flying robot onboard the International Space Station (ISS). We formulate a data-driven optimal control approach that trains a neural network to learn the structure of the trajectory generation problem being solved using sequential convex programming (SCP). Onboard, this trained neural network predicts solutions for the trajectory generation problem and relies on using the SCP solver to enforce safety constraints for the system. Our trained network reduces the number of solver iterations required for convergence in cases including rotational dynamics by 60% and in cases with obstacles drawn from the training distribution of the warm start model by 50%. This work represents a significant milestone in the use of learning-based control for spaceflight applications and a stepping stone for future advances in the use of machine learning for autonomous guidance, navigation, & control.
Figures & tables
Fig. 2 : Overview of our learning-based warm-starting approach for accelerating trajectory optimization. The baseline method uses Sequential Convex Programming (SCP) initialized with a cold start (i.e., a straight-line trajectory). In the offline phase, a neural network is trained on solved planning problems to predict effective warm starts. At deployment (online), the learned warm start is used to initialize SCP, resulting in faster convergence to an optimal trajectory.
Fig. 3 : Testing phases for software on Astrobee. Left: Ground testing on a 2D air-bearing granite table at NASA Ames to validate software on real hardware in a controlled planar environment. Right: On-orbit flight testing aboard the ISS, where Astrobee executed autonomous trajectories in a 6-DOF microgravity environment.
Method
Optimal cost ↓
# Iterations to converge (run time in seconds) ↓
TransOnly
Trans+RotOnly
Obs_Seen
Obs_OOD
TransOnly
Trans+RotOnly
Obs_Seen
Obs_OOD
SCP cold
0.6311
0.0025
0.5700
0.5844
2096 (8.3 s)
634 (2.6 s)
4182 (14.3 s)
2855 (11.2 s)
SCP warm
0.6326
0.0024
0.3902
0.6529
2357 (8.9 s)
254 (1.1 s)
2106 (7.0 s)
3065 (12.7 s)
TABLE I : Comparison of cold vs. warm start for average optimal cost and iterations / run time required to converge. Columns are trajectory categories: translation only, translation+rotation, seen (in-distribution) obstacles, and unseen (out-of-distribution) obstacles.
Fig. 4 : Iterations to convergence for cold and warm starts across trajectory categories . Horizontal bars indicate mean iterations required for planner convergence under cold and warm start conditions, with error bars showing one standard deviation. Warm starts perform similar to cold starts for translation-only trajectories. Warm starts require fewer iterations than cold starts for trajectories with rotation and seen obstacles, i.e., non-convexities learned by the model. However, the model did not generalize to unseen obstacles.
Fig. 5 : Examples of trajectories tested. Top row shows Astrobee execution and bottom row shows the plotted data. Both cold and warm starts result in the same optimal trajectory. Warm starts are qualitatively closer to the optimal trajectory, especially for trajectories with obstacle avoidance and rotations.
Fig. 6 : Comparison of convergence iterations for cold and warm starts . Each pair of connected points represents a matched run, i.e., a cold and warm start for the same start and goal states and obstacle constraints. The plot shows a trend towards reduction in iterations required for convergence when using a warm start. Black markers indicate group means, with vertical bars showing 95% confidence intervals computed using the standard error of the mean.
Trajectory optimization computes motion or control sequences that minimize costs while satisfying system dynamics, safety limits, and task constraints.
Real-time trajectory generation for on-orbit robotic servicing is challenging due to the nonlinear coupling between spacecraft bus motion, manipulator dynamics, visibility cone, and trajectory-level safety constraints. This paper studies learning-based warm-starting for sequential convex programming (SCP) in the terminal approach of a space manipulator toward a tumbling target. The proposed framework decomposes the problem into a system center-of-mass translational planning stage and a coupled attitude--manipulator torque-allocation stage, and applies a causal transformer warm-start to the latter, which constitutes the dominant computational bottleneck. Linear and flow matching action decoders are compared under different action-chunking and training dataset sizes, and the resulting warm-starts are evaluated under both cost-optimal and feasibility projection using SCP. Across 300 held-out scenarios, the learned warm-start reduces the second-stage SCP iteration count by up to 28% and the runtime by 23% while preserving the final control-cost distribution. When the learned warm-starts are used for nonconvex feasibility projection, they nearly halve the runtime relative to cost-optimal SCP, while avoiding the catastrophic high-cost tail behavior observed when initialized heuristically. These results indicate that sequence-model warm-starts can improve both the computational efficiency and trajectory robustness of optimization-based terminal guidance for space manipulation.
Yuji Takubo, Maximilian Adang, Mac Schwager +1
Department of Aeronautics and Astronautics, Stanford University. Stanford, CA, 94305.
Deep space missions increasingly require spacecraft capable of autonomously operating in environments with communication delays and bandwidth limitations. A key enabler of such autonomy is the ability to run learning-based algorithms directly onboard. In this work, we present a flight deployment on a 6U CubeSat that demonstrates (1) onboard training and inference of a thermal segmentation model on an NVIDIA Jetson TX2 NX GPU in space and (2) closed-loop attitude control using the segmentation model on a real-time, thermal image feed to track the Moon. In addition to this demonstration, we implement ROS 2 Foxy as a data transport middleware and integrate an open-source file system that provides strong data integrity guarantees and self-healing features. Over the course of approximately one year, we performed 12 experiments on the spacecraft, advancing the state of the art in autonomous spacecraft systems by showing how modern learning-based pipelines, commercially available hardware, and open-source software can operate reliably under the compute and environmental constraints of Low Earth Orbit (LEO).
Elena Sorina Lupu, Hannah Grauer, Phaedrus Leeds +9
California Institute of Technology, Pasadena, CA 91125, USA · The Aerospace Corporation, 2310 E. El Segundo Blvd., El Segundo, CA 90245-4609, USA
Trajectory optimization is a critical component for enabling safe and reliable autonomous operations in space exploration. As space missions increase in frequency, complexity, and scope, there is a growing need to rapidly formulate mathematically sound trajectory optimization problems that accurately reflect mission objectives and operational constraints. However, translating mission intent into tractable analytical formulations for trajectory optimization requires substantial domain expertise. This paper presents a framework that leverages large language models (LLMs) to translate natural language descriptions of mission requirements and constraints into executable trajectory optimization code and corresponding mathematical formulations. Experiments in spacecraft rendezvous scenarios demonstrate a high success rate in reconditioning a convex trajectory optimization problem from semantic mission requirements. Ultimately, this work highlights the potential of LLMs to bridge high-level intent and formal optimization models, enabling more flexible and efficient trajectory design of spacecraft.
Eleanor Brosius, Yuji Takubo, Daniele Gammelli +2
Stanford University 496 Lomita Mall, Stanford, CA, 94305