cs.ROOct 13, 2026
SaveLearning in Orbit: Onboard Training and Closed-Loop Attitude Control with a Thermal Segmentation Model on a Spacecraft
Organizations: California Institute of Technology, Pasadena, CA 91125, USA · The Aerospace Corporation, 2310 E. El Segundo Blvd., El Segundo, CA 90245-4609, USA
Abstract
Deep space missions increasingly require spacecraft capable of autonomously operating in environments with communication delays and bandwidth limitations. A key enabler of such autonomy is the ability to run learning-based algorithms directly onboard. In this work, we present a flight deployment on a 6U CubeSat that demonstrates (1) onboard training and inference of a thermal segmentation model on an NVIDIA Jetson TX2 NX GPU in space and (2) closed-loop attitude control using the segmentation model on a real-time, thermal image feed to track the Moon. In addition to this demonstration, we implement ROS 2 Foxy as a data transport middleware and integrate an open-source file system that provides strong data integrity guarantees and self-healing features. Over the course of approximately one year, we performed 12 experiments on the spacecraft, advancing the state of the art in autonomous spacecraft systems by showing how modern learning-based pipelines, commercially available hardware, and open-source software can operate reliably under the compute and environmental constraints of Low Earth Orbit (LEO).
Figures & tables
Figure 1: The spacecraft, alongside images captured in flight. At the top, three Moon images captured by the Boson thermal camera are shown. On the left, one image taken of the dark sky exhibiting streak artifacts (see Section 6-6.4 ). In the middle, we show an example of a thermal image acquired in orbit, with the Earth’s thermal emission and the Moon as a distinct thermal signature. The spacecraft is illustrated on the right.
| Mission | Platform | Launch Date | Hardware | Capability |
| Remote Agent (RA) on Deep Space 1 (DS1) [ 15 ] | Medium-sized interplanetary spacecraft | Oct. 1998 | RAD6000 [ 25 ] | “The first AI-based closed loop autonomous control system to take control of a spacecraft” [ 25 ] |
| NASA’s Earth Observing-1 (EO-1) [ 16 ] | Medium-sized Earth-observation satellite | Nov. 2000 | Mongoose-V [ 26 ] | First ML in space [ 27 ] (used support vector machine (SVM) and later hyperspectral unmixing [ 17 ] ) |
| CNES’s DEMETER [ 18 ] | SmallSat | June 2004 | Analog Devices ADSP 21020 DSP | “First onboard neural network dedicated to the scientific processing of wave data”, Time Delay Neural Network [ 19 ] |
| Intelligent Payload Experiment [ 20 ] | Cubesat | Dec. 2013 | Gumstix Earth Storm | ML random decision forests to classify images onboard and CV visual salience software to extract interesting regions for downlink in acquired imagery [ 28 ] |
| OPS-SAT [ 11 ] | CubeSat | Dec. 2019 | Altera Cyclone V SX SoC (FPGA) | CNN image classification, k-means, FDIR model training [ 11 ] . “First re-training of an onboard AI model with live in-flight data” |
| Phi-Sat-1 [ 12 ] | CubeSat | Sep. 2020 | Intel Movidius board with a Myriad II chip (VPU) | Inference of a CNN in space [ 12 ] |
Table 1: Flown missions with onboard machine learning or AI capabilities (based on publicly available information). Early missions without neural networks are included for historical context as pioneering AI-driven autonomy in space; certain claims are reported as stated in the literature.
Figure 2: Overview of our method. On Earth, we perform distillation of the VFM into the Fast-SCNN network using Algorithm 1 . As interfaces, we downlink ROS 2 bags and other logs from the satellite, as well as images, and uplinked labels after annotation. In space, we perform both training and inference of images captured by the onboard camera. We also close the loop on the segmented images to enable the spacecraft to point towards the Moon. The “snowflake” symbol represents the fact the DINO model is frozen (no gradients flow through the network).
Figure 3: Timeline of the different experiments run onboard the spacecraft since its launch. In total, we ran 12 experiments, the majority being inference and dynamic targeting experiments. We have run three training experiments on the TX2 NX’s GPU on Moon images with labels uploaded from the ground (Aug. 8th, 2025 and Dec. 15th, 2025). The weights from Aug. 2025 were used for the Sep., Nov., and Dec. experiments, and the ones from Dec. for the Jan. experiments. The Moon icon signifies in which phase the Moon was when the experiment was performed. See Table 2 for the parameters used in the Jan. 2026 tracking experiments. For discussion on these parameters, see Section 5-5.2 .
Figure 4: Performance of the training and inference experiments in space (August 8th). (top plot) Thermal results for the GPU. The experiment was run while the spacecraft was in eclipse. (second plot) GPU usage during training and inference. (third plot) Thermal results for the NVMe. For the first 3 plots, the data are sampled at 1 Hz. (bottom plot) Training running in two cycles for 60 epochs total.
| Experiment Date | Boson Frame Rate | KF Prediction Rate | Control Gain | Low-Pass Filter | Boson Frame Buffer |
| Run 3: Jan. 6th, 2026 | 1 Hz | 10 Hz | 2 | None | 4 |
| Run 4: Jan. 9th, 2026 | 1 Hz | 10 Hz | 2 | None | 1 |
| Run 5: Jan. 10th, 2026 | 2 Hz | 2 Hz | 1 | Enabled | 1 |
| Run 6: Jan. 26th, 2026 | 2 Hz | 2 Hz | 1/2 | Disabled | 1 |
| Run 7: Jan. 30th, 2026 | 2 Hz | 2 Hz | 1/2 | Disabled | 1 |
Table 2: Parameters used for the closed-loop control experiments in the experiments performed in January 2026.
| Experiment Date | Mean | Final | err | X | Y |
| Run 2: Dec. 10, 2025 | 167.8 | 197.2 | 127.0 | 117.6 | 71.2 |
| Run 3: Jan. 06, 2026 | 199.7 | 188.1 | 612.9 | 970.0 | 278.9 |
| Run 4: Jan. 09, 2026 | 579.8 | 2356.6 | 2335.5 | 4444.1 | 71.0 |
| Run 5: Jan. 10, 2026 | 176.8 | 37.0 | 726.8 | 1481.7 | 82.4 |
| Run 6: Jan. 26, 2026 | 7.4 | 4.0 | 20.1 | 34.1 | 6.8 |
| Run 7: Jan. 30, 2026 | 13.5 | 1.2 | 3.5 | 6.9 | 3.7 |
Table 3: Quantitative results of the pointing accuracy for dynamic targeting experiments. Values are reported in pixels.
Figure 5: Performance on the dynamic targeting task. (a) Results from the dynamic targeting experiment runs. (top) GPU temperature, (middle) CPU usage in total over the 4 cores, (lower) GPU load. The dots show when image inference was performed. For the runs on and after Jan. 10th, we increased the inference frequency from 1 Hz to 2 Hz. (b) Position of the Moon during the dynamic targeting phase (first row contains a sequence of segmented Moon images and the second row represents a sequence of true thermal images). The Moon was moving towards the center of the image, showing that Run 7 performs well. (c) Performance from two runs (image coordinates, quaternions, and reaction wheel commands). For a discussion on the cause of the oscillating behavior on Run 5, see Section 5-5.2 .
Figure 6: Example images from four datasets. The first dataset (a) consists of images taken inside our spacecraft simulator, while the second (b) is an open-source collection of spacecraft imagery [ 53 ] . The remaining two datasets (c, d) are thermal: one includes in-orbit observations of the Moon, and the other contains synthetically generated images based on Landsat data with overlaid spacecraft.
| Model | All | Body | Solar Panel | Antenna | Background |
| Fast-SCNN | 0.683 | 0.705 | 0.707 | 0.347 | 0.974 |
| DINO-distilled + Fast-SCNN (ours) | 0.756 | 0.764 | 0.768 | 0.480 | 0.982 |
Table 4: Spacecraft part segmentation performance (mIoU) on the Adelaide dataset.
| Model | All | Body | Solar Panel | Background |
| Fast-SCNN | 0.971 | 0.968 | 0.958 | 0.986 |
| DINO-distilled + Fast-SCNN (ours) | 0.972 | 0.969 | 0.961 | 0.987 |
Table 5: Spacecraft part segmentation model performance (mIoU) on our custom spacecraft simulator dataset.
| Model | % of Training Data Used | ||
| 75% | 50% | 12.5% | |
| Fast-SCNN | 0.971 | 0.966 | 0.933 |
| DINO-distilled + Fast-SCNN (ours) | 0.972 | 0.969 | 0.947 |
Table 6: Effect of training set size (% of overall training set) on short-range spacecraft part segmentation (mIoU) for the custom dataset.
| Model | % of Training Data Used | ||
| 75% | 50% | 12.5% | |
| Fast-SCNN | 0.683 | 0.598 | 0.537 |
| DINO-distilled + Fast-SCNN (ours) | 0.756 | 0.722 | 0.636 |
Table 7: Effect of training set size (% of overall training set) on short-range spacecraft part segmentation (mIoU) for the Adelaide dataset.
| Model | Task | Inference Time (ms) | Params (M) | |
| TX2 | Titan RTX | |||
| Fast-SCNN | Semantic Segm. | 49.39 | 3.54 | 1.13 |
| DINO-distilled + Fast-SCNN (ours) | Semantic Segm. | 92.56 | 4.29 | 1.55 |
Table 8: Inference time (ground results) on two different platforms.