Paired-Outcome Learning for Selective Spacecraft Attitude Capture under Torque Limits

Donghe Chen; Jia Deng; Zehao Jia; Lin Cheng; Shengping Gong

cs.LGOct 17, 2026

Paired-Outcome Learning for Selective Spacecraft Attitude Capture under Torque Limits

Authors: Donghe Chen, Jia Deng, Zehao Jia, Lin Cheng, Shengping Gong

Organizations: School of Astronautics, Beihang University, Beijing 100191, China · Astroup(Beijing) Technology Co., Ltd.

Abstract

Torque-limited spacecraft attitude capture requires adapting the maneuver speed to uncertain braking authority. Faster settings can reduce average tracking error while worsening individual maneuvers, making selective modification of an existing controller a baseline-relative learning problem. A paired-outcome method combines three mechanisms: physically interpretable continuation policies with different speed envelopes, separate prediction of task failure and successful-pair tracking gains, and context-dependent rejection that executes the unchanged baseline. Complete paired simulations provide labels under common plant parameters and measurement seeds; training, validation, and calibration are separated by context. Basilisk experiments use 108 development contexts, 324 labeling rollouts, and 624 evaluation episodes across 48 test contexts, including disclosed post-freeze diagnostics. The selective controller reduces aggregate integrated attitude error by 4.67% relative to the branch baseline, with 17 improvements, 31 exact ties, and no observed regressions. Always using the fast alternative achieves a 4.81% reduction but worsens 11 cases. Gains occur in principal-axis, near-axis, braking-boundary, and selected noise/drift maneuvers; general coupled cases retain the baseline. Ablations show no additional aggregate benefit from branch commitment, active querying, or score subtraction. The evidence supports a restricted empirical trade-off between average improvement and individual regression, without establishing a closed-loop safety guarantee.

Figures & tables

Panel of Figure 1 — Figure 1: One portfolio decision at the first capture-planning event. Rejection preserves the baseline’s existing learned branch decisions. Acceptance changes a continuation policy, not one isolated torque sample.

Action	$\bar{v}$	$\gamma$	$\tau_{v}$	$\tau_{T}$	Capture target update
0: baseline	1.6	0.65	8	15	inherited replanning
1: balanced	1.6	0.80	4	10	retain chosen target
2: fast	1.8	0.85	3	8	retain chosen target

Table 1: Portfolio parameters: speed caps in degrees per second and time constants in seconds;

K_{\perp}=5

in torque units per MRP component.

Stage	Contexts	Full rollouts	Purpose
Base training	48	144	preliminary fitting
Training-only queries	12	36	training augmentation
Validation	24	72	rule / fixed-policy selection
Calibration	24	72	score subtraction
Development total	108	324	excludes inherited branch data
Initial frozen test	48	384	eight configurations per case

Table 2: New portfolio data and evaluation budget. Symmetry copies are not independent contexts.

Configuration	Tasks	Mean IAE	Settling	Effort	Regressions
Native PD †	0/48	63,340.6	—	98.44	47
Branch baseline	48/48	26,060.4	400.92	81.72	0
Prior gated speed	48/48	26,059.7	397.21	81.93	2
Validation-fixed (= baseline)	48/48	26,060.4	400.92	81.72	0
Fixed balanced †	48/48	25,118.4	393.12	82.47	10
Fixed fast †	48/48	24,806.0	393.55	82.85	11

Table 3: Unified outcomes on 48 matched contexts. IAE: deg s; settling: s; effort:

\mathrm{N^{2}m^{2}s}

. Regressions count IAE increases relative to the branch baseline.

\dagger

denotes post-freeze diagnostics. Native PD has different gains and no disturbance observer; a missing settling mean indicates at least one unsettled episode.

Figure 3: Cross-baseline control outcomes on 48 matched contexts. a , Aggregate IAE reduction with exploratory 95% paired-case bootstrap intervals (10,000 resamples, seed 834092); a symmetric-log axis compresses the native PD reference while retaining a linear region near zero. b , Mean integral of squared actual torque. c , Case-level IAE wins, ties, and losses relative to the branch baseline. Native PD has zero task completions, while the other displayed configurations have 48; an IAE win does not imply task completion. Fixed 0 and validation-fixed duplicate the branch baseline and are represented by its single row.

Comparison	Reduction (%)	Win/tie/loss	95% interval
Branch baseline	4.67	17/31/0	$[731.7,\ 1,728.9]$
Prior gated speed	4.67	11/32/5	$[-508.5,\ 3,274.6]$
Geometric lookup	1.53	8/31/9	$[55.8,\ 779.2]$
No active queries	-0.05	2/43/3	$[-119.7,\ 112.3]$
No score subtraction	-0.12	0/45/3	$[-71.6,\ 0.0]$
Greedy gain	-1.23	4/24/20	$[-499.2,\ -139.0]$

Table 4: Main method versus each configuration. Positive reductions favor the main method. Intervals are exploratory paired-case bootstrap intervals for absolute mean IAE improvement, in deg s.

Figure 4: Paired IAE changes for all 48 cases. Positive values favor the alternative. Selective control has 17 gains and 31 ties; greedy selection has a larger aggregate gain but four regressions.

Figure 5: Direct diagnostics of the intelligent outer selector. a , Actions in all 48 cases: 0 retains the baseline, 1 uses balanced speed, and 2 uses fast speed. Cases are ordered by original index within each family. b , Predicted and realized log IAE gains for both candidates in contexts with planning features; filled markers are adopted, hollow markers are not. The diagonal is perfect prediction; the vertical dotted line marks the gain margin plus subtraction and is only one acceptance condition. c , Individual case reductions (points) and aggregate reductions (diamonds); their definitions differ. d , Actual counterfactual IAE regressions when alternatives are forced in the 31 rejected cases. These diagnostics do not establish calibrated probabilities or safety guarantees.

Family	Baseline IAE	Selective IAE	Reduction (%)	Adoptions
Ordinary	5,642.0	5,642.0	0.00	0/8
Principal axis	31,279.5	28,853.9	7.75	6/8
Near axis	32,777.6	30,564.3	6.75	5/8
Coupled	15,047.9	15,047.9	0.00	0/8
Braking boundary	45,037.7	43,048.9	4.42	4/8
Noise and drift	26,577.6	25,897.6	2.56	2/8

Table 5: Family-level means and adoption. Each row has eight successes under both methods. Reductions are relative to the branch baseline.

Figure 6: Family-level aggregate IAE reductions relative to the branch baseline for each distinct policy comparator; each cell uses eight paired contexts. Positive values indicate improvement. Fixed speed alternatives improve some families while degrading others.

Figure 7: Control transients for all distinct policy comparators: native PD reference, branch baseline, prior gated speed, geometric lookup, greedy learned selection, fixed balanced, fixed fast, and full selective learning. Columns show near-axis case 038 (median adopted near-axis gain), noise/drift case 041 (lower adopted gain), and adverse case 032 (largest regression versus prior gated speed). Rows show attitude error, angular-rate magnitude, and actual torque norm. Overlapping trajectories are not omitted. All curves use unsmoothed

0.1\,

s logs; componentwise torque limits should not be inferred from torque norms. Data/correction ablations appear separately in Fig. 11 .

Figure 8: Case 038 component histories: shadow MRPs (top), angular velocities (middle), and actual applied torques (bottom). Dotted gray lines mark

\pm 0.3\;\mathrm{N\,m}

component limits. MRP jumps are shadow-coordinate switches, not physical attitude discontinuities. Overlapping curves remain visible as a single trace. All curves use unsmoothed

0.1\,

s logs.

Figure 9: Noise/drift case 041, selected as the lower of the two adopted gains in this family. Rows show shadow MRPs, angular velocities, and actual applied torques. The same colors, line styles, time span, and component torque limits as Fig. 8 permit direct reading across the two examples.

Figure 10: Evidence for the proposed mechanisms. a , Interventions on branch commitment and selected speed; commitment alone does not improve performance, and removing commitment from the selected-speed controller leaves trajectories unchanged. b , Learned selection versus fixed alternatives, geometric lookup, and greedy learning. c , Data acquisition and score-correction ablations show no extra aggregate gain for the full pipeline. Points show aggregate reductions relative to the branch baseline; intervals use the same paired-case bootstrap as Fig. 3 . Intervals describe context variation, not repeated training.

Figure 11: Module ablation transients, compared with the branch baseline and full selector. Rows show commitment-only, selected-speed-only, no-active-query, and no-score-subtraction controllers. For each row, the case with the largest absolute IAE difference from the full method is selected, explicitly as a diagnostic extreme; the identical selected-speed-only variant uses case 038. Columns show attitude error, rate magnitude, and actual applied torque norm. Original

0.1\,

s logs are unsmoothed. Full three-axis state and torque histories for two examples appear in Figs. 8 and 9 .

Policy Learning

Policy learning develops decision rules that choose actions or interventions to maximize outcomes from data, feedback, or interaction.

See all 484 papers

Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry
Jun 30, 2026Alexander Fabisch, Melvin Laux, Mariela De Lucas Álvarez +2RL Control Spacecraft Attitude Determination
Robotics Innovation Center, German Research Center for Artificial Intelligence (DFKI GmbH), Germany · 2Airbus Defence and Space GmbH, Germany
How Tuning, Reference Preview, Actuator Modeling, and Fallback Policies Change Quadrotor Controller Comparisons: A Reproducible Simulation Study
Oct 26, 2026Xuan-Quy Dao, Le-Minh Nguyen, Ngoc-Bich LeQuadrotor Control Model Predictive Control
Eastern International University Binh Duong, Vietnam · Industrial University of Ho Chi Minh City Ho Chi Minh City, Vietnam
Prior Policy Guided Dual-Agent Coordinated Manipulation Planning of Spacecraft-Manipulator System
May 25, 2026Yuhui Hu, Dong Zhou, Kaihong Ouyang +3Robotic Control Spacecraft Attitude Determination
School of Astronautics, Harbin Institute of Technology, Harbin, 150001, China · School of Automation, Chongqing University, Chongqing, 400044, China · School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, China

Jun 30, 2026J/K move · Enter open · S save

Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry

Alexander Fabisch, Melvin Laux, Mariela De Lucas Álvarez, Edoardo Caroselli+1

Robotics Innovation Center, German Research Center for Artificial Intelligence (DFKI GmbH), Germany · 2Airbus Defence and Space GmbH, Germany

RL Control Spacecraft Attitude Determination Feedback Control Robust RL

Open paper AskPDF arXiv

Deep reinforcement learning has the potential to solve attitude control problems more adaptively, precisely, and robustly by handling nonlinear dynamics, uncertainties, and failure cases more effectively than traditional attitude control approaches. We explore reinforcement learning (RL) for attitude control in spacecraft re-entry. An industry-standard proportional-integral-derivative controller with gain scheduling serves as a strong baseline for model-free RL and hybrid controllers that combine these two approaches. We formalize the application in the RL framework to apply continuous, off-policy RL. State-of-the-art RL achieves comparable performance to traditional control approaches in this domain. However, its out-of-distribution generalization is not sufficient. Hence, we use dynamics randomization to introduce challenging task variations during training and enforce generalization in a predefined operational envelope. Finally, we assess the best obtained RL-based controllers with application-specific metrics to show superior performance in comparison to traditional controllers in the operational envelope, that is, hybrid controllers are able to track the angle of attack better and are more robust under variations of mass, inertia tensor, and flap actuator bandwidth.

Abstract

Policy Learning

Explore similar work

Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry

How Tuning, Reference Preview, Actuator Modeling, and Fallback Policies Change Quadrotor Controller Comparisons: A Reproducible Simulation Study

Prior Policy Guided Dual-Agent Coordinated Manipulation Planning of Spacecraft-Manipulator System