Focal plane wavefront control with model-based reinforcement learning

Jalo Nousiainen; Iremsu Taskin; Markus Kasper; Gilles Orban De Xivry; Olivier Absil

astro-ph.IMApr 1, 2026

Focal plane wavefront control with model-based reinforcement learning

Authors: Jalo Nousiainen, Iremsu Taskin, Markus Kasper, Gilles Orban De Xivry, Olivier Absil

Organizations: European Southern Observatory (ESO), Karl-Schwarzschild-Str. 2, 85748 Garching, Germany · STAR Institute, Université de Liège, Allée du Six Août 19C, 4000 Liège, Belgium

Abstract

The direct imaging of potentially habitable exoplanets is one prime science case for high-contrast imaging instruments on extremely large telescopes. Most such exoplanets orbit close to their host stars, where their observation is limited by fast-moving atmospheric speckles and quasi-static non-common-path aberrations (NCPA). Conventional NCPA correction methods often use mechanical mirror probes, which compromise performance during operation. This work presents machine-learning-based NCPA control methods that automatically detect and correct both dynamic and static NCPA errors by leveraging sequential phase diversity. We extend previous work in reinforcement learning for AO to focal plane control. A new model-based RL algorithm, Policy Optimization for NCPAs (PO4NCPA), interprets the focal-plane image as input data and, through sequential phase diversity, determines phase corrections that optimize both non-coronagraphic and post-coronagraphic PSFs without prior system knowledge. Further, we demonstrate the effectiveness of this approach by numerically simulating static NCPA errors on a ground-based telescope and an infrared imager affected by water-vapor-induced seeing (dynamic NCPAs). Simulations show that PO4NCPA robustly compensates static and dynamic NCPAs. In static cases, it achieves near-optimal focal-plane light suppression with a coronagraph and near-optimal Strehl without one. With dynamics NCPA, it matches the performance of the modal least-squares reconstruction combined with a 1-step delay integrator in these metrics. The method remains effective for the ELT pupil, vector vortex coronagraph, and under photon and background noise. PO4NCPA is model-free and can be directly applied to standard imaging as well as to any coronagraph. Its sub-millisecond inference times and performance also make it suitable for real-time low-order correction of atmospheric turbulence beyond HCI.

Figures & tables

Figure 1: Illustration of the preprocessing step of the focal plane data, i.e, the observation. For a non-coronagraphic system, the Airy pattern (perfect PSF) is subtracted from the PSF, and for the perfect coronagraph, the perfect PSF is simply a dark image (background). The images are then flattened with the cubic root.

Figure 2: Dynamics model NN design. Trained on closed-loop data (science camera images and residual commands), the dynamics model learns to simulate the optical path of the light. Highlighted in green are the inputs to the NN.

Figure 3: Policy model NN design. In the control loop, the image inputs ( $\bm{o}_{t}$ , and $\bm{o}_{t-1}$ ) are focal plane images from the science camera, and while training, the future inputs (Algorithm 1 line 11, $t=2,3,\cdots H$ ) are predicted/simulated by the dynamics model.

Telescope
Parameter	Value	Units
Telescope diameter	39.3	m
Sampling frequency	10	Hz
DM influence functions	“Zernike”	-
Number of modes	55	-
Wavelength	11	µm

Table 1: Simulations parameters

Panel of Figure 4 — Figure 4: Training plots of PO4NCPA on circular pupil with SI and PC. Here we plot the negative cumulative reward (loss) after each episode in the training circle. The light blue curve shows the raw negative reward, and the dark blue shows the smoothed value (moving average). For the dynamic case (c, d), the DM is flattened only at the start of the episode for the first 40k episodes to prevent saturation. Afterward, each episode starts from the previous endpoint, thereby mimicking a continuously updated closed-loop control system. Hence, the bigger episode reward after that.

Panel of Figure 5 — Figure 5: PO4NCPA convergence on circular pupil with SI (left) and PC (right) in the case of static NCPA. Top row: reward on time steps. PO4NCPA episodes are shown in green (best), blue (median), red (worst), while black and gray lines correspond to fitting error rewards. Bottom row: PO4NCPA residual wavefront RMS (inside the control region) over the same episodes (fitting error is always zero).

Panel of Figure 6 — Figure 6: Circular pupil PSF sharpness/raw contrast with static NCPA for (a) standard imaging and (b) perfect coronagraph cases. Here, we take the average (over 500 episodes) of the last PSF frame and plot the resulting PSF and radial average divided by the peak intensity.

Panel of Figure 7 — Figure 7: A 100-time-step window during the long exposure dynamic NCPA, showing the wavefront error RMS as a function of time steps. (a) Standard imaging and (b) perfect coronagraph cases.

Panel of Figure 8 — Figure 8: RMS per mode over the long episode with dynamic NCPA for (a) standard imaging and (b) perfect coronagraph cases.

Panel of Figure 9 — Figure 9: PSF sharpness/contrast with dynamic NCPA for (a) standard imaging and (b) perfect coronagraph cases.

Figure 10: PO4NCPA performance compared against “Fitting error + delay error” on larger dynamic wavefront errors in the standard imaging case. The gray dashed line is the “Fitting error + delay error”, the red line is the open-loop/no-correction Strehl, the blue line is the PO4NCPA trained on the given

r_{0}

, and the green line is the PO4NCPA trained on

r_{0}=20

m applied to

r_{0}=15

m and 10 m.

Figure 11: Raw “residual” PSF contrast for ELT-METIS. Upper row: recorded PSF. Middle row: recorded PSFs with diffraction patterns (ideal PSF) subtracted. Lower row: the azimuthal average of the subtracted PSFs (i.e., “Raw residual PSF contras”).

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 272 papers

Reinforcement learning for post-coronagraphic wavefront control
Sep 16, 2026Manuela Castañeda-Medina, Yann Gutierrez, Johan Mazoyer +3RL Control Reinforcement Learning
LIRA · DTIS, ONERA, Université Paris-Saclay, 91120 Palaiseau, France · DOTA, ONERA, Université Paris Saclay, 92322 Châtillon, France +2
On-sky demonstration of reinforcement learning for adaptive optics control
Jun 9, 2026Jalo Nousiainen, Vincent Chambouleyron, Benoit Neichel +7RL Control Reinforcement Learning
European Southern Observatory · Aix Marseille University, CNRS, CNES, LAM, Marseille, France · Wakea Consulting, 4 Rue Joseph Bouchayer, 38100 Grenoble +1
Hybrid deep learning-based phase diversity method for wavefront reconstruction
Jun 24, 2026Y. Rodimkov, A. Kotov, K. Burdonov +4Quasi-Newton Methods Inverse Problems
Lobachevskii State University of Nizhni Novgorod, pr. Gagarina 23, Nizhni Novgorod, 603022 Russia · Institute of Applied Physics, Russian Academy of Sciences, Nizhny Novgorod 603951, Russia

Sep 16, 2026J/K move · Enter open · S save

Reinforcement learning for post-coronagraphic wavefront control

Manuela Castañeda-Medina, Yann Gutierrez, Johan Mazoyer, Baptiste Abeloos+2

LIRA · DTIS, ONERA, Université Paris-Saclay, 91120 Palaiseau, France · DOTA, ONERA, Université Paris Saclay, 92322 Châtillon, France+2

RL Control Reinforcement Learning

Open paper AskPDF arXiv

Direct imaging of exoplanets is limited by the extreme contrast between the star and the planets, which is mitigated using a coronagraph. However, optical aberrations cause starlight leakage through the coronagraph, producing speckles that obscure the planetary signal. Achieving the required contrast levels demands wavefront control with subnanometric precision. Deep reinforcement learning offers a promising alternative to traditional focal-plane wavefront control techniques by enabling adaptive correction strategies learned directly from interaction with the system. In this work, we present a fully data-driven method for post-coronagraphic aberration correction in a simulated high-contrast imaging testbed. The agent controls a deformable mirror using observations consisting of focal-plane measurements (images) and physics-informed wavefront sensing information derived from these images. We evaluate different observation representations and control strategies, and the method is validated on simplified simulations of a high-contrast imaging testbed, where it successfully creates dark holes, i.e., regions of the focal plane in which residual starlight is strongly suppressed, while approaching the performance of conventional wavefront control methods.

Abstract

RL Control

Explore similar work

Reinforcement learning for post-coronagraphic wavefront control

On-sky demonstration of reinforcement learning for adaptive optics control

Hybrid deep learning-based phase diversity method for wavefront reconstruction