Inspectorch: Efficient rare event exploration in solar observations
Authors: C. J. Díaz Baso, I. J. Soler Poquet, C. Kuckein, M. van Noort, N. Poirier
Organizations: Institute of Theoretical Astrophysics, University of Oslo, P.O. Box 1029 Blindern, N-0315 Oslo, Norway · Rosseland Centre for Solar Physics, University of Oslo, P.O. Box 1029 Blindern, N-0315 Oslo, Norway · Instituto de Astrofísica de Canarias, C/Vía Láctea s/n, E-38205 La Laguna, Tenerife, Spain · Departamento de Astrofísica, Universidad de La Laguna, E-38206 La Laguna, Tenerife, Spain · Max-Planck Institute for Solar System Research, Justus-von-Liebig-Weg 3, 37077 Göttingen, Germany · LPC2E, OSUC, Univ Orléans, CNRS, CNES, F-45071 Orléans, France
The Sun is observed in unprecedented detail, enabling studies of its activity on very small spatiotemporal scales. However, the large volume of data collected by our telescopes cannot be fully analyzed with conventional methods. Popular machine learning methods identify general trends from observations, but tend to overlook unusual events due to their low frequency of occurrence. We study the applicability of unsupervised probabilistic methods to efficiently identify rare events in multidimensional solar observations and optimize our computational resources to the study of these extreme phenomena. We introduce Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations. Once optimized, it assigns a probability to each sample, allowing us to identify unusual events. We apply this approach by applying it to observations from the Hinode Spectro-Polarimeter, the Interface Region Imaging Spectrograph, the Microlensed Hyperspectral Imager at Swedish 1-m Solar Telescope, the Atmospheric Imaging Assembly on board the Solar Dynamics Observatory and the Extreme Ultraviolet Imager on board Solar Orbiter. We find that the algorithm assigns consistently lower probabilities to spectra that exhibit unusual features. For example, it identifies profiles with very strong Doppler shifts, uncommon broadening, and temporal dynamics associated with small-scale reconnection events, among others. As a result, Inspectorch demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets. The resulting probabilistic anomaly scores allow computational resources to be focused on the most informative and physically relevant events. We make our Python package publicly available at https://github.com/cdiazbas/inspectorch.
Figures & tables
FIGURE 1: Schematic illustration of the Normalizing Flow approach for rare event detection for one-dimensional data. The model learns an invertible transformation that maps a simple distribution (typically a Gaussian) to a complex distribution. Samples can be generated by mapping from latent space to data space (solid arrows), while likelihoods are evaluated by the inverse mapping (dashed arrows). Events with very low probabilities correspond to rare or anomalous samples.
FIGURE 2: Application of Inspectorch to a Hinode/SP dataset containing a sunspot. The left panels show the continuum intensity map at 630.10 nm and log-probability logpϕ(x) map. The color scale is clipped to highlight the most extreme values, although some pixels have log-probabilities below −110 . The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. All spectral profiles are normalized to their own maximum value to facilitate shape comparison (a.u.: arbitrary units); the likelihood evaluation is performed on the unnormalized spectra. A vertical dashed line indicates Doppler shifts of 10 km s -1 and an average quiet-Sun profile in gray is shown for reference.
FIGURE 3: Comparison between the application of Normalizing Flows and Isolation Forest to the same Hinode dataset shown in Fig. 2 . The top panels show the log-probability map computed when using the method based on Normalizing Flows with the Hinode dataset using one wavelength point (the continuum at 6301.0 nm) on the left and two wavelength points (the continuum at 6301.0 nm and the core at 6301.5 nm) on the right. The bottom panels show the anomaly score map computed using the Isolation Forest method with the same dataset and same wavelength points. Note that Isolation Forest produces anomaly scores, not probabilities.
FIGURE 4: Application of Inspectorch to an IRIS dataset of a coronal hole. The left panels show the intensity map at the core of the Si iv line and log-probability logpϕ(x) map. Dashed lines indicate the boundaries of the coronal hole as seen in SDO/AIA 193 Å. The color scale in the log-probability map is clipped to a minimum of −150 to enhance contrast, though some pixels exhibit lower values. The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. All spectral profiles are normalized to their own maximum value to facilitate shape comparison (a.u.: arbitrary units); the likelihood evaluation is performed on the unnormalized spectra. Vertical dashed lines indicate Doppler shifts of ±70 km s -1 and an average quiet-Sun profile in gray is also shown for reference.
FIGURE 5: Temporal evolution of the minimum (red line) and maximum (gray line) log-probability values in a time series of MiHI observations. Sharp dips in the minimum log-probability indicate the occurrence of very unusual events.
FIGURE 6: Distribution of the log-probability values in the MiHI dataset. The percentage in the legend indicates the fraction of all profiles with a log-probability below the vertical line.
FIGURE 7: Selection of the three most unusual spectra identified in the MiHI temporal series. Each row shows the log-probability map (left) and the continuum intensity map (middle) at the time of the spectrum (right). A cross marks the location of the spectrum in the maps. The color scale in the log-probability map is clipped to a minimum to enhance contrast. The spectra are normalized to the average quiet Sun intensity at the time of the observation. The gray spectra represent the average quiet Sun profile of this dataset. This figure does not include the full spectral range in MiHI but the inner region around the two Fe i lines.
FIGURE 8: Selection of the three most unusual spectra identified in the SDO dataset when combining the 1600 Å and 1700 Å channels. The log-probability map (left) tends to capture the transition-region contribution of the 1600 Å channel as the unusual behavior. The dashed contours enclose regions where the probability map has a value below −50,−18,−18 for each row respectively.
FIGURE 9: Application of Inspectorch to the spatial dimension of the Hinode/SP dataset, using a 5 pixels × 5 pixels patch. The log-probability map highlights regions with unusual spatial patterns.
FIGURE 10: Application of Inspectorch to the temporal dimension of the SDO/AIA dataset using a temporal window of 11 time steps (4.4 minutes). The upper panel shows the log-probability map derived from the temporal evolution of the 1600 Å and 1700 Å passbands, while the lower panel displays the corresponding AIA 1600 Å intensity image at the central frame of the temporal window (approx. 07:50 UT). The yellow contour marks regions with intensities exceeding three times the average quiet-Sun intensity at the time of the observation. The red contour highlights pixels with log-probability values below −30 , identifying locations with unusual temporal behavior within the interval. A movie showing the full temporal evolution is available online.
FIGURE 11: Application of Inspectorch to the temporal dimension of the Solar Orbiter/EUI dataset, using a Fourier decomposition for dimensionality reduction. The upper panel displays the log-probability map of the time series and the lower panel the 174 Å passband at the start of the observations. Solid boxes labeled A highlight regions corresponding to small-scale transient brightenings, while dashed boxes labeled B indicate areas exhibiting coronal loop oscillations. A movie showing the full temporal evolution is available online.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
FIGURE 12: Log-probability map for the Hinode/SP dataset using Stokes V information. The color scale in the log-probability map is clipped to a minimum to enhance contrast. Circled in black are the locations of four unusual profiles shown in Fig. 13 .
FIGURE 13: Four of the most unusual spectra identified in the Hinode/SP dataset using Stokes V information. The spectra are normalized to their maximum value. The gray profile represents a typical circular polarization profile in this dataset. The dashed vertical line indicates a Doppler shift of 12 km s -1 . The profile coordinates (in arcseconds) at the bottom of each panel correspond to the circled locations in Fig. 12 .
The discovery rate of optical transients will explode to 10 million public alerts per night once the Vera C. Rubin Observatory's Legacy Survey of Space and Time comes online, overwhelming the traditional physics-based inference pipelines. A continuous-time forecasting AI model is of interest because it can deliver millisecond-scale inference for thousands of objects per day, whereas legacy MCMC codes need hours per object. In this paper, we propose SELDON, a new continuous-time variational autoencoder for panels of sparse and irregularly time-sampled (gappy) astrophysical light curves that are nonstationary, heteroscedastic, and inherently dependent. SELDON combines a masked GRU-ODE encoder with a latent neural ODE propagator and an interpretable Gaussian-basis decoder. The encoder learns to summarize panels of imbalanced and correlated data even when only a handful of points are observed. The neural ODE then integrates this hidden state forward in continuous time, extrapolating to future unseen epochs. This extrapolated time series is further encoded by deep sets to a latent distribution that is decoded to a weighted sum of Gaussian basis functions, the parameters of which are physically meaningful. Such parameters (e.g., rise time, decay rate, peak flux) directly drive downstream prioritization of spectroscopic follow-up for astrophysical surveys. Beyond astronomy, the architecture of SELDON offers a generic recipe for interpretable and continuous-time sequence modeling in any time domain where data are multivariate, sparse, heteroscedastic, and irregularly spaced.
Jiezhong Wu, Jack O'Brien, Jennifer Li +6
NSF-Simons AI Institute for the Sky (SkAI), Chicago, IL, USA · Department of Industrial Engineering and Management Sciences, Northwestern University · Department of Astronomy, University of Illinois Urbana-Champaign +4
We present FlareEUV, a multimodal deep learning framework for predicting daily extreme ultraviolet (EUV) irradiance at 6.5 nm over three consecutive days during significant solar flares, using multi-instrument observations from NASA's Solar Dynamics Observatory (SDO). We consider 33 significant flares in the period between 2011 and 2014 in Solar Cycle 24. The SDO observations include 13 co-aligned full-disk images, comprising eight AIA EUV/UV and five HMI magnetic/continuum products. FlareEUV learns the relationship between magnetic structure and coronal emission from the raw imaging data using a lightweight attention-based architecture. Our experimental results demonstrate the good performance of FlareEUV in short-term EUV irradiance forecasting during the significant flares and its superiority over baseline methods.
Sathvik Soman, Jason T. L. Wang, Haimin Wang +1
New Jersey Institute of Technology, Newark, NJ, USA · Sam Houston State University, Huntsville, TX, USA
Extreme weather events, such as windstorms and heatwaves, are driven by persistent atmospheric circulation patterns that evolve over several consecutive days. While traditional circulation-based studies often focus on instantaneous atmospheric states, capturing the temporal evolution, or trajectory, of these spatial fields is essential for characterizing rare and potentially impactful atmospheric behavior. However, performing an exhaustive similarity search on multi-decadal, continental-scale gridded datasets presents significant computational and memory challenges. In this paper, we propose TRAKNN (TRajectory Aware KNN), a fully unsupervised and data-agnostic framework for detecting geometrically rare short trajectories in spatio-temporal data with an exact kNN approach. TRAKNN leverages a recurrence-based algorithm that decouples computational complexity from trajectory length and efficient batch operations, maximizing computational intensity. These optimizations enable exhaustive analysis on standard workstations, either on CPU or on GPU. We evaluate our approach on 75 years of daily European sea-level pressure data. Our results illustrate that rare trajectories identified by TRAKNN correspond to physically coherent atmospheric anomalies and align with independent extreme-event databases.
Guillaume Coulaud, Davide Faranda
UM, IROKO · University of Montpellier, Inria, CNRS, LIRMM, Montpellier, France · LMD, LSCE, ESTIMR +3