Inspectorch: Efficient rare event exploration in solar observations
Authors: C. J. Díaz Baso, I. J. Soler Poquet, C. Kuckein, M. van Noort, N. Poirier
Organizations: Institute of Theoretical Astrophysics, University of Oslo, P.O. Box 1029 Blindern, N-0315 Oslo, Norway · Rosseland Centre for Solar Physics, University of Oslo, P.O. Box 1029 Blindern, N-0315 Oslo, Norway · Instituto de Astrofísica de Canarias, C/Vía Láctea s/n, E-38205 La Laguna, Tenerife, Spain · Departamento de Astrofísica, Universidad de La Laguna, E-38206 La Laguna, Tenerife, Spain · Max-Planck Institute for Solar System Research, Justus-von-Liebig-Weg 3, 37077 Göttingen, Germany · LPC2E, OSUC, Univ Orléans, CNRS, CNES, F-45071 Orléans, France
The Sun is observed in unprecedented detail, enabling studies of its activity on very small spatiotemporal scales. However, the large volume of data collected by our telescopes cannot be fully analyzed with conventional methods. Popular machine learning methods identify general trends from observations, but tend to overlook unusual events due to their low frequency of occurrence. We study the applicability of unsupervised probabilistic methods to efficiently identify rare events in multidimensional solar observations and optimize our computational resources to the study of these extreme phenomena. We introduce Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations. Once optimized, it assigns a probability to each sample, allowing us to identify unusual events. We apply this approach by applying it to observations from the Hinode Spectro-Polarimeter, the Interface Region Imaging Spectrograph, the Microlensed Hyperspectral Imager at Swedish 1-m Solar Telescope, the Atmospheric Imaging Assembly on board the Solar Dynamics Observatory and the Extreme Ultraviolet Imager on board Solar Orbiter. We find that the algorithm assigns consistently lower probabilities to spectra that exhibit unusual features. For example, it identifies profiles with very strong Doppler shifts, uncommon broadening, and temporal dynamics associated with small-scale reconnection events, among others. As a result, Inspectorch demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets. The resulting probabilistic anomaly scores allow computational resources to be focused on the most informative and physically relevant events. We make our Python package publicly available at https://github.com/cdiazbas/inspectorch.
Figures & tables
FIGURE 1: Schematic illustration of the Normalizing Flow approach for rare event detection for one-dimensional data. The model learns an invertible transformation that maps a simple distribution (typically a Gaussian) to a complex distribution. Samples can be generated by mapping from latent space to data space (solid arrows), while likelihoods are evaluated by the inverse mapping (dashed arrows). Events with very low probabilities correspond to rare or anomalous samples.
FIGURE 2: Application of Inspectorch to a Hinode/SP dataset containing a sunspot. The left panels show the continuum intensity map at 630.10 nm and log-probability logpϕ(x) map. The color scale is clipped to highlight the most extreme values, although some pixels have log-probabilities below −110 . The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. All spectral profiles are normalized to their own maximum value to facilitate shape comparison (a.u.: arbitrary units); the likelihood evaluation is performed on the unnormalized spectra. A vertical dashed line indicates Doppler shifts of 10 km s -1 and an average quiet-Sun profile in gray is shown for reference.
FIGURE 3: Comparison between the application of Normalizing Flows and Isolation Forest to the same Hinode dataset shown in Fig. 2 . The top panels show the log-probability map computed when using the method based on Normalizing Flows with the Hinode dataset using one wavelength point (the continuum at 6301.0 nm) on the left and two wavelength points (the continuum at 6301.0 nm and the core at 6301.5 nm) on the right. The bottom panels show the anomaly score map computed using the Isolation Forest method with the same dataset and same wavelength points. Note that Isolation Forest produces anomaly scores, not probabilities.
FIGURE 4: Application of Inspectorch to an IRIS dataset of a coronal hole. The left panels show the intensity map at the core of the Si iv line and log-probability logpϕ(x) map. Dashed lines indicate the boundaries of the coronal hole as seen in SDO/AIA 193 Å. The color scale in the log-probability map is clipped to a minimum of −150 to enhance contrast, though some pixels exhibit lower values. The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. All spectral profiles are normalized to their own maximum value to facilitate shape comparison (a.u.: arbitrary units); the likelihood evaluation is performed on the unnormalized spectra. Vertical dashed lines indicate Doppler shifts of ±70 km s -1 and an average quiet-Sun profile in gray is also shown for reference.
FIGURE 5: Temporal evolution of the minimum (red line) and maximum (gray line) log-probability values in a time series of MiHI observations. Sharp dips in the minimum log-probability indicate the occurrence of very unusual events.
FIGURE 6: Distribution of the log-probability values in the MiHI dataset. The percentage in the legend indicates the fraction of all profiles with a log-probability below the vertical line.
FIGURE 7: Selection of the three most unusual spectra identified in the MiHI temporal series. Each row shows the log-probability map (left) and the continuum intensity map (middle) at the time of the spectrum (right). A cross marks the location of the spectrum in the maps. The color scale in the log-probability map is clipped to a minimum to enhance contrast. The spectra are normalized to the average quiet Sun intensity at the time of the observation. The gray spectra represent the average quiet Sun profile of this dataset. This figure does not include the full spectral range in MiHI but the inner region around the two Fe i lines.
FIGURE 8: Selection of the three most unusual spectra identified in the SDO dataset when combining the 1600 Å and 1700 Å channels. The log-probability map (left) tends to capture the transition-region contribution of the 1600 Å channel as the unusual behavior. The dashed contours enclose regions where the probability map has a value below −50,−18,−18 for each row respectively.
FIGURE 9: Application of Inspectorch to the spatial dimension of the Hinode/SP dataset, using a 5 pixels × 5 pixels patch. The log-probability map highlights regions with unusual spatial patterns.
FIGURE 10: Application of Inspectorch to the temporal dimension of the SDO/AIA dataset using a temporal window of 11 time steps (4.4 minutes). The upper panel shows the log-probability map derived from the temporal evolution of the 1600 Å and 1700 Å passbands, while the lower panel displays the corresponding AIA 1600 Å intensity image at the central frame of the temporal window (approx. 07:50 UT). The yellow contour marks regions with intensities exceeding three times the average quiet-Sun intensity at the time of the observation. The red contour highlights pixels with log-probability values below −30 , identifying locations with unusual temporal behavior within the interval. A movie showing the full temporal evolution is available online.
FIGURE 11: Application of Inspectorch to the temporal dimension of the Solar Orbiter/EUI dataset, using a Fourier decomposition for dimensionality reduction. The upper panel displays the log-probability map of the time series and the lower panel the 174 Å passband at the start of the observations. Solid boxes labeled A highlight regions corresponding to small-scale transient brightenings, while dashed boxes labeled B indicate areas exhibiting coronal loop oscillations. A movie showing the full temporal evolution is available online.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
FIGURE 12: Log-probability map for the Hinode/SP dataset using Stokes V information. The color scale in the log-probability map is clipped to a minimum to enhance contrast. Circled in black are the locations of four unusual profiles shown in Fig. 13 .
FIGURE 13: Four of the most unusual spectra identified in the Hinode/SP dataset using Stokes V information. The spectra are normalized to their maximum value. The gray profile represents a typical circular polarization profile in this dataset. The dashed vertical line indicates a Doppler shift of 12 km s -1 . The profile coordinates (in arcseconds) at the bottom of each panel correspond to the circled locations in Fig. 12 .
NSF-Simons AI Institute for the Sky (SkAI), Chicago, IL, USA · Department of Industrial Engineering and Management Sciences, Northwestern University · Department of Astronomy, University of Illinois Urbana-Champaign +4