High-Quality and Efficient Turbulence Mitigation with Events
Authors: Xiaoran Zhang, Jian Ding, Yuxing Duan, Haoyue Liu, Gang Chen, Yi Chang, Luxin Yan
Organizations: State Key Laboratory of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · School of Computer Science and Engineering, Sun Yat-sen University
Turbulence mitigation (TM) is highly ill-posed due to the stochastic nature of atmospheric turbulence. Most methods rely on multiple frames recorded by conventional cameras to capture stable patterns in natural scenarios. However, they inevitably suffer from a trade-off between accuracy and efficiency: more frames enhance restoration at the cost of higher system latency and larger data overhead. Event cameras, equipped with microsecond temporal resolution and efficient sensing of dynamic changes, offer an opportunity to break the bottleneck. In this work, we present EHETM, a high-quality and efficient TM method inspired by the superiority of events to model motions in continuous sequences. We discover two key phenomena: (1) turbulence-induced events exhibit distinct polarity alternation correlated with sharp image gradients, providing structural cues for restoring scenes; and (2) dynamic objects form spatiotemporally coherent ``event tubes'' in contrast to irregular patterns within turbulent events, providing motion priors for disentangling objects from turbulence. Based on these insights, we design two complementary modules that respectively leverage polarity-weighted gradients for scene refinement and event-tube constraints for motion decoupling, achieving high-quality restoration with few frames. Furthermore, we construct two real-world event-frame turbulence datasets covering atmospheric and thermal cases. Experiments show that EHETM outperforms SOTA methods, especially under scenes with dynamic objects, while reducing data overhead and system latency by approximately 77.3% and 89.5%, respectively. Our code is available at: https://github.com/Xavier667/EHETM.
Figures & tables
Figure 1: Workflow comparison between our EHETM and multi-frame TM methods (MFTM, exemplified by DATUM [ 8 ] ). (a) DATUM [ 8 ] suffers from high system latency and large overhead due to multi-frame accumulation, while distortions at dynamic object edges remains. (b) Our EHETM uses events for guidance, producing higher-quality and efficient restoration with few frames.
Figure 2: Merits of high-speed observations. (a) It requires less accumulation time to reach the same error (P1 vs. P3) and achieves lower error within the same time (P1 vs. P2). (b) It captures finer-gained motion cues compared to low-temporal-resolution ones.
Figure 3: Our real-world event-frame turbulence datasets and acquisition setups. (a) CTTH dataset is captured under adjustable thermal turbulence, containing about 30k image-event pairs with dynamic objects and GTs. (b) LATH dataset is acquired using a 1000 mm lens covering multiple scenes at various shooting distances.
Figure 4: Event responses of scenes caused by turbulence. (a) Turbulence induces event polarity alternation along structural edges. (b) The PAEP numbers correlate with sharp image gradients, providing structural cues for suppressing edge distortions.
Figure 5: Event responses of the dynamic object under turbulence. (a) Turbulence-free: the object triggers temporally continuous and spatially dense events, forming a stable event tube. (b) Under turbulence: the event tube is structurally degraded, while turbulence produces temporally discontinuous and spatially sparse events.
Figure 6: Overall Architecture of the EHETM. (a) ET-Stable stabilizes object motion via the rigid motion-aware block and event tube optimization, leveraging dense spatiotemporal cues to efficiently decouple object motion from turbulence. (b) EPAW-Stable generates masks to isolate turbulence-affected scene regions, and encodes the frequency of event polarity alternation as adaptive weights on temporally averaged gradients to suppress scene distortions. (c) ET-Stable and EPAW-Stable extract stable patterns from turbulence to guide a light-weight Mamba-based video restoration network, enabling high-quality and efficient restoration with few input frames.
Methods
Task / Input Type
CTTH / UDET Datasets
LATH Dataset
Efficiency (512 × 512 Patches)
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
#Params (M) ↓
FPS ↑
FLOPs (G) ↓
Latency (ms) ↓
Data Size (%) ↓
TurbNet [ 18 ]
TM / Single-frame
28.17 / 24.12
0.8315 / 0.7784
0.2210 / 0.2679
16.15
26.6
5.52
686.7
–
2.5
VRT [ 70 ]
VR / Multi-frame
32.55 / 26.58
0.8986 / 0.8139
0.1613 / 0.2334
14.37
18.3
0.31
2509.8
600
40.0
TSRWGAN [ 1 ]
TM / Multi-frame
30.19 / 26.72
0.8569 / 0.8027
0.1812 / 0.2358
13.87
46.3
1.33
1280.3
280
37.5
TMT [ 37 ]
TM / Multi-frame
32.56 / 27.73
0.9046 / 0.8449
0.1743 / 0.1975
13.09
26.1
0.11
463.9
760
50.0
DATUM [ 8 ]
TM / Multi-frame
34.89 / 28.85
0.9340 / 0.8688
0.1448 / 0.1565
10.87
5.8
12.1
191.5
1560
100.0
Table 1: Quantitative comparisons on CTTH, LATH and UDET datasets. The best results are in bold and the second-best are underlined .
Figure 7: Visual comparisons on CTTH and UDET datasets. Our EHETM yields more visual pleasing results, especially around regions with moving objects.
Figure 8: Visual comparisons on LATH dataset. EHETM produces more satisfactory visual results and achieves outstanding generalization under different real-world scenes. The number of inference frames from EHETM to VRT are as follows: 8 , 15, 10, 40, 20, 20, 15, 16.
RMAB
ETO
EPAW
PSNR ↑
SSIM ↑
LPIPS ↓
✓
31.13
0.8962
0.2261
✓
33.85
0.9241
0.1736
✓
✓
34.19
0.9309
0.1563
✓
✓
✓
35.17
0.9425
0.1281
Table 2: Ablation studies of EHETM
Figure 9: Impact of the inference data size. With only few frames and events, our method demonstrates comparable or even higher quality restoration compared to other multi-frame methods.
Figure 10: Effectiveness of ETO. It extracts clean event tubes triggered by objects with various motions from turbulent events.
Figure 11: Effectiveness of EPAW. (a) How does EPAW work. (b) Visualization of selected section lines and the variance computed along the t axis.
This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second.
Yu-Hsiang Huang, Levi Burner, Sachin Shah +3
University of Maryland, College Park MD 20742, USA · Dartmouth College, Hanover NH 03755, USA
Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering, posing significant challenges to both visual clarity and temporal consistency. Current state-of-the-art methods are based on 3D architectures and require multi-frame input, but their large computational cost and memory usage limit real-time deployment. Here, we propose ReMATF, a lightweight recurrent framework that restores videos using only two frames at a time while preserving spatial detail and temporal stability. ReMATF combines a multi-scale encoder-decoder with temporal warping and a motion-adaptive temporal fusion module that performs per-pixel fusion between the warped previous output and the current prediction to enhance coherence without enlarging the temporal window. This design reduces flicker, sharpens details, and remains efficient. Experiments on synthetic and real turbulence datasets show consistent improvements in PSNR(+2.83 dB), SSIM(+0.05) and LPIPS(-0.048), along with substantially faster inference, making ReMATF suitable for turbulence mitigation in resource-constrained scenarios.
Atmospheric turbulence degrades image quality in long-range imaging. The degradation is generally modeled with two spatio-temporally varying components, tilt and blur. Recent unsupervised turbulence mitigation methods have demonstrated the effectiveness of representing a sharp image as an implicit neural representation (INR). However, their tilt and blur representations remain frame-dependent, and the number of optimization parameters grows with the number of input frames. To address this limitation, we propose TriINR, an unsupervised turbulence mitigation method that recovers a sharp image from multiple degraded frames. TriINR represents the sharp image, tilt, and blur as three implicit neural representations. By modeling tilt and blur as spatio-temporal INRs, TriINR achieves frame-count-independent degradation representation and memory-efficient optimization. We further incorporate physical and statistical properties of atmospheric turbulence as constraints on the INRs to improve restoration accuracy. Experiments on synthesized turbulence data with three degradation strengths show that TriINR restores images more robustly across varying turbulence strengths than prior TM methods and achieves the highest accuracy among the evaluated unsupervised methods. Our method also demonstrates robustness on real long-range scenes containing additional non-ideal motion.