High-speed Imaging through Turbulence with Event-based Light Fields

Yu-Hsiang Huang; Levi Burner; Sachin Shah; Ziyuan Qu; Adithya Pediredla; Christopher A. Metzler

cs.CVMar 14, 2026

High-speed Imaging through Turbulence with Event-based Light Fields

Authors: Yu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler

Organizations: University of Maryland, College Park MD 20742, USA · Dartmouth College, Hanover NH 03755, USA

Abstract

This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second.

Figures & tables

Figure 1: Method overview. We introduce an event-based light field camera to image through turbulence at high speed. The event camera avoids motion blur by detecting brightness changes at microsecond time resolution. The light field offers cross-view constraints on the scene.

Figure 2: Image model for event-based light field camera under turbulence. A light field camera captures $N$ simultaneous sub-aperture views, each observing the same scene through an independent turbulence realization, which injects different turbulence-induced fluctuations across views while leaving the scene contribution identical.

Figure 3: Network architecture. Each sub-aperture event stream is converted to a voxel grid $E_{k}^{(i)}\in\mathbb{R}^{B\times H\times W}$ ; the $N$ grids are stacked along the channel dimension to form the joint input $\tilde{E}_{k}\in\mathbb{R}^{NB\times H\times W}$ . A recurrent convolutional encoder-decoder processes the stacked input window by window and outputs a reconstructed frame $\hat{V}_{k}$ .

Figure 4: Simulation pipeline. Turbulence is added to clean frames using a turbulence simulator based on the phase-to-space (P2S) transform [ 33 ] . V2E converts turbulent frames to events [ 20 ] . The process is repeated 9 times to simulate a $3\times 3$ light field. The network is trained on simulated data and generalizes to tabletop experimental settings.

Figure 5: Tabletop experimental setup. Real turbulence is generated from temperature fluctuations caused by candles and a space heater. We placed a beam splitter behind the kaleidoscope to simultaneously capture events and reference CMOS frames.

Figure 6: Light fields improve temporal consistency on experimental data. In the laboratory, we imaged a static checkerboard and a rotating dot, both displayed on an LCD monitor, through turbulence. The left four columns show reconstructed frames at $120$ fps. The right column visualizes $x$ - $t$ and $r$ - $t$ plots showing the time evolution the slice marked in yellow. The light field results in straighter lines, indicating less wander.

Method	Resolution	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ITF $\uparrow$	$E_{\text{warp}}$ $\downarrow$
Single-view	$768\times 768$	$14.84$	$0.501$	$0.5185$	$22.71$	$0.0050$
Single-view	$256\times 256$	$14.84$	$0.461$	$0.4607$	$22.72$	$0.0047$
Light field	$3\times 3\times 256\times 256$	$\mathbf{15.40}$	$\mathbf{0.505}$	$\mathbf{0.4332}$	$\mathbf{25.16}$	$\mathbf{0.0022}$

Table 1: Simulated quantitative results. On the REDS dataset, the light field improves frame quality in comparison to a single-view configuration.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ITF $\uparrow$	$E_{\text{warp}}\downarrow$
Single-view	$12.37$	$0.351$	$0.543$	$20.87$	$0.0085$
Light field	$\mathbf{12.99}$	$\mathbf{0.367}$	$\mathbf{0.531}$	$\mathbf{23.30}$	$\mathbf{0.0051}$

Table 2: Tabletop experiment quantitative results. Reconstructing REDS videos displayed through turbulence, light field outperforms single-view.

Figure 7: Reconstruction with light field input outperforms single-view on simulated and experimental data. Each group shows the input events (blue box indicates the central sub-aperture used for the single-view baseline), single-view reconstruction, the light field reconstruction, and the ground truth. Simulated single-view results are at $768\times 768$ resolution, which has the same number of input pixels as the $3\times 3$ light field whose views are simulated at $256\times 256$ . Experimental single-view and light field results are at $176\times 176$ , with the target videos displayed on an LCD monitor. The light field model recovers sharper edges with fewer artifacts than the single-view baselines at either resolution. The $x$ - $t$ plots below each group show the time evolution of the slice marked in yellow. Their relative smoothness shows that the light field reconstruction maintains a consistent appearance over time, while the single-view output exhibits frame-to-frame instability caused by turbulence. Videos in supplemental material more clearly illustrate turbulence reduction than images.

Figure 8: Simulated high-speed scenes show that event light fields recover sharper images than frame-based methods. Motion-blur cannot be compensated for by frame-based methods, which occurs when scene dynamics or turbulence are fast. Event-based light fields operate directly on the event stream and reconstruct the underlying scene without blur caused by integration. Motion-blur is simulated by aggregating 4 conventional frames into a single blurred frame.

Figure 9: Event-based light field design corrects turbulence in experimental high-speed regimes. In lab, we imaged a spinning reflective stripe (top) and a bouncing ball (bottom). A $600$ fps video is reconstructed, with samples shown four frames apart. Our method successfully reconstructs frame detail while correcting for wobble, as shown in the last column. The $\theta$ - $t$ and $x$ - $t$ plots show a $200$ ms interval. Videos in supplemental material more clearly illustrate turbulence reduction than images.

Figure 10: High-speed video reconstruction in experimental turbulent scenes with complex dynamics. In lab, our method recovers a water balloon bursting on top of pins at $600$ fps. The displayed images are spaced 5 frames apart. Please see the videos included in the supplemental material for a clearer comparison.

Figure 11: Experimental reconstruction of a Nerf dart at 12,000 fps. The first row shows a turbulence-free HSC capture, where a Nerf dart follows a straight trajectory. The second two rows show reconstructions of a dart taking a similar, but not identical, trajectory imaged through turbulence with the event camera. Compared with the single-view baseline, the light-field model more accurately reconstructs the straight path of a passing dart traveling at 16,000 pixels per second. The $y$ - $t$ plot shows 8.3 ms of data, with images spaced 16 frames apart.

Figure 12: Comparison with high-speed camera. With a $100k, high-speed camera MambaTM retains residual distortion, while ours, at $\sim$ 20 $\times$ lower cost, stays sharp. Note that the reference was captured separately from the turbulent HSC and event data, and therefore temporally post-aligned. — Figure 12: Comparison with high-speed camera. With a $100k, high-speed camera MambaTM retains residual distortion, while ours, at$ \sim $20$ \times$ lower cost, stays sharp. Note that the reference was captured separately from the turbulent HSC and event data, and therefore temporally post-aligned.

Method	Resolution	MambaTM Postprocess	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
E2VID+ [ 42 ]	$768\times 768$	✗	$14.51$	$0.493$	$0.5854$
	$256\times 256$	✗	$14.51$	$0.435$	$0.5258$
	$768\times 768$	✓	$14.48$	$0.489$	$0.5602$
	$256\times 256$	✓	$14.48$	$0.435$	$0.5040$
ET-Net [ 45 ]	$768\times 768$	✗	$13.65$	$0.485$	$0.5952$
	$256\times 256$	✗	$13.65$	$0.417$	$0.5383$

Table 3: Ablation of event-to-video methods. E2VID methods trained on clean events fail to handle turbulence. Postprocessing with conventional turbulence mitigation also fails to recover scene quality, confirming the importance of our approach.

Figure 13: Postprocessing event-to-video reconstructions with frame-based turbulence mitigation is insufficient. Frames reconstructed by event-to-video methods and subsequently postprocessed by MambaTM [ 56 ] retain significant geometric distortion and artifacts. The distortion arises because turbulence-induced events violate the fundamental assumption of event-to-video networks, producing corrupted reconstructions that MambaTM was not trained to handle. An end-to-end approach trained directly on turbulent event streams is necessary.

Figure 14: Contrast threshold estimation for simulated dataset. The average event rate (in $\text{events}\cdot\text{pix}^{-1}\cdot\text{s}^{-1}$ ) is computed from experimental light field event recordings and compared against synthetic light field event sequences generated with varying contrast threshold $C$ . The experimental event rate falls within the range of synthetic rates for $C\in[0.1,0.7]$ . The range is deliberately picked wider than the experimental estimate to improve the network’s robustness to contrast threshold variation across different scenes and lighting conditions.

Figure 15: Dot grid imaged through turbulence. (a) The $3\times 3$ dot grid displayed on the monitor, used as the measurement target. (b) Centroid position vs. time under turbulence (colored) and ground truth (GT, gray); the top and bottom rows show $x$ - and $y$ -displacement, respectively. (c) Example frame illustrating turbulence-induced tilt (red arrows, magnified $5\times$ for visibility) and blur (green ellipses). Frames are recorded at 60 fps with the video version shown in the webpage.

Figure 16: Turbulence statistics of our tabletop setup. We characterize turbulence-induced tilt and blur across time, space, and viewpoints. The $3\times 3$ dots are indexed in row-alternating (snake) order. (a) Tilt temporal autocorrelation decays within 0.08 s, confirming the turbulence is dynamic. (b) Nearby dots within the same view exhibit higher tilt correlation than distant ones, consistent with a finite spatial coherence length. (c) Tilt across sub-apertures viewing the same scene location is largely uncorrelated, confirming that each viewpoint experiences distinct degradation. (d) Blur $\bar{\sigma}$ fluctuates over time and is consistently larger than GT, confirming turbulence-induced PSF spreading. (e) Normalized intensity $I(t)/\langle I\rangle$ for GT and turbulence recordings. The scintillation index $\sigma^{2}_{I}=\langle I^{2}\rangle/\langle I\rangle^{2}-1$ is larger under turbulence, confirming measurable intensity fluctuations induced by the tabletop setup.

View	Native Resolution					Upsampled to Match SV
View	Res.	PSNR $\uparrow$	LPIPS $\downarrow$	ITF $\uparrow$	$E_{\text{warp}}$ $\downarrow$	Res.	PSNR $\uparrow$	LPIPS $\downarrow$	ITF $\uparrow$	$E_{\text{warp}}$ $\downarrow$
SV	$768$	$14.84$	$0.519$	$22.71$	$0.0050$	$768$	$14.84$	$\mathbf{0.519}$	$22.71$	$0.0050$
$3\times 3$	$256$	$15.40$	$\mathbf{0.433}$	$25.16$	$0.0022$	$768$	$\mathbf{15.19}$	$0.544$	$25.42$	$0.0022$
$5\times 5$	$160$	$\mathbf{15.41}$	$0.443$	$\mathbf{29.85}$	$\mathbf{0.0010}$	$768$	$15.14$	$0.606$	$\mathbf{30.11}$	$\mathbf{0.0010}$

Table 4: View-count vs. resolution trade-off in simulation , showing that greater view diversity improves video stability.

Panel of Figure 17 — Figure 17: Qualitative ablation of training objectives and regularization. Compared with our LPIPS-trained baseline, replacing LPIPS with MSE introduces visible blur due to event-only intensity ambiguity. Removing spectral normalization produces severe artifacts in textureless regions.

Atmospheric Turbulence Mitigation

See all 11 papers

Mar 21, 2026

High-Quality and Efficient Turbulence Mitigation with Events

Turbulence mitigation (TM) is highly ill-posed due to the stochastic nature of atmospheric turbulence. Most methods rely on multiple frames recorded by conventional cameras to capture stable patterns in natural scenarios. However, they inevitably suffer from a trade-off between accuracy and efficiency: more frames enhance restoration at the cost of higher system latency and larger data overhead. Event cameras, equipped with microsecond temporal resolution and efficient sensing of dynamic changes, offer an opportunity to break the bottleneck. In this work, we present EHETM, a high-quality and efficient TM method inspired by the superiority of events to model motions in continuous sequences. We discover two key phenomena: (1) turbulence-induced events exhibit distinct polarity alternation correlated with sharp image gradients, providing structural cues for restoring scenes; and (2) dynamic objects form spatiotemporally coherent ``event tubes'' in contrast to irregular patterns within turbulent events, providing motion priors for disentangling objects from turbulence. Based on these insights, we design two complementary modules that respectively leverage polarity-weighted gradients for scene refinement and event-tube constraints for motion decoupling, achieving high-quality restoration with few frames. Furthermore, we construct two real-world event-frame turbulence datasets covering atmospheric and thermal cases. Experiments show that EHETM outperforms SOTA methods, especially under scenes with dynamic objects, while reducing data overhead and system latency by approximately 77.3% and 89.5%, respectively. Our code is available at: https://github.com/Xavier667/EHETM.

Xiaoran Zhang, Jian Ding, Yuxing Duan +4

State Key Laboratory of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · School of Computer Science and Engineering, Sun Yat-sen University

Atmospheric Turbulence Mitigation Image Restoration

Ask PDF

Sep 17, 2026

An Event Preserving Velocity Invariant Representation for Event Cameras

Event cameras provide low-latency, high temporal resolution perception for real-time vision tasks such as robotics.The novel circuitry (i.e. asynchronous, independent pixels) that enables these advantages also introduces new algorithmic challenges. Velocity-invariant representations alleviate missing observations under slow motion and motion blur under fast motion, but most discard temporal information by converting events into image-like representations. We propose Set of Centre Active Receptive Fields (SCARF), a real-time velocity-invariant representation that preserves raw events while consistently handling fast motion, stationary scenes, and independently moving objects. SCARF achieves state-of-the-art performance in both computational efficiency and representation quality.

Mikihiro Ikura, Luna Gava, Jiahang Wu +2

Istituto Italiano di Tecnologia, Via Morego 30 16163 Genova, Italy

Event-Based Vision

Ask PDF

Apr 27, 2026

Event-based SLAM Benchmark for High-Speed Maneuvers

Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in high-speed maneuvering scenarios. Existing event-based approaches, although successful in mitigating motion blur caused by high-speed maneuvers, suffer from many limitations. Some of them highlight a success of pose tracking for a fronto-parallel fast shaking camera closed to the structure, while others assume pure (optionally aggressive) three-degree-of-freedom rotations. The former requires persistent local map visibility within the field of view (FOV), whereas the latter fails to generalize to six-degree-of-freedom (6-DoF) motions where both linear and angular velocities may be large. Consequently, current successes do not fully demonstrate that event-based state estimation under arbitrary aggressive maneuvers is a fully solved problem. To quantitatively assess the extent to which the potential of event cameras has been unlocked, we conduct a thorough analysis of state-of-the-art (SOTA) event-based visual odometry (VO)/visual-inertial odometry (VIO) methods and report shortcomings in current public datasets. Furthermore, we introduce a benchmarking framework for event-based state estimation, called EvSLAM, characterized by sufficient variation in data collection platforms, diverse extreme lighting scenarios, and a wide scope of challenging motion patterns under a clear and rigorous definition of high-speed maneuvers for mobile robots, along with a novel evaluation metric designed to fairly assess the operational limits of event-based solutions. This framework benchmarks state-of-the-art methods, yielding insights into optimal architectures and persistent challenges.

Sheng Zhong, Junkai Niu, Guillermo Gallego +7

Neuro-morphic Automation and Intelligence Lab (NAIL) at the School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · Department of Electrical Engineering and Computer Science, Technische Universität Berlin, the Science of Intelligence Excellence Cluster, the Robotics Institute Germany and the Einstein Center Digital Future, Berlin, Germany. · College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China. +2

Event-Based Vision Visual Odometry Visual-Inertial Odometry

Ask PDF

Abstract

Atmospheric Turbulence Mitigation

Explore similar work

High-Quality and Efficient Turbulence Mitigation with Events

An Event Preserving Velocity Invariant Representation for Event Cameras

Event-based SLAM Benchmark for High-Speed Maneuvers