cs.CVMar 14, 2026
SaveHigh-speed Imaging through Turbulence with Event-based Light Fields
Organizations: University of Maryland, College Park MD 20742, USA · Dartmouth College, Hanover NH 03755, USA
Abstract
This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second.
Figures & tables
Figure 1: Method overview. We introduce an event-based light field camera to image through turbulence at high speed. The event camera avoids motion blur by detecting brightness changes at microsecond time resolution. The light field offers cross-view constraints on the scene.
Figure 2: Image model for event-based light field camera under turbulence. A light field camera captures simultaneous sub-aperture views, each observing the same scene through an independent turbulence realization, which injects different turbulence-induced fluctuations across views while leaving the scene contribution identical.
Figure 3: Network architecture. Each sub-aperture event stream is converted to a voxel grid ; the grids are stacked along the channel dimension to form the joint input . A recurrent convolutional encoder-decoder processes the stacked input window by window and outputs a reconstructed frame .
Figure 4: Simulation pipeline. Turbulence is added to clean frames using a turbulence simulator based on the phase-to-space (P2S) transform [ 33 ] . V2E converts turbulent frames to events [ 20 ] . The process is repeated 9 times to simulate a light field. The network is trained on simulated data and generalizes to tabletop experimental settings.
Figure 5: Tabletop experimental setup. Real turbulence is generated from temperature fluctuations caused by candles and a space heater. We placed a beam splitter behind the kaleidoscope to simultaneously capture events and reference CMOS frames.
Figure 6: Light fields improve temporal consistency on experimental data. In the laboratory, we imaged a static checkerboard and a rotating dot, both displayed on an LCD monitor, through turbulence. The left four columns show reconstructed frames at fps. The right column visualizes - and - plots showing the time evolution the slice marked in yellow. The light field results in straighter lines, indicating less wander.
| Method | Resolution | PSNR | SSIM | LPIPS | ITF | |
|---|---|---|---|---|---|---|
| Single-view | ||||||
| Single-view | ||||||
| Light field |
Table 1: Simulated quantitative results. On the REDS dataset, the light field improves frame quality in comparison to a single-view configuration.
| Method | PSNR | SSIM | LPIPS | ITF | |
|---|---|---|---|---|---|
| Single-view | |||||
| Light field |
Table 2: Tabletop experiment quantitative results. Reconstructing REDS videos displayed through turbulence, light field outperforms single-view.
Figure 7: Reconstruction with light field input outperforms single-view on simulated and experimental data. Each group shows the input events (blue box indicates the central sub-aperture used for the single-view baseline), single-view reconstruction, the light field reconstruction, and the ground truth. Simulated single-view results are at resolution, which has the same number of input pixels as the light field whose views are simulated at . Experimental single-view and light field results are at , with the target videos displayed on an LCD monitor. The light field model recovers sharper edges with fewer artifacts than the single-view baselines at either resolution. The - plots below each group show the time evolution of the slice marked in yellow. Their relative smoothness shows that the light field reconstruction maintains a consistent appearance over time, while the single-view output exhibits frame-to-frame instability caused by turbulence. Videos in supplemental material more clearly illustrate turbulence reduction than images.
Figure 8: Simulated high-speed scenes show that event light fields recover sharper images than frame-based methods. Motion-blur cannot be compensated for by frame-based methods, which occurs when scene dynamics or turbulence are fast. Event-based light fields operate directly on the event stream and reconstruct the underlying scene without blur caused by integration. Motion-blur is simulated by aggregating 4 conventional frames into a single blurred frame.
Figure 9: Event-based light field design corrects turbulence in experimental high-speed regimes. In lab, we imaged a spinning reflective stripe (top) and a bouncing ball (bottom). A fps video is reconstructed, with samples shown four frames apart. Our method successfully reconstructs frame detail while correcting for wobble, as shown in the last column. The - and - plots show a ms interval. Videos in supplemental material more clearly illustrate turbulence reduction than images.
Figure 10: High-speed video reconstruction in experimental turbulent scenes with complex dynamics. In lab, our method recovers a water balloon bursting on top of pins at fps. The displayed images are spaced 5 frames apart. Please see the videos included in the supplemental material for a clearer comparison.
Figure 11: Experimental reconstruction of a Nerf dart at 12,000 fps. The first row shows a turbulence-free HSC capture, where a Nerf dart follows a straight trajectory. The second two rows show reconstructions of a dart taking a similar, but not identical, trajectory imaged through turbulence with the event camera. Compared with the single-view baseline, the light-field model more accurately reconstructs the straight path of a passing dart traveling at 16,000 pixels per second. The - plot shows 8.3 ms of data, with images spaced 16 frames apart.
Figure 12: Comparison with high-speed camera. With a \sim\times$ lower cost, stays sharp. Note that the reference was captured separately from the turbulent HSC and event data, and therefore temporally post-aligned.
| Method | Resolution | MambaTM Postprocess | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|
| E2VID+ [ 42 ] | ✗ | ||||
| ✗ | |||||
| ✓ | |||||
| ✓ | |||||
| ET-Net [ 45 ] | ✗ | ||||
| ✗ |
Table 3: Ablation of event-to-video methods. E2VID methods trained on clean events fail to handle turbulence. Postprocessing with conventional turbulence mitigation also fails to recover scene quality, confirming the importance of our approach.
Figure 13: Postprocessing event-to-video reconstructions with frame-based turbulence mitigation is insufficient. Frames reconstructed by event-to-video methods and subsequently postprocessed by MambaTM [ 56 ] retain significant geometric distortion and artifacts. The distortion arises because turbulence-induced events violate the fundamental assumption of event-to-video networks, producing corrupted reconstructions that MambaTM was not trained to handle. An end-to-end approach trained directly on turbulent event streams is necessary.
Figure 14: Contrast threshold estimation for simulated dataset. The average event rate (in ) is computed from experimental light field event recordings and compared against synthetic light field event sequences generated with varying contrast threshold . The experimental event rate falls within the range of synthetic rates for . The range is deliberately picked wider than the experimental estimate to improve the network’s robustness to contrast threshold variation across different scenes and lighting conditions.
Figure 15: Dot grid imaged through turbulence. (a) The dot grid displayed on the monitor, used as the measurement target. (b) Centroid position vs. time under turbulence (colored) and ground truth (GT, gray); the top and bottom rows show - and -displacement, respectively. (c) Example frame illustrating turbulence-induced tilt (red arrows, magnified for visibility) and blur (green ellipses). Frames are recorded at 60 fps with the video version shown in the webpage.
Figure 16: Turbulence statistics of our tabletop setup. We characterize turbulence-induced tilt and blur across time, space, and viewpoints. The dots are indexed in row-alternating (snake) order. (a) Tilt temporal autocorrelation decays within 0.08 s, confirming the turbulence is dynamic. (b) Nearby dots within the same view exhibit higher tilt correlation than distant ones, consistent with a finite spatial coherence length. (c) Tilt across sub-apertures viewing the same scene location is largely uncorrelated, confirming that each viewpoint experiences distinct degradation. (d) Blur fluctuates over time and is consistently larger than GT, confirming turbulence-induced PSF spreading. (e) Normalized intensity for GT and turbulence recordings. The scintillation index is larger under turbulence, confirming measurable intensity fluctuations induced by the tabletop setup.
| View | Native Resolution | Upsampled to Match SV | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Res. | PSNR | LPIPS | ITF | Res. | PSNR | LPIPS | ITF | |||
| SV | ||||||||||
Table 4: View-count vs. resolution trade-off in simulation , showing that greater view diversity improves video stability.
Figure 17: Qualitative ablation of training objectives and regularization. Compared with our LPIPS-trained baseline, replacing LPIPS with MSE introduces visible blur due to event-only intensity ambiguity. Removing spectral normalization produces severe artifacts in textureless regions.