Towards Real-Time Full-Waveform LiDAR Transformers via Intensity-Guided Token Reduction and Physics-Aware Augmentation

Kotaro Oishi; Kazuma Ikeda; Ryosei Hara; Ryo Yoshida; Mariko Isogawa; Kentaro Yoshioka

cs.CVNov 5, 2026

Towards Real-Time Full-Waveform LiDAR Transformers via Intensity-Guided Token Reduction and Physics-Aware Augmentation

Authors: Kotaro Oishi, Kazuma Ikeda, Ryosei Hara, Ryo Yoshida, Mariko Isogawa, Kentaro Yoshioka

Organizations: Keio University

Abstract

LiDAR is a critical sensor for autonomous driving and robotics, yet it remains vulnerable to adverse conditions such as fog, rain, transparent objects, and low-reflectance surfaces. Full-waveform LiDAR (FWL) addresses these limitations by capturing the complete return waveform as a histogram, preserving rich reflection characteristics including multi-path echoes and weak signals. While recent Transformer-based approaches have demonstrated promise on FWL data, two fundamental bottlenecks remain: high computational cost from redundant tokens in low-intensity background regions, and limited generalization due to scarce annotated training data. We address both bottlenecks by exploiting the physical structure inherent to FWL data. First, we propose FWL-ToPM, a token pruning and merging mechanism that leverages intensity distributions to selectively reduce tokens while retaining critical waveform content. Second, we introduce FWLAug, a data augmentation framework designed as a regularization method that preserves waveform structure, retaining the Time-of-Flight waveform of each ray and the frustum geometry, to improve generalization to unseen scenes. Evaluated on the Ghost-FWL benchmark, FWL-ToPM achieves over 8 times speedup at real-time throughput alongside a nearly 9-point F1 gain over the prior state of the art, establishing the first real-time FWL Transformer that simultaneously improves accuracy. FWLAug further improves performance on both the standard benchmark and an out-of-distribution test set of unseen scenes. Together, these methods push FWL-based perception toward practical deployment. The code and the OOD dataset are publicly available and can be accessed via the project page: https://keio-csg.github.io/FWL-ToPM

Figures & tables

Figure 1: Overview of the proposed framework. (Top) During training, FWLAug augments FWL samples while preserving ToF waveform structure and frustum geometry. High-intensity patches are selected by pruning, spatially reordered along a space-filling curve, and processed by the FWL-ToPM Encoder, which merges redundant tokens before self-attention and unmerges afterward across $L$ Transformer blocks. (Bottom Left) Pruning removes low-intensity background tokens, and merging aggregates the remainder, concentrating computation on object-relevant waveform content. (Bottom Right) Proportion of pixel classes across intensity bins in FWL data. Most regions in FWL data correspond to noisy background, and our pruning strategy removes these dominant low-intensity regions.

Figure 2: Illustration of FWLAug. FWLShiftMix (top): an object in Sample 1 is shifted away from the LiDAR, creating a blank region $\Omega_{\mathrm{blank}}$ . A glass-containing region $\Omega_{\mathrm{src}}$ is then selected from Sample 2 and pasted into the blank, producing an augmented scene in which the waveform structure of each ray is preserved. FWLCutMix (bottom): a crop region $\Omega_{\mathrm{src}}$ containing minority class (glass) is selected from a source frame and pasted into a destination frame at a random location. Both methods prioritize glass-containing regions to mitigate class imbalance.

Method	Pruning	Merging	F1-mean	FPS
Scheuble et al. [ 31 ]	–	–	0.497	1.97
Ikeda et al. [ 10 ]	–	–	0.503	6.54
Ours w/o FWL-ToPM	–	–	0.555	2.01
DynamicViT [ 30 ]	✓	–	0.480	20.0
Ours w/Pruning	✓	–	0.591	20.3
ToMe [ 1 ]	–	✓	0.543	5.24

Table 1: Comparison with FWL-based methods and ablation of token pruning and merging components. For FWL-ToPM, we set

\rho_{\mathrm{low}}=0.7

and

\rho_{m}=0.9

. FWL-ToPM achieves the highest F1 and improves throughput by 8

\times

over the prior state of the art. Our pruning and merging components each outperform their image-based counterparts (DynamicViT and ToMe) and their combination achieves the best F1–FPS trade-off. All methods in this table are trained without FWLAug.

Figure 3: (Left) F1-mean vs. FPS trade-off for all compared methods. (Right) Comparison of F1-mean and FPS without additional training. For pruning and merging,

\rho_{\mathrm{low}}

and

\rho_{m}

are varied from 0 to 0.9, respectively. For pruning+merging,

\rho_{\mathrm{low}}

is fixed to 0.7 while

\rho_{m}

is varied. All models are trained without FWLAug.

Figure 4: Visualization of token merging on two real Ghost-FWL scenes. Color indicates patch intensity (red: high, white: low). After pruning low-intensity background tokens, the remaining high-intensity tokens are merged by aggregating similar-feature source tokens into destination tokens, reducing the total token count while preserving object-relevant content. The zoomed-in regions show that merged tokens retain the spatial structure of reflected objects across diverse scenes.

Method	FWLCutMix	FWLShiftMix	Ghost-FWL [ 10 ]		OOD Test
Method	FWLCutMix	FWLShiftMix	F1-mean	Glass	F1-mean	Glass
Ours w/o Aug	✗	✗	0.592	0.352	0.406	0.491
Ours w/ FWLAug	✗	✓	0.622	0.376	0.425	0.494
	✓	✓	0.617	0.385	0.452	0.575
	✓	✗	0.586	0.315	0.434	0.606

Table 2: Ablation study of FWLAug. To evaluate out-of-distribution generalization, we construct an OOD test set captured at a location entirely separate from Ghost-FWL [ 10 ] . The full FWLAug achieves the best F1 on the OOD test set, particularly for the minority Glass class, confirming improved generalization to unseen scenes.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Our OOD Test Set. Based on the dense 3D maps as shown in Scene, we annotated FWL data with 4 labels: Ghost (red), Object (green), Glass (blue), Noise. Blue crosses indicate the LiDAR positions during data acquisition. The RGB images show the scenery of the capture locations.

Method	F1-mean	Object-F1	Glass-F1	Ghost-F1	FPS
Scheuble et al. [ 31 ]	0.497 $\pm$ 0.043	0.648	0.304	0.541	1.97
Ikeda et al. [ 10 ]	0.503 $\pm$ 0.005	0.680	0.268	0.561	6.54
Ours	0.592 $\pm$ 0.005	0.741	0.358	0.678	53.2

Appendix

Table 1: Statistical significance of the main comparison with FWL-based methods. F1-mean is reported as mean

\pm

standard deviation over 3 independent runs with different random seeds. All methods are trained without FWLAug.

Figure 2: Waveforms along single rays in Ghost-FWL Scene 001 (left) and Scene 007 (right). Markers indicate annotated Object (green), Glass (blue), and Ghost (red) bins, labels indicate the Top-3 highest-intensity peaks of each ray, and the dashed line indicates the intensity threshold below which patches are pruned at $\rho_{\mathrm{low}}=0.7$ . The photographs show the corresponding scenes, with the red marker indicating the observed ray. Top-3 peak extraction discards the Object and the Glass return, whereas our intensity-based pruning retains both because they lie above the pruning threshold.

Setting	Curve	Object-F1	Glass-F1	Ghost-F1	F1-mean
Merging	Hilbert	0.690	0.349	0.652	0.564
Merging	Morton (Ours)	0.702	0.369	0.624	0.565
Pruning + Merging	Hilbert	0.740	0.355	0.672	0.589
Pruning + Merging	Morton (Ours)	0.741	0.358	0.678	0.592

Appendix

Table 2: Comparison of space-filling curves used to serialize tokens for merging. F1 scores on the Ghost-FWL test set are reported. We set

\rho_{\mathrm{low}}=0.7

(pruning + merging only),

\rho_{m}=0.9

, and

\rho_{d}=0.1

. All models are trained without FWLAug.

Figure 3: Qualitative comparison of classification results with existing FWL-based recognition methods.

Autonomous Driving Perception

Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.

See all 397 papers

Ghost-FWL: A Large-Scale Full-Waveform LiDAR Dataset for Ghost Detection and Removal
Mar 30, 2026Kazuma Ikeda, Ryosei Hara, Rokuto Nagata +6Point Cloud Anomaly Detection
Keio University · Sony Semiconductor Solutions
Neural Reconstruction of LiDAR Point Clouds under Jamming Attacks via Full-Waveform Representation and Simultaneous Laser Sensing
Apr 1, 2026Ryo Yoshida, Takami Sato, Wenlun Zhang +7Point Cloud Learning Point Cloud Surface Reconstruction
Keio University, Yokohama, Japan · Sony Semiconductor Solutions, Atsugi, Japan · Hainan University, Haikou, China
ATLAS: A Large-Scale Evaluation Benchmark for Adversarial LiDAR Perception
Jun 1, 2026Mellon M. Zhang, Siddhant Panse, Zimo Fan +3Lidar-Based 3D Object Detection Adversarial Attacks
Georgia Institute of Technology

Mar 30, 2026J/K move · Enter open · S save

Ghost-FWL: A Large-Scale Full-Waveform LiDAR Dataset for Ghost Detection and Removal

Kazuma Ikeda, Ryosei Hara, Rokuto Nagata, Ozora Sako. Zihao Ding+5

Keio University · Sony Semiconductor Solutions

Point Cloud Anomaly Detection

Open paper AskPDF arXiv

LiDAR has become an essential sensing modality in autonomous driving, robotics, and smart-city applications. However, ghost points (or ghosts), which are false reflections caused by multi-path laser returns from glass and reflective surfaces, severely degrade 3D mapping and localization accuracy. Prior ghost removal relies on geometric consistency in dense point clouds, failing on mobile LiDAR's sparse, dynamic data. We address this by exploiting full-waveform LiDAR (FWL), which captures complete temporal intensity profiles rather than just peak distances, providing crucial cues for distinguishing ghosts from genuine reflections in mobile scenarios. As this is a new task, we present Ghost-FWL, the first and largest annotated mobile FWL dataset for ghost detection and removal. Ghost-FWL comprises 24K frames across 10 diverse scenes with 7.5 billion peak-level annotations, which is 100x larger than existing annotated FWL datasets. Benefiting from this large-scale dataset, we establish a FWL-based baseline model for ghost detection and propose FWL-MAE, a masked autoencoder for efficient self-supervised representation learning on FWL data. Experiments show that our baseline outperforms existing methods in ghost removal accuracy, and our ghost removal further enhances downstream tasks such as LiDAR-based SLAM (66% trajectory error reduction) and 3D object detection (50x false positive reduction). The dataset and code is publicly available and can be accessed via the project page: https://keio-csg.github.io/Ghost-FWL

Abstract

Autonomous Driving Perception

Explore similar work

Ghost-FWL: A Large-Scale Full-Waveform LiDAR Dataset for Ghost Detection and Removal

Neural Reconstruction of LiDAR Point Clouds under Jamming Attacks via Full-Waveform Representation and Simultaneous Laser Sensing

ATLAS: A Large-Scale Evaluation Benchmark for Adversarial LiDAR Perception