Organizations: School of Data Science and Society University of North Carolina at Chapel Hill Chapel Hill, NC, USA · The Pennsylvania State University State College, PA, USA · Google DeepMind
Full Waveform Inversion (FWI) is a highly nonlinear and ill-posed problem that aims to recover subsurface velocity maps from surface-recorded seismic waveforms data. Existing data-driven FWI typically uses small models, as available datasets have limited volume, geological diversity, and spatial extent, leading to substantial concerns about overfitting. Although they perform well on synthetic datasets, current methods fail to generalize to more realistic geological structures. In this work, we show that a model trained entirely on simulated and relatively simple data can generalize remarkably well to challenging and unseen geological benchmarks. We provide a working recipe that tames a billion-parameter model for FWI through coordinated scaling across three axes: model capacity, data diversity, and training strategy. Our model achieves state-of-the-art performance on OpenFWI and significantly narrows the generalization gap in data-driven FWI. Across six challenging geophysical benchmarks, including Marmousi, 2D SEG/EAGE Salt and Overthrust, 2004 BP, Sigsbee, and SEAM Phase I, it infers complex structures absent from the training set and delivers significant performance improvements (SSIM from 0.5844 to 0.7669). Overall, our results demonstrate that with an appropriate scaling strategy, large models trained on simple synthetic data can achieve substantial generalization to more complex and realistic geological structures.
Figures & tables
Figure 1: Reconstruction results on challenging realistic benchmarks. BigFWI tends to collapse toward over-smoothed, mean-shaped solutions that miss key interfaces and salt bodies. In contrast, our model produces sharper boundaries and geologically meaningful structures.
Figure 2: Overview of progressive model enhancements and final reconstruction quality. (a) Step-wise performance improvement as each component is introduced—from baseline to data augmentation, non-causal modeling, ViT VQGAN (without bottleneck), reinforcement learning alignment, and latent gradient refinement. (b) Illustration of the final reconstruction results, showing sharp structures and fine geological details across diverse scenarios.
Model
MAE ↓
#Param
BigFWI-M ( Jin et al., 2024 )
0.0437
28M
BigFWI-L ( Jin et al., 2024 )
0.0432
87M
Causal
Baseline
0.0770
1.09B
+ Data x2
0.0463
1.09B
+ Data x10
0.0368
1.09B
Non-Causal
Data x2
0.0330
1.09B
Table 1: Quantitative comparison with baseline methods, on OpenFWI testset. Models are organized incrementally, with each row cumulatively including components introduced in prior rows.
Figure 3: Qualitative comparison of reconstructed velocity maps.
Figure 4: Qualitative comparison of reconstructed velocity maps. Our method produces sharper and finer structures.
Metrics
Model
Overall
2004 BP
Marmosi
SEAM Phase I
Overthrust
Salt
Sigsbee
MAE ↓
BigFWI-M
199.20
247.15
226.63
185.02
221.57
150.13
431.60
BigFWI-L
210.99
238.31
233.22
190.84
243.25
159.07
398.08
Ours
139.81
133.45
145.62
199.44
118.25
155.22
148.40
RMSE ↓
BigFWI-M
312.15
390.30
346.13
289.37
297.96
281.57
715.32
BigFWI-L
327.79
377.88
349.42
294.67
320.46
301.74
656.29
Ours
241.78
247.61
251.12
281.57
199.76
268.67
353.37
Table 2: Quantitative performance across six benchmark families. Our method achieves substantially lower MAE/RMSE and higher SSIM, with especially strong gains on SSIM.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model Configuration
MAE ↓
Causal + Data x2
0.0463
Causal + Data x2 + RL
0.0344
Non-Causal + Data x2
0.0330
Non-Causal + Data x2 + RL
0.0295
Non-Causal + Data x2 + ViT-VQGAN
0.0167
Non-Causal + Data x2 + ViT-VQGAN + RL
0.0159
Appendix
Table 3: Qualitative comparison of reinforcement learning across different model configurations.