Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems
Authors: Charles Dove, Laura Waller
Organizations: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, California 94720, United States
Neural network surrogates are an emerging alternative to traditional electromagnetic wave simulators like finite-difference time-domain (FDTD); their goal is to replace rigorous physical simulations with pre-trained neural networks that solve wave-scattering forward and inverse problems orders of magnitude faster. However, nonrecurrent, single-step surrogates have scaled only to a few tens of simulation variables. Here, we show that this barrier can be overcome by dynamically generating salient training examples during training, rather than randomly sampling the large space of possible examples. We introduce an algorithm that runs in parallel with surrogate training, using gradient ascent to search refractive-index and source configurations for cases where the surrogate disagrees with a full-wave ground-truth simulator. We also use source and ground-truth normalization with an evolving replay dataset to stabilize and accelerate learning. Using this approach, we train a fast, single-step surrogate for two-dimensional wave scattering with up to 41,772 controllable variables, including dense, freely configurable grids of refractive indices and complex-valued sources. The resulting neural surrogate is robustly accurate across diverse structured and unstructured examples and generalizes inductively to larger domains, reaching over 3 million controllable variables without retraining, a 73.8× increase. We demonstrate the surrogate on large-scale forward simulations and inverse design of freeform beam splitters and gradient-index (GRIN) lenses up to 98 wavelengths wide, showing comparable or better performance than FDTD-based designs, with speedups from 1.29× to 26.5×. These results demonstrate a practical path toward fast, robustly accurate, inductively scalable neural simulators for photonic inverse design and other wave-scattering inverse problems.
Figures & tables
Figure 1: Workflow for dynamic training data generation and neural surrogate training : (A) A dynamic dataset generation process searches for training data examples that maximize disagreement between the current neural surrogate and a ground-truth Finite-Difference Time-Domain (FDTD) output. These high-loss examples (consisting of refractive-index (n) and source (s) configurations) where the neural surrogate performs poorly are then added to the dynamic training dataset, along with their ground-truth FDTD output (E). The neural surrogate is periodically replaced by its updated version from the bottom process. (B) In parallel, the dynamic dataset elements are used to train the neural surrogate. As the surrogate improves, the dataset generation process discovers new failure modes, creating increasingly informative training examples.
Figure 2: Our dynamic training approach (orange) performs significantly better than random sampling of the training examples (blue), with respect to validation loss. We also demonstrate the importance of each component of our method by showing results for our proposed dynamic sampling method without the loss normalization (green) and without using a buffer of training examples (pink), i.e. not using the dynamic dataset and instead training immediately on generated examples.
Figure 3: Dynamically generating hard example training data improves neural surrogate performance. Here we show a diverse collection of challenging examples of source and refractive index configurations. We simulate the electric field ground truth with FDTD, then compare with the neural surrogate output either trained using our dynamic hard-example generation approach or traditional random sampling. Each example shows the refractive index map, as well as the magnitude and phase of the source, ground truth (FDTD) field, surrogate predictions, and error. To make maps more comparable across examples with different total intensities, error values report complex-field error normalized by the root-mean-square (RMS) intensity of the ground truth field.
Figure 4: High-loss examples generated by our dynamic dataset generation process. (Top) An example from early in training, after 500 training steps. (Bottom) An example from late in training, after 200,000 training steps. High-loss examples are rarer, become more structurally complex, and exhibit higher refractive index contrasts as training progresses, and the surrogate’s error on the identified high-loss examples diminishes as it becomes more robustly accurate over the course of training.
Figure 5: Our method inductively scales to simulation sizes much larger than it was trained on. Here, we show an example simulation of a high-contrast binary structure at increasing grid sizes, covering a progressively larger area. For each scale, we show amplitude and phase of the refractive index/source inputs, the FDTD ground truth field, our neural surrogate prediction, and the normalized prediction error. MSE values report ground-truth-RMS-normalized complex-field error.
Figure 6: Inverse design of freeform GRIN lenses, demonstrating accuracy and inductive scalability of surrogate. Top: a 512×512 simulation designs an approximately 49λ -wide gradient-index (GRIN) lens that focuses a plane wave to a single point. Bottom: a 1024×1024 simulation designs an approximately 98λ -wide GRIN lens that splits the plane wave into a periodic array of focal spots. For each case, surrogate- and FDTD-optimized designs are compared by refractive index, predicted E-field at the end of optimization (from the neural surrogate model or FDTD respectively), the FDTD-validated E-field of the finished designs, focal-plane cut, optimization history, and matched zoomed sections of the GRIN lenses.
Figure 7: Inverse design of a waveguide beam splitter. A 128×128 simulation designs an 8λ×8λ freeform junction that routes a left-input waveguide mode equally into top and bottom output waveguides. Surrogate- and FDTD-optimized designs are compared by refractive index, optimization field, FDTD-validated field, guided-mode output split, and mode-coupled power during optimization.
Task
Grid
Steps
NN (s)
FDTD (s)
Speedup
FDTD-validated result, NN / FDTD
Single-focus GRIN lens
5122
800
92.2
474
5.14×
Focal intensity: 1.67×103 / 1.84×103
Periodic-focus GRIN lens
10242
800
372
479
1.29×
Mean target intensity: 1.22×103 / 1.15×103
50:50 beam splitter
1282
400
8.97
238
26.5×
Total mode power: 10.55 / 10.24; Split: 51.2/48.8% / 49.9/50.1%
Table 1: FDTD-validated inverse-design performance and optimization timing. Optimization time includes only the recorded optimization loop and excludes final FDTD validation/evaluation, figure writing, model compilation, warmup, and other script overhead. Speedup is defined as tFDTD/tNN .
Designing an effective electromagnetic inverse-scattering solver requires faithful enforcement of nonlinear full-wave physics together with an expressive prior on the unknown permittivity contrast. We propose ScoreField, a neural inverse scattering framework that integrates coupled implicit neural representations (INRs) with a pretrained score-based generative prior. ScoreField employs two INRs to parameterize the permittivity contrast and the induced current fields, and jointly optimize them under the Lippmann-Schwinger equations. In addition to the implicit regularization by the INR architecture, the score model provides a learned prior gradient on the contrast, which is propagated to the contrast INR through the chain rule. This formulation enables ScoreField to effectively handle strong multiple scattering, where nonlinear wave interactions require accurate modeling of the coupled full-wave physics. We evaluate ScoreField on simulated weak- and strong-scattering benchmarks, the canonical Austria phantom, and experimental Fresnel measurements. We note that ScoreField significantly improves reconstruction fidelity and suppresses artifacts relative to classical full-wave methods and deep learning baselines, achieving an average PSNR improvement of 1.8dB over the best competing method on real Fresnel data.
Wenhan Guo, Yuan Gao, Yu Sun
Hopkins Computational Imaging Group · Johns Hopkins University
Neural surrogates provide computationally efficient auto-regressive modeling of wave propagation, enabling e.g. fast inverse solutions. Despite recent progress, the current surrogates are not yet sufficient for the needs of ultrasonic sensing, characterized by high frequency and heterogeneous propagation media. We show how methods building on Fourier Neural Operators (FNO) fail to capture essential signal features, such as reflections, whereas U-Net -based methods learn the wave fields well but lack the ability to transfer information across different spatial scales. We then propose a new surrogate that captures the best of the two worlds, by integrating FNOs and local-integral resampling into a U-Net architecture. We show it can be trained on limited data, and that the predictions are robust to input noise and work across spatial scales.
Burla Nur Korkmaz, Petro Moilanen, Ari Salmi +2
Department of Computer Science, University of Helsinki, Finland
Neural surrogates for Partial Differential Equations (PDEs) are usually trained on synthetic data from numerical simulators. As models grow and scaling laws demand larger datasets, the cost of data generation often outweighs the training cost and becomes the bottleneck in the scaling quest. Unlike data in other domains, however, each PDE sample has a tunable cost that depends on its numerical fidelity, with the resulting cost-accuracy trade-off governed by known convergence rates. We exploit this property by formalizing neural surrogate training as a function of both data quantity and fidelity under a fixed data generation compute budget, treating fidelity as a fine-grained discrete spectrum rather than a binary distinction. Building on this, we show that multi-fidelity training enables Pareto-optimal error scaling against data generation compute. By using simulations at lower fidelities as an additional training signal, training on the optimal fidelity mixture matches the accuracy of high-fidelity-only training, with up to 6.3× less data generation compute across our settings. We validate these findings across six PDE problems spanning structured grids and unstructured meshes, two distinct error sources (discretization and iterative-solver error), and four surrogate architectures, totaling over 7,000 training runs.
Paul Setinek, Fabian Paischer, Nils Thuerey +2
Institute for Machine Learning, JKU Linz · Mistral AI · Technical University of Munich +1