Digital twins, virtual simulated replicas of physical scenes, are transforming system design across industries. However, their potential in radio frequency (RF) systems has been limited by the non-differentiable nature of conventional RF simulators. The visibility of propagation paths causes severe discontinuities, and differentiable rendering techniques from computer graphics cannot easily transfer due to point-source antennas and dominant specular reflections. In this paper, we present RFDT, a physically based differentiable RF simulation framework that enables gradient-based interaction between virtual and physical worlds. RFDT resolves discontinuities with a physically grounded edge-diffraction transition function, and mitigates non-convexity from Fourier-domain processing through a signal domain transform surrogate. Our implementation demonstrates RFDT's ability to accurately reconstruct digital twins from real RF measurements. Moreover, RFDT can augment diverse downstream applications, such as test-time adaptation of machine learning-based RF sensing and physically constrained optimization of communication systems.
Figures & tables
Figure 1. RFDT constructs digital twins by solving the inverse problem of RF simulation. Top: 3D reconstruction through radar simulation. Bottom: Communication system optimization through RF scene rendering.
Table 1. Comparison of RFDT with related works in Differentiable Rendering
Figure 2. Reparametrized RF Ray Tracing and RFDT Secondary Visibility.
Figure 3. Wedge discontinuity as Rx location translates from Prx0 to Prxt . S : continuity of changes in signal strength. C : total energy conservation during translation. Conventional: discontinuous step. Softened triangle: heuristic sigmoid, physically incorrect. Ours: continuous and physically correct—As the reflection point approaches the edge, the purely reflection contribution is smoothly redistributed into a diffraction component, as captured by the transition function F(⋅) .
Figure 4. RFDT ’s surrogate model provides a smoother loss landscape for faster convergence in the early stages, while progressively annealing to the conventional signal-domain transform for unbiased results.
Figure 5. RFDT forward simulation and backward propagation flow.
Figure 6. Replacing Neural Representation with DT.
Figure 7. Enhancing Neural Network with DT.
Figure 8. Field simulation (Real part, top) and gradient estimation (bottom) w.r.t. scene parameter θ .
Figure 9. Gradients of the field magnitude w.r.t. different scene parameters (object position, rotation, and Tx position) .
Figure 10. Snapshots of DT optimization process. Left: reconstruction of metal desktop objects using the Cascade radar; Right: reconstruction of a real-world vehicle using the Vayyar radar, without and with pre-trained knowledge of “possible shapes of vehicles”.
Figure 11. Reconstructing human posture and unknown object shape using radar signals.
Figure 12. Impact of the granularity of knowledge provided to RFDT when performing shape reconstruction.
Figure 13. Normalized Received Signal Strength (RSS) in a given scene. (a) The 3D-scanned environment and floor layout, with actual measurement points shown in the top-right inset. (b) Radio map simulated by Sionna (non-coherent). (c) Results generated by RFDT (coherent).
Figure 14. (a) Mean vertices error of DT reconstruction; (a) Loss history of the convergence for human posture.
Figure 15. (a) 3D reconstruction accuracy under different NLOS occlusions; (b) Impact of the number of objects per radar view.
Figure 16. (a) Impact of Material for DT Reconstruction; (b) Effectiveness of the Surrogate model.
Figure 17. (a) Impact of Ray Density under Scene Complexity; (b) Use case I: Enhancing generalization of ML-based sensing model.
Figure 18. Optimizing metasurface (MS) deployment to maximize coverage. Overview: 3D scanned mesh; Coverage: simulated signal strength at each location; Gradient: variation of signal strength w.r.t. MS translation.
Figure 19. (a) RF field simulation accuracy, w and w/o optimizing shape/material parameters in the digital twin; (b) Impact of scene dynamics simulation.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 20. RFDT estimates the gradient of the coverage map w.r.t. Tx translations θ for optimization.
Figure 22. Sionna results under different versions. Version 0.19 includes diffraction simulation, whereas version 1.10 has removed this feature. Both results significantly differ from the full-wave solution due to non-coherent combinations.
Explicit neural representations such as 3D Gaussian Splatting (3DGS) enable high-fidelity and real-time novel view synthesis, yet optimize for alpha-composited optical appearance rather than ray-intersectable geometry. In contrast, radio-frequency (RF) digital twins require deterministic multi-bounce paths, where the geometry dictates trajectories and their associated attenuation and delay. We introduce a framework enabling differentiable RF propagation simulation directly within visually reconstructed neural scenes, allowing point-to-point path computation between arbitrary 3D locations while preserving high-quality visual rendering. Unlike conventional RF simulation pipelines that rely on manually constructed meshes, we embed Gaussian primitives into a hardware-accelerated ray tracing structure as the underlying spatial representation. By extracting physically meaningful channel impulse responses from visual-only reconstructions, we provide cross-modal evidence that neural reconstructions can serve as unified spatial representations for both electromagnetic propagation simulation and photorealistic view synthesis.
Niklas Vaara, Lam Huynh, Pekka Sangi +2
Center for Machine Vision and Signal Analysis, University of Oulu, Finland · CubiCasa Oy, Finland
3D reconstruction typically strives for geometric fidelity or visual plausibility. Radio frequency digital twins (RFDT) are instead judged by whether communication channels behave in them as they do in the real world. RFDTs promise site-specific channel prediction but current practice forces a choice between coarse automated scenes and hand-built, measurement-calibrated models that take weeks to construct per-site. We present KODAMA, an automated pipeline that reconstructs ray tracing-ready RFDTs at city scale from off-the-shelf geospatial data alone: aerial imagery, LiDAR, and photogrammetry yield terrain and watertight building meshes, while exposure-weighted multi-view fusion of street-level imagery recovers façade relief, electromagnetic materials, and clutter---all without site visits or calibration. Across three sites spanning 3.6 to 28 GHz, KODAMA's uncalibrated predictions achieve single-digit RMSE, reducing point-to-point error by up to 5.35 dB over automated baselines and coming within 0.22 dB of a measurement-calibrated, hand-built RFDT.
Accurately modeling millimeter-wave (mmWave) propagation is essential for real-time AR and autonomous systems. Differentiable ray tracing offers a physics-grounded solution but still facing deployment challenges due to its over-reliance on exhaustive channel measurements or brittle, hand-tuned scene models for material properties. We present VisRFTwin, a scalable and data-efficient digital-twin framework that integrates vision-derived material priors with differentiable ray tracing. Multi-view images from commodity cameras are processed by a frozen Vision-Language Model to extract dense semantic embeddings, which are translated into initial estimates of permittivity and conductivity for scene surfaces. These priors initialize a Sionna-based differentiable ray tracer, which rapidly calibrates material parameters via gradient descent with only a few dozen sparse channel soundings. Once calibrated, the association between vision features and material parameters is retained, enabling fast transfer to new scenarios without repeated calibration. Evaluations across three real-world scenarios, including office interiors, urban canyons, and dynamic public spaces show that VisRFTwin reduces channel measurement needs by up to 10× while achieving a 59% lower median delay spread error than pure data-driven deep learning methods.
Zhenlin An, Longfei Shangguan, John Kaewell +5
University of Georgia · University of Pittsburgh · Interdigital +2