Organizations: School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University, Shenzhen 518107, China · School of Computing, National University of Singapore, 117417, Singapore · School of Intelligent Systems Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen 518107, China
Physical adversarial camouflage poses a severe security threat to autonomous driving systems by mapping adversarial textures onto 3D objects. Nevertheless, current methods remain brittle in complex dynamic scenarios, failing to generalize across diverse geometric (e.g., viewing configurations) and radiometric (e.g., dynamic illumination, atmospheric scattering) variations. We attribute this deficiency to two fundamental limitations in simulation and optimization. First, the reliance on coarse, oversimplified simulations (e.g., via CARLA) induces a significant domain gap, confining optimization to a biased feature space. Second, standard strategies targeting average performance result in a rugged loss landscape, leaving the camouflage vulnerable to configuration shifts.To bridge these gaps, we propose the Relightable Physical 3D Gaussian Splatting (3DGS) based Attack framework (R-PGA). Technically, to address the simulation fidelity issue, we leverage 3DGS to ensure photo-realistic reconstruction and augment it with physically disentangled attributes to decouple intrinsic material from lighting. Furthermore, we design a hybrid rendering pipeline that leverages precise Relightable 3DGS for foreground rendering, while employing a pre-trained image translation model to synthesize plausible relighted backgrounds that align with the relighted foreground.To address the optimization robustness issue, we propose the Hard Physical Configuration Mining (HPCM) module, designed to actively mine worst-case physical configurations and suppress their corresponding loss peaks. This strategy not only diminishes the overall loss magnitude but also effectively flattens the rugged loss landscape, ensuring consistent adversarial effectiveness and robustness across varying physical configurations.
Figures & tables
Fig. 1: Demonstration of the framework of R-PGA. It consists of the High-Fidelity Relightable Scene Simulator and the HPCM Module. The simulator features a reconstruction and rendering pipeline composed of the Physically Disentangled Reconstruction Module and the Hybrid Rendering Module. The HPCM module constructs a discretized Configuration Space to build a Global Difficulty Table, which is utilized to select physical configurations during each iteration.
Fig. 2: Illustration of cross-view texture inconsistency observed when generating adversarial camouflage using vanilla 3DGS. This issue stems from the fact that lighting information, which is baked into the Spherical Harmonics (SH) coefficients and should remain invariant, is erroneously modified during the iterative optimization process.
Fig. 3: Detection heatmaps (Red: Detected, Blue: Evaded) across Azimuth and Pitch angles, averaged over other configuration dimensions and multiple detectors, and three EoT-based SOTA methods exhibit similar failure regions.
Method
One-Stage
Two-Stage
Transformer-based
Average
Yolo-V3
YoloX*
FrRCN*
MkRCN*
D-DETR*
PVT*
ORI
0.5251
0.7395
0.5712
0.6270
0.5877
0.7312
0.6126
FCA
0.4982
0.6550
0.3511
0.4035
0.4070
0.6875
0.4758
ACTIVE
0.1934
0.3749
0.1460
0.1841
0.2175
0.4160
0.2376
DTA
0.3462
0.4069
0.2292
0.3143
0.2924
0.5600
0.3240
RAUCA
0.1869
0.3643
0.1474
0.1725
0.1059
0.5081
0.2256
TABLE I: Comparison of [email protected] for different physical attack methods against various detection models. The reported results represent the average detection performance on the test dataset collected across multiple viewpoints, shooting distances, and weather conditions. Note that the adversarial camouflage is generated using Yolo-V3 and evaluated for black-box transferability (marked with *) on Yolo-X, Faster R-CNN, Mask R-CNN, Deformable-DETR and PVT.
Fig. 4: Comparison of detection results for different attack methods across various weather conditions, specifically reporting the average [email protected] for different detectors averaged over diverse shooting distances and pitch angles.
Fig. 5: Comparison of detection results for different attack methods across various shooting distances, specifically reporting the average [email protected] for different detectors averaged over diverse weather conditions and pitch angles.
Fig. 6: Comparison of detection results for different attack methods across various pitch angles, specifically reporting the average [email protected] for different detectors averaged over diverse weather conditions and distances.
Fig. 7: Comparison of detection results for different attack methods across various azimuth angles, specifically reporting the average [email protected] for different detectors averaged over diverse weather conditions and distances.
Fig. 8: Visual comparison of loss landscapes for different attack methods across various physical configurations. For visualization clarity, we report the mean results averaged over weather conditions, distances, and target detectors.
Fig. 9: Visual comparison of detection results across different methods in the digital domain. We display camouflaged vehicles captured under diverse configurations, including varying pitch angles, azimuths, distances, and weather conditions. Green bounding boxes indicate correct detections (attack failure), while red bounding boxes denote incorrect detections (successful evasion).
ORI
FCA
DTA
ACTIVE
RAUCA
RAUCA-E2E
GCAC
GRAC
PGA
R-PGA
Yolo-V3
0.5251
0.4982
0.1934
0.3462
0.1869
0.1207
0.1515
0.1121
0.0318
0.0227
DINO*
0.8993
0.8903
0.8642
0.7872
0.8460
0.7674
0.7928
0.8778
0.8324
0.7421
GLIP*
0.9387
0.8732
0.7702
0.6970
0.7144
0.7037
0.6886
0.7589
0.7256
0.6749
TABLE II: Comparison of [email protected] for different physical attack methods against vision foundation models. The reported results represent the average detection performance on the test dataset collected across multiple viewpoints, shooting distances, and weather conditions. Note that the adversarial camouflage is generated using Yolo-V3 and evaluated for black-box transferability (marked with *) on GLIP and DINO
Relit
HR
HPCM
One-Stage
Two-Stage
Transformer-based
Average
Yolo-V3
YoloX*
FrRCN*
MkRCN*
D-DETR*
PVT*
✓
✓
0.0823
0.0556
0.0864
0.1025
0.0850
0.3445
0.1261
✓
✓
0.0681
0.0359
0.0479
0.1545
0.0469
0.2902
0.1073
✓
✓
0.0645
0.0344
0.0503
0.0866
0.0567
0.2881
0.0968
✓
✓
✓
0.0227
0.0325
0.0223
0.0565
0.0206
0.2521
0.0677
TABLE III: Ablation study of R-PGA components. The results verify the necessity of Physically Disentangled Reconstruction Module (Relit), Hybrid Rendering (HR), and the HPCM module for achieving robust attack performance ([email protected]).
Fig. 10: Visual comparison of detection results across different methods in the physical domain. We display camouflaged vehicles captured under diverse physical configurations, including varying pitch angles, azimuths, distances, and lighting conditions. Green bounding boxes indicate correct detections (attack failure), while red bounding boxes denote incorrect detections (successful evasion).
SL
NL
LL
Average
ORI
0.9091
0.9134
0.9169
0.9131
DTA
0.7872
0.8035
0.9074
0.8327
ACTIVE
0.3466
0.2425
0.3958
0.3283
GCAC
0.6354
0.2345
0.8151
0.5617
GRAC
0.6144
0.4502
0.8163
0.6270
RAUCA-E2E
0.3611
0.2212
0.4534
0.3452
TABLE IV: Physical domain comparison of [email protected] against Yolo-V3 across different lighting conditions: Strong Light (SL), Normal Light (NL), and Low Light (LL), as well as the average. Each reported value represents the mean performance averaged over diverse pitch angles, azimuths, and shooting distances.
Deep learning models deployed in safety critical applications like autonomous driving use simulations to test their robustness against adversarial attacks in realistic conditions. However, these simulations are non-differentiable, forcing researchers to create attacks that do not integrate simulation environmental factors, reducing attack success. To address this limitation, we introduce UNDREAM, the first software framework that bridges the gap between photorealistic simulators and differentiable renderers to enable end-to-end optimization of adversarial perturbations on any 3D objects. UNDREAM enables manipulation of the environment by offering complete control over weather, lighting, backgrounds, camera angles, trajectories, and realistic human and object movements, thereby allowing the creation of diverse scenes. We showcase a wide array of distinct physically plausible adversarial objects that UNDREAM enables researchers to swiftly explore in different configurable environments. This combination of photorealistic simulation and differentiable optimization opens new avenues for advancing research of physical adversarial attacks.
Mansi Phute, Matthew Hull, Haoran Wang +6
Georgia Institute of Technology, USA · Technology Innovation Institute, UAE
3D Gaussian Splatting (3DGS) has recently emerged as a powerful explicit representation enabling fast, high-fidelity rendering, making it a promising foundation for closed-loop simulators and perception models in autonomous driving. However, conventional 3DGS implicitly assumes consistent exposure and tone mapping across views. Real driving data violates this assumption due to heterogeneous camera pipelines and dynamic outdoor illumination, baking exposure discrepancies and sensor noise into the radiance field and producing artifacts and inconsistent illumination especially in static backgrounds crucial for realistic simulation. These issues are amplified in autonomous driving, where sparse viewpoints, varying exposures, and outdoor lighting interact, while prior work mainly targets dynamic-object reconstruction and overlooks cross-view photometric consistency. To address this limitation, we introduce P2GS, a physically consistent Gaussian Splatting framework that jointly decomposes a view-invariant linear HDR radiance field, per-view exposure scales, and tone-mapping functions from only LDR images without HDR supervision. P2GS employs a unified optimization strategy grounded in the physical image-formation process, enforcing relative-exposure consistency and HDR-domain radiance regularization. This yields a radiance field robust to inter-camera illumination differences while preserving the real-time efficiency of standard 3DGS. Experiments across real and simulated driving environments show that P2GS matches or surpasses prior methods in LDR reconstruction while providing substantially improved photometric consistency, reliable exposure normalization, and physically coherent illumination across diverse scenes.
Kota Shimomura, Hidehisa Arai, Tsubasa Takahashi +2
Adversarial camouflage in the physical world remains highly challenging, particularly under UAV reconnaissance where targets undergo continuous geometric changes and extreme illumination variations. Existing methods either optimize 2D digital perturbations that fail to generalize to dynamic viewpoints or produce visually unnatural textures that cannot be deployed in real scenarios. Therefore, we propose an end-to-end framework for adversarial camouflage generation that automatically produces wearable adversarial patterns and maintains stable attack performance in real physical environments with changing viewpoints, poses, and lighting conditions. Our method integrates UV-volume rendering with a diffusion-based texture generator, enabling consistent appearance under varying scales, poses, and lighting conditions. To ensure environmental realism, we propose an illumination color consistency estimator that extracts dominant background attributes and guides a natural texture loss to align the generated UV texture with the surrounding environment. A multi-scale dynamic training strategy further enhances robustness against viewpoint shifts and body deformation. Extensive experiments across multiple mainstream detectors demonstrate that our method achieves strong and stable physical attack performance while maintaining high perceptual naturalness, reducing human detection rates without introducing unnatural artifacts.
Shihui Yan, Hu Liu, Junyu Shi +6
State Key Laboratory of Intelligent Vehicle Safety Technology · School of Cyber Science and Engineering, Huazhong University of Science and Technology · School of Computer Science and Technology, Huazhong University of Science and Technology +2