In-the-Wild Camouflage Attack on Vehicle Detectors through Controllable Image Editing

Xiao Fang; Yiming Gong; Stanislav Panev; Celso de Melo; Shuowen Hu; Shayok Chakraborty; Fernando De la Torre

cs.CVMar 19, 2026

In-the-Wild Camouflage Attack on Vehicle Detectors through Controllable Image Editing

Authors: Xiao Fang, Yiming Gong, Stanislav Panev, Celso de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre

Organizations: Carnegie Mellon University · DEVCOM Army Research Laboratory · Florida State University

Abstract

Deep neural networks (DNNs) have achieved remarkable success in computer vision but remain highly vulnerable to adversarial attacks. Among them, camouflage attacks manipulate an object's visible appearance to deceive detectors while remaining stealthy to humans. In this paper, we propose a new framework that formulates vehicle camouflage attacks as a conditional image-editing problem. Specifically, we explore both image-level and scene-level camouflage generation strategies, and fine-tune a ControlNet to synthesize camouflaged vehicles directly on real images. We design a unified objective that jointly enforces vehicle structural fidelity, style consistency, and adversarial effectiveness. Extensive experiments on the COCO and LINZ datasets show that our method achieves significantly stronger attack effectiveness, leading to more than 38% AP50 decrease, while better preserving vehicle structure and improving human-perceived stealthiness compared to existing approaches. Furthermore, our framework generalizes effectively to unseen black-box detectors and exhibits promising transferability to the physical world. Project page is available at https://humansensinglab.github.io/CtrlCamo

Figures & tables

Figure 1: Overview. Given a real image, our pipeline stylizes the target vehicle based on either its immediate surroundings (image-level) or a visual concept present in the overall scene (scene-level), producing stealthy adversarial examples. The numbers on the bounding boxes indicate detector confidence scores, and the absence of a box indicates that the vehicle is not detected.

Figure 2: Overview of our pipeline. As shown in (a) and (b), the pipeline consists of a No-Box Attack stage and a White-Box Attack stage. In (a), the ControlNet is fine-tuned to stylize vehicles using a reference region while preserving geometry and background through structure, style, and background supervisions ( $L_{\text{struct}}$ , $L_{\text{s}}$ , $L_{\text{b}}$ ). (b) further optimizes the model against a detector $\mathcal{M}_{\text{det}}$ by incorporating an additional adversarial loss $L_{\text{adv}}$ and a color-consistency loss $L_{\text{c}}$ . (c) summarizes the conditions provided to ControlNet under the image-level and scene-level settings, and (d) illustrates the style loss $L_{\text{s}}$ that aligns vehicle latent features with the reference area.

Method	Faster-RCNN					ViTDet
	image-level		scene-level		Inf. Latency (s) $\downarrow$	image-level		scene-level		Inf. Latency (s) $\downarrow$
	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$
Normal	98.3	-	98.3	-	-	97.8	-	97.8	-	-
AdvCAM [ 9 ]	90.5	0.968	88.6	0.963	21.8	84.1	0.964	80.8	0.963	21.8
Diff-PGD [ 44 ]	73.2	0.966	66.4	0.959	32.0	62.3	0.961	59.6	0.96	32.4
Ours	18.3	0.972	27.5	0.961	7.00	13.7	0.972	11.1	0.964	7.67

Table 1: Quantitative comparison on the LINZ Dataset for stylization-based camouflage attacks . We report the

\mathrm{AP}_{50}

, SSIM, and average sampling time.

Method	Faster-RCNN					ViTDet
	image-level		scene-level		Inf. Latency (s) $\downarrow$	image-level		scene-level		Inf. Latency (s) $\downarrow$
	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$
Normal	85.6	-	85.6	-	-	91.4	-	91.4	-	-
AdvCAM [ 9 ]	64.7	0.680	66.4	0.651	21.8	70.5	0.678	68.3	0.650	21.8
Diff-PGD [ 44 ]	61.1	0.692	64.6	0.664	33.2	68.5	0.677	68.5	0.663	34.2
Ours	15.0	0.850	16.6	0.837	7.32	19.2	0.849	12.5	0.840	8.15

Table 2: Quantative comparison on the COCO Dataset for stylization-based camouflage attacks . We report the

\mathrm{AP}_{50}

, SSIM, and average sampling time.

Method	Type	LINZ		COCO
		Faster-RCNN	ViTDet	Faster-RCNN	ViTDet
		$\mathrm{AP}_{50}(\%)$	$\mathrm{AP}_{50}(\%)$	$\mathrm{AP}_{50}(\%)$	$\mathrm{AP}_{50}(\%)$
Normal	-	98.3	97.8	85.6	91.4
ToG [ 5 ]	Noise	83.5	82.8	72.5	80.0
DiffAttack [ 1 ]	Noise	98.0	97.5	77.7	83.2
NAP [ 16 ]	Patch	89.2	88.6	75.4	87.1

Table 3: Quantitative comparison with non-stylization camouflage attacks. “Noise” and “Patch” denote adding random perturbations and patches on vehicles to conduct camouflage attack. We report the average

\mathrm{AP}_{50}

of both strategies.

Figure 3: Qualitative comparison with other methods. The first two rows show results from the COCO dataset, and the last two rows are from the LINZ dataset. Within each dataset, the first row corresponds to the image-level strategy, and the second row corresponds to the scene-level strategy. Scene types are indicated on the left. In the “lake” scene, boats are stylized toward the water, while in the “parking lot” scene, cars are stylized toward trees. All camouflaged images are composed with real images background.

Dataset	Surrogate	Faster-RCNN	ViTDet	YOLOv5	YOLOv8	MLLM
LINZ	Normal	98.3	97.8	97.4	95.4	96.3
LINZ	Faster-RCNN	-	33.7	32.3	19.3	10.7
LINZ	ViTDet	11.4	-	38.6	25.2	11.3
COCO	Normal	85.6	91.4	88.3	90.7	87.5
COCO	Faster-RCNN	-	40.0	30.2	39.0	31.8
COCO	ViTDet	38.5	-	34.4	39.6	33.0

Table 4: Evaluation of camouflages across various models. We report the average of image-level and scene-level strategies using

\mathrm{AP}_{50}

for detectors, and classification accuracy for MLLMs.

Figure 4: Projector-based physical experiment. (a) Real images in digital space. (b) Photos captured from real images. (c) Reference areas used for style guidance. (d) Camouflaged images generated from the captured photos. (e) Photos taken after projecting the camouflaged images back onto the 3D physical models.

Figure 5: Effectiveness of background supervision.

Figure 6: Effectiveness of color-consistency loss. (a) Real images. (b) Reference areas used for style guidance. (c) Camouflaged images generated in the “No-Box Attack” stage. Camouflaged images with $L_{\text{c}}$ . (d) Camouflaged images without $L_{\text{c}}$ .

Figure S7 : Workflow for selecting style reference areas for the scene-level camouflage generation strategy. The process consists of four steps. In the first step, we query MoonDream for initial scene types. In the second step, we refine these categories using GPT-4o and recollect labels under five representative scene groups. In the third step, we query object-level information in each image. In the last step, we select a concept in each scene and synthesize an exemplar image containing that concept. We then segment the target concept with SAM 2. The extracted region serves as the reference area for scene-level stylization. Red circles denote we use the “house” concept in “Residential” area as an example to illustrate how we generate reference area in the last step.

Figure S8 : Qualitative evaluation of SD v3.5 on COCO. We visualize scene-level camouflage generation results. Each row corresponds to a scene type. Numbers above the bounding boxes indicate detector confidence, and the absence of a box indicates a missed detection. All camouflaged vehicles are composited onto the original real-image backgrounds.

Figure S9 : Qualitative evaluation of SD v3.5 on LINZ. We visualize image-level camouflage generation results.

Method	Faster-RCNN					ViTDet
	image-level		scene-level		Inf. Latency (s) $\downarrow$	image-level		scene-level		Inf. Latency (s) $\downarrow$
	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	Inf. Latency (s) $\downarrow$
Normal	98.3	-	85.6	-	-	97.8	-	91.4	-	-
Ours (SD v1.5)	18.3	0.972	16.6	0.837	7.32	13.7	0.972	12.5	0.840	8.15
Ours (SD v3.5)	17.0	0.978	27.9	0.845	4.86	8.8	0.978	29.7	0.844	4.88

Table S5 : Quantative comparison between SD v1.5 and v3.5. The image-level strategy is evaluated on the LINZ dataset, while the scene-level strategy is evaluated on the COCO dataset. We report the

\mathrm{AP}_{50}

, SSIM, and average sampling time per image.

Dataset	Target	Strategy	No-Box Attack					White-Box Attack
Dataset	Target	Strategy	$L_{\text{struct}}$	$L_{\text{s}}$	$L_{\text{b}}$	lr	iter	$L_{\text{struct}}$	$L_{\text{s}}$	$L_{\text{b}}$	$L_{\text{c}}$	$L_{\text{adv}}$	lr	iter
COCO	Faster-RCNN	Scene-Level	5.0	1.0	0.0	5e-6	12000	5.0	1.0	0.0	2.0	1.0	4e-6	15000
COCO	Faster-RCNN	Image-Level	2.5	2.5	1.0	2e-6	15000	4.0	1.0	1.0	2.0	1.0	2.5e-6	20000
COCO	ViTDet	Scene-Level	5.0	1.0	0.0	5e-6	12000	5.0	1.0	0.0	2.0	1.0	4e-6	15000
COCO	ViTDet	Image-Level	2.5	2.5	1.0	2e-6	15000	10.0	2.5	1.0	2.5	1.0	4e-6	20000
LINZ	Faster-RCNN	Scene-Level	4.0	1.0	0.0	2.5e-6	10000	10.0	2.5	0.0	2.5	1.0	2e-6	12000

Table S6 : Training parameters of each experiment . “lr” denotes learning rate. “iter” denotes the total number of training iterations. “

L_{\text{struct}}

L_{\text{s}}

L_{\text{b}}

L_{\text{c}}

L_{\text{adv}}

” denotes the coefficents of structure preservation loss, style loss, background reconstruction loss, color-consistency loss, and adversarial loss, respectively.

Figure S10 : More Visualizations on the COCO dataset. We visualize scene-level camouflage generation results. We present examples in urban, rural, road, sky , and lake scenes, where they are associated with building, grass, tree, sky , and water as representative concepts.

Figure S11 : More Visualizations on the COCO dataset. We visualize image-level camouflage generation results.

Figure S12 : Visualizations on the LINZ dataset. We present examples in residential, industrial, agricultural, parking lot, and highway scenes, where they are matched with house, building, field, tree , and grass as representative concepts.

Figure S13 : More Visualizations on the LINZ dataset. We visualize image-level camouflage generation results.

Dataset	Method	Faster-RCNN		ViTDet
Dataset	Method	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$	$\mathrm{AP}_{50}(\%)\downarrow$	SSIM $\uparrow$
COCO	Normal	85.6	-	91.4	-
COCO	Ours	16.6	0.837	12.5	0.840
COCO	Ours*	21.1	0.846	20.2	0.845
LINZ	Normal	98.3	-	97.8	-
LINZ	Ours	27.5	0.961	11.1	0.964

Table S7 : Quantitative evaluation of alternative scene–concept pairings for the scene-level strategy on COCO and LINZ. “Ours” denotes the scene–concept groups described in Sec. 4.1 , while “Ours*” adopts the alternative set evaluated in Sec. H.3 .

Figure S14 : Visualizations on the COCO dataset. We present examples in urban, rural, road, sky , and lake scenes, where they are matched with with platform, tree, road, cloud , and beach as representative concepts.

Figure S15 : Visualizations on the LINZ dataset. We present examples in residential, industrial, agricultural, parking lot, and highway scenes, where they are matched with with grass, sidewalk, parking lot, dirt , and parking lot as representative concepts.

Method	Faster-RCNN		ViTDet
Method	image-level	scene-level	image-level	scene-level
Normal	85.6	85.6	91.4	91.4
Baseline	15.0	16.6	19.2	12.5
Denoise	29.8	28.2	36.4	27.4
Smooth	21.4	25.4	25.5	23.0

Table S8 : Evaluation under preprocessing-based defense strategies on the COCO dataset. “Normal” denotes clean images, “Baseline” denotes adversarial images without defense, “Denoise” applies Non-local Means denoising, and “Smooth” applies bilateral filtering.

Figure S16 : Comparing our pipeline with one-stage variants .

Dataset	Method	Faster-RCNN		ViTDet
Dataset	Method	image-level	scene-level	image-level	scene-level
COCO	Normal	85.6	85.6	91.4	91.4
COCO	W/O $L_{\text{adv}}$	67.2	65.1	73.6	72.1
COCO	Ours	18.3	27.5	13.7	11.1
LINZ	Normal	98.3	98.3	97.8	97.8
LINZ	W/O $L_{\text{adv}}$	92.4	77.2	85.2	52.9

Table S9 : Effectiveness of adversarial loss.

Figure S17 : Effectiveness of structure preservation loss.

Figure S18 : Effectiveness of style loss.

Figure S19 : Human Evaluation of stealthiness comparing with state-of-the-art methods.

Figure S20 : Human Evaluation of stealthiness across ablation variants.

Figure S21 : Projector-based physical setup for the LINZ dataset. (a) Device configuration with the projector and camera aligned to minimize parallax. (b) A real image is projected onto a whiteboard with a 3D-printed car model placed at the corresponding location. (c) Alternative view showing attachment of the printed model to the board.

Figure S22 : Projector-based physical setup for the COCO dataset. (a) Hardware configuration. (b) 3D-printed scene reconstructed from two COCO images via single-view reconstruction. (c-d) Real images projected onto the corresponding 3D-printed models.

Figure S23 : Transferability of scene-level camouflage across locations in COCO dataset. For each scene type, we composite the vehicle into five additional backgrounds to create synthetic scenes while keeping the vehicle appearance unchanged. In each scene group, the first row shows detection results on clean vehicles, and the second row shows results on camouflaged vehicles. “Original” denotes the original COCO scene, whereas “Synthetic” refers to the newly composed scenes with replaced backgrounds.

Figure S24 : Projector-based physical experiment for the LINZ dataset. The first three rows present examples produced using the image-level strategy, and the last three rows illustrate the scene-level strategy. (a) Real images in digital space. (b) Photos captured from real images. (c) Reference areas used for style guidance. (d) Camouflaged images generated from the captured photos. (e) Photos taken after projecting the camouflaged images back onto the 3D car models.

Figure S25 : Projector-based physical experiment for the COCO dataset. The first three rows present examples produced using the image-level strategy, and the last three rows illustrate the scene-level strategy. (a) Real images in digital space. (b) Photos captured from real images projected on the 3D-printed scenes. (c) Reference areas used for style guidance. (d) Camouflaged images generated from the captured photos. (e) Photos taken after projecting the camouflaged images back onto the 3D-printed scenes.

Figure S26 : Comparison of camouflaged vehicles and their corresponding style reference areas in HSV hue space.

Figure S27 : Failure cases of the image-level camouflage generation strategy on the COCO dataset.

Figure S28 : Image-level user study guideline.

Figure S29 : Image-level example question on the COCO dataset.

Figure S30 : Image-level example question on the LINZ dataset.

Figure S31 : Scene-level user study guideline.

Figure S32 : Scene-level example question on the COCO dataset.

Figure S33 : Scene-level example question on the LINZ dataset.

Figure S34 : Example question on the COCO dataset.

Figure S35 : Example question on the LINZ dataset.

Adversarial Attacks on Object Detectors

See all 20 papers

Jun 16, 2026

Structured Adversarial Camouflage via Voronoi Diagrams

Pixel-wise adversarial patches are computationally heavy and often visually detectable, limiting utility in security-critical systems. We present adversarial Voronoi camouflage that optimizes only seed-point locations under fixed, printable palettes using a soft assignment, producing structured, splinter camouflage-like patterns without additional regularization. Evaluated on person detection with COCO-style AP@[.5:.95], naive placement (Inria -> COCO) performs comparably bad, while garment-level application via segmentation mask (3DPeople) results in a significant AP drop. The attack transfers to out-of-domain backgrounds and across detector families (YOLOv9/10/11/12), indicating robustness in black-box settings. Repainting with different palettes largely nullifies the effect, and single-color tweaks show limited tolerance (<=0.17), highlighting a structure-palette coupling. The parameter-efficient, palette-constrained design improves visual plausibility while degrading real-time detector performance. Physical validation and color calibration are left for future work. Code: https://github.com/JensBayer/Voronoi This paper was originally presented at the International Conference on Military Communication and Information Systems (ICMCIS), organized by the Information Systems Technology (IST) Scientific and Technical Committee, IST-224-RSY - the ICMCIS, held in Bath, United Kingdom, 12-13 May 2026.

Jens Bayer, Stefan Becker, David Münch +2

Fraunhofer IOSB and Fraunhofer Center for Machine Learning · Karlsruhe Institute of Technology

Adversarial Patch Attacks Object Detection Black-Box Adversarial Attacks Physical Adversarial Attacks Adversarial Attacks on Object Detectors

Ask PDF

Aug 7, 2026

AdvTiles: Physical Adversarial Camouflage Clothing against Person Detectors via Learnable Tiles

Physical adversarial attacks against person detectors have evolved from localized patches to full-body textures. However, achieving both visual naturalness and strong attack effectiveness remains challenging. Existing natural-looking methods typically optimize camouflage textures as a whole, limiting the flexibility to refine local adversarial patterns and their spatial arrangement. To address this issue, we propose AdvTiles, a physical adversarial camouflage framework built from learnable tiles, enabling strong attack performance while preserving a natural camouflage appearance. Specifically, we use a Straight-through (ST) Gumbel-Softmax estimator for differentiable tile selection, enabling joint optimization of tile patterns and spatial layouts. This design provides fine-grained control over adversarial texture generation. To improve robustness in diverse physical conditions, we further optimize the camouflage through differentiable 3D Gaussian Splatting rendering with variations in viewpoints, scales, illuminations and backgrounds. Extensive experiments across multiple detectors demonstrate that AdvTiles achieves an average ASR of 86.2%, outperforming existing state-of-the-art attack methods. We further fabricate the optimized camouflage into wearable adversarial clothing, validating its effectiveness in real-world scenarios across diverse distances, angles and backgrounds.

Jinlei Wang, Jiahuan Long, Mingkai Sun +9

Sun Yat-sen University · 2Chinese Academy of Military Science · 3Shenzhen University +2

Adversarial Attacks Adversarial Attacks on Object Detectors Physical Adversarial Attacks

Ask PDF

Jul 23, 2026

Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks

AI-enabled visual perception systems are increasingly deployed in intelligent transportation infrastructure and autonomous vehicle related applications. However, physically realizable adversarial appearances pose a significant reliability challenge for these safety-critical systems. Adversarial training is effective, but repeated co-occurrence between adversarial texture and positive person instances can cause detectors to treat the texture itself as evidence of object presence, forming a patch texture shortcut. The detector may then treat texture as evidence for the target, causing false detections on texture-only inputs and weakening cross attack generalisation. We propose InsCAT, an instance-level contrastive adversarial training framework that prevents detectors from using adversarial texture as an independent decision cue. SICA aligns adversarial person features with matched clean features and separates them from texture-only negatives, while ROPO and Guard maintain online attack pressure and coordinate training. We evaluate eight independently generated attack textures on rendered nuScenes, INRIAPerson, printed garments, and three detector families. InsCAT achieves an average attack AP of 82.3% on rendered nuScenes, exceeding the strongest baseline by 11.1 points.Relative to AT-Mix, texture FPR decreases from 46.9% to 7.3%. Physical tests yield an F1 score of 96.6% and an FPR of 1.8%. Consistent gains across separately trained detectors demonstrate applicability across architectures with direct inference. The findings show that robust physical detection depends on preserving target related evidence while preventing adversarial texture from becoming an independent decision cu

Yuanhao Huang, Yilong Ren, Jinlei Wang +3

School of Transportation Science and Engineering, Beihang University, 100191, Beijing, P.R China · Zhongguancun Laboratory, 100191, Beijing, P.R China · School of Systems Science and Engineering, Sun Yat-Sen University, 510006, Guangzhou, P.R China +2

Adversarial Training Adversarial Robustness Object Detection Shortcut Learning Physical Adversarial Attacks Adversarial Attacks on Object Detectors

Ask PDF

Abstract

Adversarial Attacks on Object Detectors

Explore similar work

Structured Adversarial Camouflage via Voronoi Diagrams

AdvTiles: Physical Adversarial Camouflage Clothing against Person Detectors via Learnable Tiles

Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks