SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing

Sy-Tuyen Ho; Anirudh Satheesh; Soham Chawathe; Huy Nghiem; Yongkyun Lee; Andrew Adams Schoen; Furong Huang

cs.CVOct 22, 2026

SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing

Authors: Sy-Tuyen Ho, Anirudh Satheesh, Soham Chawathe, Huy Nghiem, Yongkyun Lee, Andrew Adams Schoen, Furong Huang

Organizations: University of Maryland · All Purpose AI

Abstract

Auditing whether a text-to-image (T2I) model was fine-tuned on a particular image collection is difficult: a model can absorb collection-level visual characteristics without reproducing training images. Proactive dataset-use auditing plants a shared signal in a dataset before release, then later checks whether that signal appears in a suspect model's generated outputs. Existing approaches often trade off verifiability, fidelity, robustness, and deployability. We introduce $\textbf{SpectralCanary}$ , which uses a smooth low-pass mask to blend a fixed canary image into every collection image while leaving captions unchanged and requiring neither auxiliary generation nor per-image optimization. A lightweight verifier detects the transferred signal in suspect-model outputs. Across eight tests on the Pokemon dataset and three realistic data transformations, spanning two architectures families, SpectralCanary achieves the highest average per-generation TPR@ $1\%$ FPR ( $91.1\%$ ) and fixed threshold model-level accuracy using 30 generations ( $94.8\%$ ) among the evaluated methods. Across five datasets spanning artistic, medical, satellite, and fashion domains with SD3.5, SpectralCanary averages $92.4\%$ per-generation TPR@ $1\%$ FPR while preserving the released images (SSIM $0.989$ , LPIPS $0.022$ ) with a near-zero mean change FID (mean $Δ$ FID $0.05$ ) across datasets. Matched-energy ablations on Pokemon and ROCOv2 support our design choice, low-frequency placement transfers more reliably compared to mid/high-frequency placement. Under the evaluated adaptive attacks, the most effective removal requires aggressive modification and incurs more than $25$ $Δ$ FID points. These results identify low-frequency visual signals as effective carriers for collection-level T2I audit traces.

Figures & tables

Figure 1: SpectralCanary overview. (a) Auditing workflow. An owner protects a collection before release, then queries a suspect T2I model and tests whether its generated outputs contain the canary. (b) Spectral hybridization. SpectralCanary blends a fixed canary through a smooth low-pass mask $M_{c}$ with strength $\alpha$ , retaining most of the remaining spectrum. A lightweight verifier trained on original–protected pairs detects the trace in model generations. (c) Visual fidelity. Original–protected pairs across five datasets, including sparse sketches.

Figure 2: SpectralCanary maintains strong per-generation discrimination across model and data changes. Each cell reports per-generation TPR at $1\%$ FPR. “Worst” is the lowest of a method’s eight results, and “Avg.” is their mean.

Figure 3: SpectralCanary is more consistent across transformations and models than the baselines. Each cell is Acc. “Worst” is the lowest of a method’s eight results, and “Avg.” is their mean.

Figure 4: SpectralCanary works consistently across five image types. Each point shows one method on one dataset. The vertical axis is the per-generation TPR at

1\%

FPR: the percentage of outputs from protected-data models detected while

1\%

of outputs from clean-data models are falsely flagged. In (a), the horizontal axis measures the change in the trained model’s output quality; zero means no change. In (b), it measures the change from an original image to its protected copy; zero means identical images. The best results are near the upper-left corner.

Figure 5: SpectralCanary achieves high per-generation discrimination with low quality and runtime cost. Average per-generation TPR at

1\%

FPR over the eight Pokemon settings is compared with (a) change in FID, (b) protection time, and (c) paired-image LPIPS. Preferred results lie upward and to the left. EnTruth adds

0.5\%

auxiliary samples, so paired-image LPIPS is not defined for it.

Figure 6: Matched low-frequency traces transfer more reliably than mid- or high-frequency traces. Points show mean per-generation TPR at

1\%

FPR over three canary images. Arrows show the performance gaps.

Figure 7: The evaluated attacks erase the trace only after causing substantial damage. From the unattacked point, stronger attacks move down and to the right, indicating per-generation TPR falls, but

\Delta

FID and LPIPS increase. The clean-training marker shows the target TPR.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Method	Per-Generation Discrimination (%) $\uparrow$			KS Rejection Rate on Protected-Data Models (%) $\uparrow$			Utility $\Delta$ FID $\downarrow$	Fidelity
		AUC	TPR@5% FPR	TPR@1% FPR	$\gamma=10^{-5}$	$\gamma=10^{-9}$	$\gamma=10^{-15}$		SSIM $\uparrow$	LPIPS $\downarrow$
Pokemon	EnTruth	$96.5\pm 1.1$	$94.9\pm 1.7$	$93.9\pm 1.1$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$-3.06\pm 0.60$	N/A	N/A
	SIREN	$99.1\pm 0.1$	$96.9\pm 0.7$	$91.9\pm 1.8$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$-6.58\pm 1.04$	0.980	0.050
	DIAGNOSIS	$53.6\pm 1.5$	$8.2\pm 0.4$	$2.3\pm 1.1$	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$3.65\pm 2.54$	0.994	0.007
	SpectralCanary	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$-9.00\pm 1.82$	0.998	0.007
Sketch-Scene	EnTruth	$99.8\pm 0.1$	$98.8\pm 0.7$	$97.5\pm 1.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$-5.98\pm 1.50$	N/A	N/A

Appendix

Table 1: SD3.5 comparison across datasets. Paired training-image fidelity is deterministic for a fixed protected dataset. EnTruth injects 0.5% auxiliary samples and therefore has no directly comparable paired-LPIPS value.

Transformation	Model	Method	AUC (%) $\uparrow$	TPR@5% FPR (%) $\uparrow$	TPR@1% FPR (%) $\uparrow$
None	SD1.5	EnTruth	$98.8\pm 0.2$	$94.1\pm 0.5$	$85.3\pm 2.6$
		SIREN	$98.4\pm 0.1$	$93.9\pm 0.3$	$66.3\pm 1.1$
		DIAGNOSIS	$99.7\pm 0.2$	$100\pm 0.0$	$99.9\pm 0.1$
		SpectralCanary	$99.7\pm 0.1$	$98.8\pm 0.4$	$97.7\pm 0.5$
	SD3.5	EnTruth	$96.5\pm 1.1$	$94.9\pm 1.7$	$93.9\pm 1.1$
		SIREN	$99.1\pm 0.1$	$96.9\pm 0.7$	$91.9\pm 1.8$

Appendix

Table 2: Per-generation discrimination on Pokemon. Means and standard deviations over three runs.

Figure 8: Protected images under different blending strengths $\alpha$ and Gaussian bandwidths $c$ .

Figure 9: Protection-hyperparameter ablations on Pokemon with SD1.5: (a) blending strength, fixing

c=0.2,\delta=10\%

; (b) protected fraction, fixing

c=0.2,\alpha=0.02

; and (c) Gaussian bandwidth, fixing

\delta=100\%,\alpha=0.02

	Protection time (s/img) $\downarrow$		Per-Generation Discrimination (%) $\uparrow$			KS-Based Multi-Query Audit Rejection Rate (%) $\uparrow$
Method	CPU	GPU	AUC	TPR@5% FPR	TPR@1% FPR	$\gamma=10^{-5}$	$\gamma=10^{-9}$	$\gamma=10^{-15}$
DIAGNOSIS	0.02	0.02	74.7	46.7	32.2	49.7	38.2	37.5
SIREN	–	71.38	89.5	69.7	52.6	84.8	70.8	57.2
EnTruth	6.92	0.40	93.2	81.8	73.2	93.3	80.8	75.2
Silent Branding	–	273.13	80.9	32.5	12.8	73.8	34.1	2.7
SpectralCanary	0.39	0.39	98.0	94.0	91.1	99.6	96.4	83.6

Appendix

Table 3: End-to-end dataset protection cost and downstream per-generation discrimination on Pokemon. Protection time includes every stage required to produce the protected dataset and is amortized over samples. We omit CPU measurements for SIREN and Silent Branding because their pipelines require GPU-based optimization. DIAGNOSIS and SpectralCanary require no GPU for protection and therefore do not benefit from GPU acceleration. KS-based multi-query auditing uses 30 generated images per test.

Figure 10: Per-generation discrimination emerges early and remains stable during fine-tuning. Using a defender’s verifier, we report (left) per-generation AUC and TPR at FPRs of

5\%

and

1\%

, and (right) the mean predicted protected-class probability for clean-data-model and protected-data-model generations. Each checkpoint uses 100 matched Pokemon generations per condition with the same captions.

Transformation	Per-Generation Discrimination (%) $\uparrow$			KS Rejection Rate on Protected-Data Models (%) $\uparrow$
	AUC	TPR@5% FPR	TPR@1% FPR	$\gamma=10^{-5}$	$\gamma=10^{-9}$	$\gamma=10^{-15}$
None	$98.8\pm 0.3$	$94.0\pm 1.7$	$91.7\pm 2.9$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
Training-Time Augmentation	$84.3\pm 4.9$	$66.4\pm 10.2$	$61.9\pm 12.0$	$91.7\pm 11.6$	$68.2\pm 36.3$	$30.0\pm 27.9$
Prompt Recaptioning	$97.9\pm 0.2$	$88.8\pm 1.5$	$85.5\pm 3.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$99.5\pm 0.2$
Image Regeneration	$99.7\pm 0.1$	$97.8\pm 0.8$	$96.0\pm 1.6$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
Average	$95.2$	$86.7$	$83.8$	$97.9$	$92.1$	$82.3$

Appendix

Table 4: SpectralCanary under full SD1.5 fine-tuning on Pokemon. We report mean and standard deviation across three runs.

Transformations	Model	Method	TPR (%) $\uparrow$	FPR (%) $\downarrow$	Precision (%) $\uparrow$	F1 (%) $\uparrow$	Accuracy (%) $\uparrow$
None	SD1.5	EnTruth	$100.0\pm 0.0$	$0.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
		SIREN	$100.0\pm 0.0$	$0.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
		DIAGNOSIS	$100.0\pm 0.0$	$0.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
		SpectralCanary	$100.0\pm 0.0$	$0.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
	SD3.5	EnTruth	$64.1\pm 8.1$	$0.0\pm 0.0$	$100.0\pm 0.0$	$77.8\pm 5.6$	$82.0\pm 4.0$
		SIREN	$100.0\pm 0.0$	$0.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$

Appendix

Table 5: Thirty-query fixed-threshold model-level verification across transformation settings. Thresholds are calibrated once from all 1,000 SD1.5 or 500 SD3.5 queries of separate no-transformation calibration models. Each suspect model is evaluated using 30 queries. We repeat the 30-query sampling 1,000 times from the held-out generations and report the mean and standard deviation of model-level metrics across these resampled query sets.

Figure 11: Qualitative generations from unauthorized models fine-tuned on SpectralCanary protected data.

Panel of Figure 12 — Figure 12: KS-based multi-query auditing improves with the query budget.

Dataset	Band	$E/E_{\mathrm{ref}}$	AUC $\uparrow$	TPR@5% FPR $\uparrow$	TPR@1% FPR $\uparrow$	$\Delta$ FID $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Pokemon	Low	$1.000$	$0.991\pm 0.007$	$0.948\pm 0.038$	$0.923\pm 0.042$	$-2.84\pm 4.35$	$0.998\pm 0.000$	$0.005\pm 0.002$
	Mid	$1.000$	$0.908\pm 0.069$	$0.664\pm 0.226$	$0.481\pm 0.361$	$-4.81\pm 3.87$	$0.947\pm 0.003$	$0.049\pm 0.004$
	High	$1.000$	$0.976\pm 0.017$	$0.899\pm 0.071$	$0.754\pm 0.155$	$-5.16\pm 3.82$	$0.961\pm 0.003$	$0.006\pm 0.001$
ROCOv2	Low	$1.000$	$0.999\pm 0.001$	$0.996\pm 0.005$	$0.986\pm 0.010$	$6.92\pm 2.19$	$0.905\pm 0.008$	$0.069\pm 0.025$
	Mid	$1.000$	$0.879\pm 0.066$	$0.492\pm 0.158$	$0.259\pm 0.096$	$2.28\pm 2.62$	$0.866\pm 0.009$	$0.204\pm 0.018$
	High	$1.000$	$0.788\pm 0.111$	$0.303\pm 0.199$	$0.097\pm 0.064$	$7.95\pm 0.93$	$0.874\pm 0.011$	$0.035\pm 0.005$

Appendix

Table 6: Frequency location affects traceability under matched perturbation energy. We use disjoint binary low-, mid-, and high-frequency bands, each containing one-third of Fourier coefficients. For each canary, we match the mean pre-IFFT spectral perturbation energy to the default Gaussian low-pass SpectralCanary reference; thus, we report the dimensionless energy ratio

E/E_{\mathrm{ref}}

rather than resolution-dependent raw DFT energy. Results are mean

\pm

standard deviation across three canaries.

Figure 13: Natural spatial structure enables more reliable visual-canary transfer.

Figure 14: Visualization of adaptive spectral sanitization settings. (a) Fixed donor quality-constrained replacement, (b) Per-image donor quality constrained replacement, (c) Shared-structure estimate, (d) Full donor replacement, (e) Direct low-frequency removal. Stronger sanitization suppresses more of the protection signal but increasingly alters original-image content.

Figure 15: Visualization on Sketch-Scene. Paired visualization of original image and protected image by SpectralCanary

Figure 16: Visualization on Pokemon. Paired visualization of original image and protected image by SpectralCanary.

Figure 17: Visualization on ROCOv2. Paired visualization of original image and protected image by SpectralCanary.

Figure 18: Visualization on RSICD. Paired visualization of original image and protected image by SpectralCanary.

Figure 19: Visualization on FACAD. Paired visualization of original image and protected image by SpectralCanary.

Figure 20: EnTruth’s injected samples are visually distinct from the protected samples. The injected samples share a repeated framed template and fruit foreground, while representative Pokemon samples exhibit the diverse character appearance of the protected dataset.

Dataset	Transformation	Audit Prompt	AUC (%) $\uparrow$	TPR@5% FPR (%) $\uparrow$	TPR@1% FPR (%) $\uparrow$
Pokemon	None	Hard Trigger	$\mathbf{96.5\pm 1.1}$	$\mathbf{94.9\pm 1.7}$	$\mathbf{93.9\pm 1.1}$
	None	Soft Trigger	$59.2\pm 1.8$	$27.0\pm 1.0$	$20.6\pm 1.8$
	Training-Time Augmentation	Hard Trigger	$\mathbf{99.1\pm 0.6}$	$\mathbf{97.3\pm 1.7}$	$\mathbf{95.3\pm 1.6}$
	Training-Time Augmentation	Soft Trigger	$63.8\pm 3.7$	$19.7\pm 12.1$	$12.6\pm 10.4$
	Prompt Recaptioning	Hard Trigger	$\mathbf{83.6\pm 1.7}$	$\mathbf{52.3\pm 3.1}$	$\mathbf{35.9\pm 1.6}$
	Prompt Recaptioning	Soft Trigger	$61.9\pm 0.6$	$25.7\pm 1.5$	$14.5\pm 3.4$

Appendix

Table 7: EnTruth auditing under hard-trigger and soft-trigger prompting across the Pokemon and Sketch-Scene benchmarks using SD3.5.

Transformation	Model	Method	AUC (%) $\uparrow$	TPR@5% FPR (%) $\uparrow$	TPR@1% FPR (%) $\uparrow$
None	SD1.5	Silent Branding	$85.1\pm 5.1$	$44.8\pm 15.3$	$19.2\pm 6.0$
	SD1.5	SpectralCanary	$\mathbf{99.7\pm 0.1}$	$\mathbf{98.8\pm 0.4}$	$\mathbf{97.7\pm 0.5}$
	SD3.5	Silent Branding	$78.9\pm 1.6$	$24.2\pm 5.9$	$10.6\pm 6.3$
	SD3.5	SpectralCanary	$\mathbf{100\pm 0.0}$	$\mathbf{100\pm 0.0}$	$\mathbf{100\pm 0.0}$
Training-Time Augmentation	SD1.5	Silent Branding	$85.2\pm 2.5$	$45.1\pm 3.4$	$23.2\pm 6.7$
	SD1.5	SpectralCanary	$\mathbf{94.9\pm 1.5}$	$\mathbf{85.5\pm 7.8}$	$\mathbf{79.2\pm 12.3}$

Appendix

Table 8: Comparison between adapted Silent Branding and SpectralCanary across transformation settings. We follow the presentation of Tab. 2 on Pokemon with SD1.5 and SD3.5. We report mean and standard deviation across three runs; the best result within each setting is bolded.

Method	Venue	Goal	Image modification	Caption modification
Nightshade	IEEE S&P 2024	Disrupt T2I concepts	Content-preserving adversarial perturbation	None
SilentBadDiffusion	ICML 2024	Induce copyright reproduction	Semantic elements inserted into synthetic images	Element-specific phrases
Bilateral Data Poisoning	NeurIPS 2024	Corrupt T2I generation	Adversarial image modification	Poisoned captions
Silent Branding	CVPR 2025	Inject persistent visual branding	Semantic elements inserted into protected samples	None
DIAGNOSIS	ICLR 2024	Dataset-use auditing	Content-preserving image warping	Rare trigger token added
SIREN	IEEE S&P 2025	Dataset-use auditing	Content-preserving perturbation	None

Appendix

Table 9: Comparison of T2I data-poisoning and proactive dataset-use auditing methods. We consider methods that modify only the released training data and assume a standard training or fine-tuning pipeline. Content-preserving indicates that the modification does not intentionally alter the image’s original semantic content.

Dataset Watermarking

Dataset watermarking embeds detectable signals in data to verify ownership or trace its use in trained models while preserving data utility.

See all 21 papers

Sep 30, 2026

Persistent Watermarking of Text-to-Image Models

Text-to-image (T2I) generation is gaining increasing popularity with the general public, motivating the development of reliable mechanisms for copyrighting such models given their expensive training costs. An adversary may obtain and reuse a pretrained T2I model without authorization, and then serve a modified version through an API service. Such modifications may arise from ordinary downstream adaptation or deliberate attempts to erase ownership, including input-prompt preprocessing, model fine-tuning, and output post-processing. From the model owner's perspective, a key challenge is therefore to embed trigger data that remain persistent under such changes while preserving the model's normal image-generation capabilities. In this work, we propose a contrastive-style watermarking objective with a term that explicitly encourages the watermarked model to behave differently from the original model on trigger inputs. Experiments show substantially stronger trigger-data persistence than prior methods across a wide range of downstream modifications and deliberate attempts to weaken the watermark, resulting in higher detection rates, often approaching 100% TPR@FPR<

10^{-4}

Dixi Yao, Kaiwen Chen, Tahseen Rabbani +1

University of Chicago

Diffusion Model Watermarking Robust Watermarking T2I Diffusion Models AI-Generated Image Watermarking

Ask PDF

Sep 5, 2026

Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models

Red-teaming Text-to-Image (T2I) models is essential for safe deployment, yet it remains particularly challenging against implicit adversarial prompts. Unlike explicit adversarial prompts that can be readily identified and blocked, implicit ones are much harder to detect: the prompts appear benign on the text surface yet still lead to inappropriate visual content. To address this, we propose Adversarial Probing for Implicit VulnErabilities (AdvPIE), a multimodal agentic framework to expose implicit vulnerabilities without requiring access to the parameters of target models. AdvPIE adopts a policy agent to generate and refine implicit adversarial prompts based on the feedback from a judge agent. To construct informative feedback, the judge agent provides modality-specific safety evaluation at both global and relative levels across iterations. To effectively leverage the feedback, we propose a novel Cumulative Adversarial Decoding strategy for the policy agent, which dynamically reweights token distributions to favor tokens that lead to more harmful images while preserving sampling diversity. Extensive experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities, outperforming various baseline methods.

Chang Ma, Junlin Han, Shuo Chen +3

Columbia University, USA · University of Oxford, UK · LMU Munich, Germany +2

Safe T2I Generation Adversarial Prompt Generation Black-Box Adversarial Attacks T2I Generation Evaluation

Ask PDF

Mar 27, 2026

Verify Claimed Text-to-Image Models via Boundary-Aware Prompt Optimization

As Text-to-Image (T2I) generation becomes widespread, third-party platforms increasingly integrate multiple model APIs for convenient image creation. However, false claims of using official models can mislead users and harm model owners' reputations, making model verification essential to confirm whether an API's underlying model matches its claim. Existing methods address this by using verification prompts generated by official model owners, but the generation relies on multiple reference models for optimization, leading to high computational cost and sensitivity to model selection. To address this problem, we propose a reference-free T2I model verification method called Boundary-aware Prompt Optimization (BPO). It directly explores the intrinsic characteristics of the target model. The key insight is that although different T2I models produce similar outputs for normal prompts, their semantic boundaries in the embedding space (transition zones between two concepts such as "corgi" and "bagel") are distinct. Prompts near these boundaries generate unstable outputs (e.g., sometimes a corgi and sometimes a bagel) on the target model but remain stable on other models. By identifying such boundary-adjacent prompts, BPO captures model-specific behaviors that serve as reliable verification cues for distinguishing T2I models. Experiments on five T2I models and four baselines demonstrate that BPO achieves superior verification accuracy.

Zidong Zhao, Yihao Huang, Qing Guo +5

Zhejiang University, Hangzhou, China · East China Normal University, Shanghai, China · Nankai University, Tianjin, China +4

T2I Generation Evaluation T2I Diffusion Models Prompt Optimization for T2I Generation

Ask PDF

Abstract

Dataset Watermarking

Explore similar work

Persistent Watermarking of Text-to-Image Models

Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models

Verify Claimed Text-to-Image Models via Boundary-Aware Prompt Optimization