Authorize-on-Demand: Dynamic Authorization with Legality-Aware Intellectual Property Protection for VLMs

Lianyu Wang; Meng Wang; Huazhu Fu; Daoqiang Zhang

cs.AIMar 5, 2026

Authorize-on-Demand: Dynamic Authorization with Legality-Aware Intellectual Property Protection for VLMs

Authors: Lianyu Wang, Meng Wang, Huazhu Fu, Daoqiang Zhang

Organizations: The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, China · Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, NUS, Singapore · Department of Ophthalmology, Yong Loo Lin School of Medicine, NUS, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore

Abstract

The rapid adoption of vision-language models (VLMs) has heightened the demand for robust intellectual property (IP) protection of these high-value pretrained models. Effective IP protection should proactively confine model deployment within authorized domains and prevent unauthorized transfers. However, existing methods rely on static training-time definitions, limiting flexibility in dynamic environments and often producing opaque responses to unauthorized inputs. To address these limitations, we propose a novel dynamic authorization with legality-aware intellectual property protection (AoD-IP) for VLMs, a framework that supports authorize-on-demand and legality-aware assessment. AoD-IP introduces a lightweight dynamic authorization module that enables flexible, user-controlled authorization, allowing users to actively specify or switch authorized domains on demand at deployment time. This enables the model to adapt seamlessly as application scenarios evolve and provides substantially greater extensibility than existing static-domain approaches. In addition, AoD-IP incorporates a dual-path inference mechanism that jointly predicts input legality-aware and task-specific outputs. Comprehensive experimental results on multiple cross-domain benchmarks demonstrate that AoD-IP maintains strong authorized-domain performance and reliable unauthorized detection, while supporting user-controlled authorization for adaptive deployment in dynamic environments.

Figures & tables

Figure 1: (a): Classical VLMs without IP protection; (b): Existing IP protection strategy (e.g., CUTI-Domain, CUPI-Domain) with static authorized domain; (c) The proposed AoD-IP allows users to actively specify or switch authorized domains on demand with legality-aware output.

Figure 2: (a) During training, authorized data ( $x_{a}$ ), extended data ( $x_{e}$ ), and unauthorized data ( $x_{u}$ ) are simultaneously processed by the frozen CLIP visual encoder $E^{v}$ to extract visual features ( $f_{a}^{v}$ , $f_{e}^{v}$ , $f_{u}^{v}$ ). The image projector $P_{\mathrm{img}}$ and domain projector $P_{\mathrm{dom}}$ generate image tokens ( $\tau_{a}^{g},\tau_{e}^{g},\tau_{u}^{g}$ ) and domain-discriminative tokens ( $\tau_{a}^{d},\tau_{e}^{d},\tau_{u}^{d}$ ) for the three domains, respectively. In parallel, an encryption projector $P_{\mathrm{enc}}$ produces a credential token $\tau_{a}^{c}$ for authorized data. These tokens are concatenated and fed into the frozen text encoder $E^{t}$ , producing the corresponding textual features ( $f_{a}^{t}$ , $f_{e}^{t}$ , $f_{u}^{t}$ ). The final prediction $p$ is derived from the similarity between visual features ( $f^{v}$ ) and textual features ( $f^{t}$ ), while an auxiliary output path verifies the legitimacy of each prediction. Frozen modules are indicated by snowflakes, and trainable modules by spark markers. (b) During inference, users may request additional credential tokens from the model owner, which function as “domain keys.” By selecting or switching these keys, users can dynamically control which domain is activated, thereby obtaining valid predictions accordingly. (c) Several inference cases are shown: only matching credentials and inputs lead to valid outputs (e.g., Cases A-C), while mismatched inputs result in invalid predictions and trigger security alerts (e.g., Cases D-F).

$X_{a}\rightarrow X_{u}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$R_{a}$	$R_{e}$	$R_{u}$
Am $\rightarrow$ Ds	78.8 $\rightarrow$ 0.0	We	97.5 $\rightarrow$ 0.0	100.0	98.5	100.0
Am $\rightarrow$ We	79.5 $\rightarrow$ 0.0	Ds	95.1 $\rightarrow$ 2.5	100.0	96.3	98.8
Ds $\rightarrow$ Am	95.7 $\rightarrow$ 0.0	We	94.3 $\rightarrow$ 0.0	97.5	97.5	99.4
Ds $\rightarrow$ We	97.2 $\rightarrow$ 0.0	Am	79.0 $\rightarrow$ 0.0	98.8	97.5	100.0
We $\rightarrow$ Am	93.1 $\rightarrow$ 0.0	Ds	94.6 $\rightarrow$ 1.3	100.0	98.8	98.8
We $\rightarrow$ Ds	92.6 $\rightarrow$ 0.0	Am	80.3 $\rightarrow$ 0.0	100.0	96.3	100.0

Table 1: The results of target-specified AoD-IP on the Office-31 [ 26 ] .

$X_{a}\rightarrow X_{u}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$X_{e^{2}}$	$A^{ip}_{e^{2}}\rightarrow A^{ip}_{u}$	$R_{a}$	$R_{e}$	$R_{u}$
Ar $\rightarrow$ Cl	84.8 $\rightarrow$ 2.8	Pr	92.3 $\rightarrow$ 5.0	Re	89.5 $\rightarrow$ 6.3	93.8	98.1	94.6
Ar $\rightarrow$ Pr	86.3 $\rightarrow$ 4.3	Cl	75.3 $\rightarrow$ 2.0	Re	89.8 $\rightarrow$ 18.0	99.0	95.9	91.1
Ar $\rightarrow$ Re	86.3 $\rightarrow$ 14.3	Cl	77.0 $\rightarrow$ 8.5	Pr	92.8 $\rightarrow$ 23.8	91.5	98.9	83.9
Cl $\rightarrow$ Ar	75.5 $\rightarrow$ 8.0	Pr	92.8 $\rightarrow$ 5.5	Re	89.3 $\rightarrow$ 14.5	93.8	95.6	86.1
Cl $\rightarrow$ Pr	73.5 $\rightarrow$ 1.8	Ar	85.0 $\rightarrow$ 4.8	Re	90.8 $\rightarrow$ 7.5	96.3	94.6	95.1
Cl $\rightarrow$ Re	78.0 $\rightarrow$ 5.8	Ar	85.8 $\rightarrow$ 10.8	Pr	92.3 $\rightarrow$ 11.3	93.8	92.8	90.3

Table 2: The results of target-specified AoD-IP on Office-Home-65 [ 31 ] .

$X_{a}\rightarrow X_{u}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$X_{e^{2}}$	$A^{ip}_{e^{2}}\rightarrow A^{ip}_{u}$	$R_{a}$	$R_{e}$	$R_{u}$
Cl $\rightarrow$ Pa	84.0 $\rightarrow$ 11.9	Re	90.9 $\rightarrow$ 12.6	Sk	81.3 $\rightarrow$ 12.1	94.1	95.7	86.4
Cl $\rightarrow$ Re	84.4 $\rightarrow$ 13.6	Pa	80.5 $\rightarrow$ 12.2	Sk	81.6 $\rightarrow$ 8.6	91.2	91.0	88.1
Cl $\rightarrow$ Sk	84.4 $\rightarrow$ 17.7	Pa	81.9 $\rightarrow$ 8.2	Re	91.3 $\rightarrow$ 7.4	91.1	87.5	88.5
Pa $\rightarrow$ Cl	86.4 $\rightarrow$ 9.5	Re	90.2 $\rightarrow$ 16.1	Sk	77.6 $\rightarrow$ 13.5	89.9	89.8	86.3
Pa $\rightarrow$ Re	80.8 $\rightarrow$ 23.1	Cl	85.0 $\rightarrow$ 15.7	Sk	83.3 $\rightarrow$ 13.8	93.3	92.5	81.9
Pa $\rightarrow$ Sk	79.8 $\rightarrow$ 10.3	Cl	85.8 $\rightarrow$ 13.6	Re	90.7 $\rightarrow$ 12.1	92.0	91.0	86.5

Table 3: The results of target-specified AoD-IP on the Mini-DomainNet [ 48 ] .

$X_{a}/X_{u}$	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Ar	85.5 $\Rightarrow$ 85.4	68.0 $\Rightarrow$ 2.7	89.8 $\Rightarrow$ 4.2	88.5 $\Rightarrow$ 14.8	63.83	74.88	0.12
Cl	81.0 $\Rightarrow$ 5.1	75.0 $\Rightarrow$ 74.9	90.8 $\Rightarrow$ 3.3	89.5 $\Rightarrow$ 6.2	61.57	82.24	0.07
Pr	78.8 $\Rightarrow$ 4.8	73.3 $\Rightarrow$ 4.3	92.8 $\Rightarrow$ 92.6	87.5 $\Rightarrow$ 16.9	65.69	71.23	0.25
Re	83.0 $\Rightarrow$ 16.8	71.3 $\Rightarrow$ 3.9	90.8 $\Rightarrow$ 14.7	90.0 $\Rightarrow$ 89.9	62.79	69.92	0.09
Mean	/				63.47	74.57	0.13

Table 4: The accuracy (

\%

) of target-specified AoD-IP on Office-Home-65 [ 31 ] .

Datasets	$X_{a}$	$W_{u-a}\uparrow$							$Drop_{u}\uparrow$							$Drop_{a}\downarrow$
Datasets	$X_{a}$	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD
Office-31 [ 26 ]	Am	56.34	62.06	62.88	55.76	60.00	63.52	69.98	74.40	79.35	80.20	80.35	84.40	80.00	88.15	1.80	0.60	0.50	5.20	4.40	0.00	0.00
	Ds	76.09	80.13	80.75	82.87	80.81	82.54	86.43	81.90	85.05	85.70	88.70	86.90	86.25	90.36	1.30	0.70	0.70	1.10	1.30	0.00	0.00
	We	32.50	75.24	77.15	70.44	77.20	78.45	81.40	38.70	84.38	84.55	81.55	83.75	83.10	86.25	3.10	2.50	1.50	3.80	1.05	0.00	0.00
	Mean	54.98	72.48	73.60	69.69	72.67	74.84	79.27 ∗	65.00	82.93	83.48	83.53	85.02	83.12	88.25 ∗	2.07	1.27	0.90	3.37	2.25	0.00	0.00 ∗
Office- Home-65 [ 31 ]	Ar	13.44	41.58	44.12	38.81	18.99	52.00	63.83	15.83	53.40	55.00	70.77	38.33	61.33	74.88	0.10	3.00	2.10	15.40	12.37	0.30	0.12

Table 5:

W_{u-a}

Drop_{u}

, and

Drop_{a}

of target-specified NTL, CUTI, CUPI, HNTL, SOPHON, IP-CLIP and AoD-IP. The best performance is indicated by the numbers in bold. Statistical significance (p-value

<

0.05 [ 14 , 4 ] ) is denoted with: ∗ (AoD-IP vs. others).

$X_{a}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$X_{e^{2}}$	$A^{ip}_{e^{2}}\rightarrow A^{ip}_{u}$	$X_{e^{3}}$	$A^{ip}_{e^{3}}\rightarrow A^{ip}_{u}$	$R_{a}$ / $R_{e}$ / $R_{u}$
Ar $\dagger$	82.5 $\rightarrow$ 1.3 / 2.3 / 7.5 / 1.8	Cl $\dagger$	56.5 $\rightarrow$ 2.0 / 4.0 / 8.0 / 7.5	Pr $\dagger$	83.3 $\rightarrow$ 1.5 / 3.8 / 7.5 / 1.8	Re $\dagger$	84.5 $\rightarrow$ 1.0 / 1.8 / 3.0 / 1.0	99.5 / 99.8 / 94.7
Cl $\dagger$	56.3 $\rightarrow$ 6.8 / 9.0 / 11.8 / 11.8	Ar $\dagger$	81.3 $\rightarrow$ 2.5 / 1.5 / 8.0 / 4.8	Pr $\dagger$	82.5 $\rightarrow$ 2.3 / 2.3 / 10.0 / 4.8	Re $\dagger$	88.0 $\rightarrow$ 2.0 / 2.3 / 8.0 / 3.5	100.0 / 99.8 / 90.3
Pr $\dagger$	83.5 $\rightarrow$ 1.8 / 2.3 / 7.3 / 1.3	Ar $\dagger$	82.3 $\rightarrow$ 0.8 / 1.0 / 2.8 / 1.3	Cl $\dagger$	56.8 $\rightarrow$ 0.8 / 3.3 / 4.5 / 1.5	Re $\dagger$	89.4 $\rightarrow$ 1.0 / 1.5 / 2.8 / 1.5	100.0 / 99.8 / 96.4
Re $\dagger$	88.0 $\rightarrow$ 2.8 / 4.8 / 10.3 / 3.3	Ar $\dagger$	81.0 $\rightarrow$ 1.8 / 2.5 / 6.3 / 4.0	Cl $\dagger$	56.5 $\rightarrow$ 1.8 / 3.5 / 5.5 / 3.0	Pr $\dagger$	87.0 $\rightarrow$ 1.3 / 2.0 / 8.5 / 1.8	100.0 / 100.0 / 92.5

Table 6: The detailed results of authorization application AoD-IP on Office-Home-65 [ 31 ] .

Datasets	$X_{a}$	$D_{u-a}\uparrow$							$A_{u}^{ip}\downarrow$							$A_{a}^{ip}\uparrow$
Datasets	$X_{a}$	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD	NTL [ 37 ]	CUTI [ 35 ]	CUPI [ 33 ]	HNTL [ 15 ]	SOPH [ 12 ]	IPC [ 34 ]	AoD
Office-31 [ 26 ]	Am $\dagger$	15.67	29.26	31.51	5.55	39.11	37.46	44.64	37.43	20.83	16.92	5.73	9.57	3.53	2.61	62.50	65.50	65.23	26.60	67.50	63.00	68.13
	Ds $\dagger$	39.25	54.47	58.62	73.19	85.31	82.42	87.55	50.50	36.53	33.30	21.37	10.00	9.77	7.83	92.80	94.30	95.00	96.90	97.50	95.80	97.57
	We $\dagger$	54.59	40.56	49.70	78.36	74.09	56.45	75.96	21.30	30.60	25.68	18.77	12.40	15.53	11.07	85.30	80.80	84.50	98.40	92.50	83.30	92.87
	Mean	36.50	41.43	46.61	52.37	66.17	58.78	69.39 ∗	36.41	29.32	25.30	15.29	10.66	9.61	7.17 ∗	80.20	80.20	81.58	73.97	85.83	80.70	86.19 ∗
Office- Home-65 [ 31 ]	Ar $\dagger$	49.47	54.95	56.53	45.23	50.14	60.12	64.29	20.45	10.38	8.39	2.48	6.25	3.88	3.11	81.30	79.50	79.50	68.50	74.00	79.50	81.75

Table 7:

D_{u-a}

A_{u}^{ip}

, and

A_{a}^{ip}

of authorization application NTL, CUTI, CUPI, HNTL, SOPHON, IP-CLIP and AoD-IP.

Figure 10

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 77.6	87.5 $\Rightarrow$ 10.0	88.8 $\Rightarrow$ 17.5	56.34	74.40	1.80
Dslr	83.8 $\Rightarrow$ 10.0	95.7 $\Rightarrow$ 94.4	98.8 $\Rightarrow$ 8.8	76.09	81.90	1.30
Webcam	80.0 $\Rightarrow$ 3.8	92.5 $\Rightarrow$ 91.3	94.4 $\Rightarrow$ 91.3	32.50	38.70	3.10
Mean	/			54.98	65.00	2.07

Table 1: The accuracy (

\%

) of target-specified NTL on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of NTL.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 78.8	87.5 $\Rightarrow$ 6.3	88.8 $\Rightarrow$ 11.3	62.06	79.35	0.60
Dslr	83.8 $\Rightarrow$ 5.0	95.7 $\Rightarrow$ 95.0	98.8 $\Rightarrow$ 7.5	80.13	85.05	0.70
Webcam	80.0 $\Rightarrow$ 1.3	92.5 $\Rightarrow$ 2.5	94.4 $\Rightarrow$ 91.9	75.24	84.38	2.50
Mean	/			72.48	82.93	1.27

Table 2: The accuracy (

\%

) of target-specified CUTI-Domain on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUTI-Domain.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 78.9	87.5 $\Rightarrow$ 5.2	88.8 $\Rightarrow$ 10.7	62.88	80.20	0.50
Dslr	83.8 $\Rightarrow$ 4.3	95.7 $\Rightarrow$ 95.0	98.8 $\Rightarrow$ 6.9	80.75	85.70	0.70
Webcam	80.0 $\Rightarrow$ 0.9	92.5 $\Rightarrow$ 2.5	94.4 $\Rightarrow$ 92.9	77.15	84.55	1.50
Mean	/			73.60	83.48	0.90

Table 3: The accuracy (

\%

) of target-specified CUPI-Domain on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUPI-Domain.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 74.2	87.5 $\Rightarrow$ 12.5	88.8 $\Rightarrow$ 3.1	55.76	80.35	5.20
Dslr	83.8 $\Rightarrow$ 3.1	95.7 $\Rightarrow$ 94.6	98.8 $\Rightarrow$ 2.1	82.87	88.70	1.10
Webcam	80.0 $\Rightarrow$ 6.3	92.5 $\Rightarrow$ 3.1	94.4 $\Rightarrow$ 90.6	70.44	81.55	3.80
Mean	/			69.69	83.53	3.37

Table 4: The accuracy (

\%

) of target-specified HNTL on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of HNTL.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 75.0	87.5 $\Rightarrow$ 2.5	88.8 $\Rightarrow$ 5.0	60.00	84.40	4.40
Dslr	83.8 $\Rightarrow$ 3.8	95.7 $\Rightarrow$ 94.4	98.8 $\Rightarrow$ 5.0	80.81	86.90	1.30
Webcam	80.0 $\Rightarrow$ 2.5	92.5 $\Rightarrow$ 2.5	94.4 $\Rightarrow$ 93.4	77.20	83.75	1.05
Mean	/			72.67	85.02	2.25

Table 5: The accuracy (

\%

) of target-specified SOPHON on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of SOPHON.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 79.4	87.5 $\Rightarrow$ 7.5	88.8 $\Rightarrow$ 8.8	63.52	80.00	0.00
Dslr	83.8 $\Rightarrow$ 3.8	95.7 $\Rightarrow$ 95.7	98.8 $\Rightarrow$ 6.3	82.54	86.25	0.00
Webcam	80.0 $\Rightarrow$ 3.8	92.5 $\Rightarrow$ 2.5	94.4 $\Rightarrow$ 94.4	78.45	83.10	0.00
Mean	/			74.84	83.12	0.00

Table 6: The accuracy (

\%

) of target-specified IP-CLIP on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of IP-CLIP.

Authorized/Unauthorized	Amazon	Dslr	Webcam	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Amazon	79.4 $\Rightarrow$ 79.4	87.5 $\Rightarrow$ 0.0	88.8 $\Rightarrow$ 0.0	69.98	88.15	0.00
Dslr	83.8 $\Rightarrow$ 0.6	95.7 $\Rightarrow$ 95.7	98.8 $\Rightarrow$ 1.3	86.43	90.36	0.00
Webcam	80.0 $\Rightarrow$ 0.0	92.5 $\Rightarrow$ 0.0	94.4 $\Rightarrow$ 94.4	81.40	86.25	0.00
Mean	/			79.27	88.25	0.00

Table 7: The accuracy (

\%

) of target-specified AoD-IP on Office-31. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of AoD-IP.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 85.4	68.0 $\Rightarrow$ 23.8	89.8 $\Rightarrow$ 86.5	88.5 $\Rightarrow$ 88.5	13.44	15.83	0.10
Clipart	81.0 $\Rightarrow$ 18.3	75.0 $\Rightarrow$ 74.7	90.8 $\Rightarrow$ 15.0	89.5 $\Rightarrow$ 31.0	48.83	65.67	0.30
Product	78.8 $\Rightarrow$ 11.0	73.3 $\Rightarrow$ 13.8	92.8 $\Rightarrow$ 92.8	87.5 $\Rightarrow$ 85.8	39.90	43.00	0.00
Real-World	83.0 $\Rightarrow$ 80.8	71.3 $\Rightarrow$ 7.8	90.8 $\Rightarrow$ 52.5	90.0 $\Rightarrow$ 88.1	28.87	34.67	1.90
Mean	/				32.76	39.79	0.57

Table 8: The accuracy (

\%

) of target-specified NTL on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of NTL.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 82.5	68.0 $\Rightarrow$ 16.8	89.8 $\Rightarrow$ 21.3	88.5 $\Rightarrow$ 48.0	41.58	53.40	3.00
Clipart	81.0 $\Rightarrow$ 15.8	75.0 $\Rightarrow$ 75.0	90.8 $\Rightarrow$ 9.0	89.5 $\Rightarrow$ 19.3	53.37	72.40	0.63
Product	78.8 $\Rightarrow$ 16.3	73.3 $\Rightarrow$ 10.8	92.8 $\Rightarrow$ 92.4	87.5 $\Rightarrow$ 27.0	56.82	61.83	0.37
Real-World	83.0 $\Rightarrow$ 30.3	71.3 $\Rightarrow$ 10.0	90.8 $\Rightarrow$ 32.8	90.0 $\Rightarrow$ 88.5	49.41	57.33	1.50
Mean	/				50.29	61.24	1.38

Table 9: The accuracy (

\%

) of target-specified CUTI-Domain on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUTI-Domain.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 83.4	68.0 $\Rightarrow$ 13.0	89.8 $\Rightarrow$ 21.0	88.5 $\Rightarrow$ 47.3	44.12	55.00	2.10
Clipart	81.0 $\Rightarrow$ 15.0	75.0 $\Rightarrow$ 74.9	90.8 $\Rightarrow$ 10.0	89.5 $\Rightarrow$ 16.1	54.90	73.40	0.10
Product	78.8 $\Rightarrow$ 15.9	73.3 $\Rightarrow$ 9.8	92.8 $\Rightarrow$ 92.5	87.5 $\Rightarrow$ 25.1	57.94	62.93	0.30
Real-World	83.0 $\Rightarrow$ 25.4	71.3 $\Rightarrow$ 10.0	90.8 $\Rightarrow$ 26.7	90.0 $\Rightarrow$ 89.5	54.15	61.00	0.50
Mean	/				52.78	63.08	0.75

Table 10: The accuracy (

\%

) of target-specified CUPI-Domain on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUPI-Domain.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 70.1	68.0 $\Rightarrow$ 4.1	89.8 $\Rightarrow$ 7.0	88.5 $\Rightarrow$ 22.9	38.81	70.77	15.40
Clipart	81.0 $\Rightarrow$ 5.5	75.0 $\Rightarrow$ 46.2	90.8 $\Rightarrow$ 5.2	89.5 $\Rightarrow$ 9.1	23.85	80.50	28.83
Product	78.8 $\Rightarrow$ 4.7	73.3 $\Rightarrow$ 5.7	92.8 $\Rightarrow$ 70.7	87.5 $\Rightarrow$ 27.3	31.96	67.30	22.10
Real-World	83.0 $\Rightarrow$ 35.9	71.3 $\Rightarrow$ 6.2	90.8 $\Rightarrow$ 26.6	90.0 $\Rightarrow$ 78.8	37.51	58.80	11.20
Mean	/				33.03	69.34	19.38

Table 11: The accuracy (

\%

) of target-specified HNTL on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of HNTL.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 73.1	68.0 $\Rightarrow$ 21.5	89.8 $\Rightarrow$ 45.8	88.5 $\Rightarrow$ 64.0	18.99	38.33	12.37
Clipart	81.0 $\Rightarrow$ 13.5	75.0 $\Rightarrow$ 74.5	90.8 $\Rightarrow$ 21.2	89.5 $\Rightarrow$ 22.0	50.44	68.20	0.50
Product	78.8 $\Rightarrow$ 29.2	73.3 $\Rightarrow$ 18.2	92.8 $\Rightarrow$ 89.0	87.5 $\Rightarrow$ 58.5	36.28	44.57	3.80
Real-World	83.0 $\Rightarrow$ 51.5	71.3 $\Rightarrow$ 24.0	90.8 $\Rightarrow$ 61.3	90.0 $\Rightarrow$ 80.5	21.38	36.10	9.53
Mean	/				31.77	46.80	6.55

Table 12: The accuracy (

\%

) of target-specified SOPHON on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of SOPHON.

Authorized/Unauthorized	Art	Clipart	Product	Real-World	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Art	85.5 $\Rightarrow$ 85.2	68.0 $\Rightarrow$ 12.8	89.8 $\Rightarrow$ 15.0	88.5 $\Rightarrow$ 34.5	52.00	61.33	0.30
Clipart	81.0 $\Rightarrow$ 11.8	75.0 $\Rightarrow$ 74.9	90.8 $\Rightarrow$ 5.3	89.5 $\Rightarrow$ 17.8	56.45	75.47	0.10
Product	78.8 $\Rightarrow$ 14.0	73.3 $\Rightarrow$ 8.5	92.8 $\Rightarrow$ 92.5	87.5 $\Rightarrow$ 25.8	58.71	63.77	0.30
Real-World	83.0 $\Rightarrow$ 30.3	71.3 $\Rightarrow$ 7.5	90.8 $\Rightarrow$ 29.3	90.0 $\Rightarrow$ 89.9	53.25	59.33	0.10
Mean	/				55.10	64.98	0.20

Table 13: The accuracy (

\%

) of target-specified IP-CLIP on Office-Home-65. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of IP-CLIP.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 84.5	79.8 $\Rightarrow$ 33.3	89.8 $\Rightarrow$ 34.0	78.7 $\Rightarrow$ 42.1	38.62	46.30	0.60
Painting	83.8 $\Rightarrow$ 11.9	81.4 $\Rightarrow$ 79.8	89.1 $\Rightarrow$ 60.5	78.4 $\Rightarrow$ 17.5	41.66	53.80	1.60
Real	84.6 $\Rightarrow$ 26.4	80.5 $\Rightarrow$ 30.8	90.6 $\Rightarrow$ 89.8	80.0 $\Rightarrow$ 10.8	52.29	59.03	0.80
Sketch	84.3 $\Rightarrow$ 83.0	79.1 $\Rightarrow$ 32.7	90.3 $\Rightarrow$ 9.7	80.7 $\Rightarrow$ 80.1	33.78	42.77	0.60
Mean	/				41.59	50.48	1.00

Table 14: The accuracy (

\%

) of target-specified NTL on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of NTL.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 84.9	79.8 $\Rightarrow$ 22.7	89.8 $\Rightarrow$ 23.7	78.7 $\Rightarrow$ 23.7	50.26	59.40	0.20
Painting	83.8 $\Rightarrow$ 17.9	81.4 $\Rightarrow$ 76.1	89.1 $\Rightarrow$ 15.6	78.4 $\Rightarrow$ 17.1	46.88	66.90	5.30
Real	84.6 $\Rightarrow$ 24.9	80.5 $\Rightarrow$ 23.8	90.6 $\Rightarrow$ 89.5	80.0 $\Rightarrow$ 9.5	54.77	62.30	1.10
Sketch	84.3 $\Rightarrow$ 24.0	79.1 $\Rightarrow$ 25.2	90.3 $\Rightarrow$ 10.8	80.7 $\Rightarrow$ 80.0	51.09	64.57	0.70
Mean	/				50.75	63.29	2.20

Table 15: The accuracy (

\%

) of target-specified CUTI-Domain on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUTI-Domain.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 84.9	79.8 $\Rightarrow$ 20.8	89.8 $\Rightarrow$ 21.4	78.7 $\Rightarrow$ 19.9	52.52	62.07	0.20
Painting	83.8 $\Rightarrow$ 16.7	81.4 $\Rightarrow$ 78.3	89.1 $\Rightarrow$ 13.7	78.4 $\Rightarrow$ 16.9	50.82	68.00	3.10
Real	84.6 $\Rightarrow$ 25.1	80.5 $\Rightarrow$ 20.7	90.6 $\Rightarrow$ 89.7	80.0 $\Rightarrow$ 8.9	56.12	63.47	0.90
Sketch	84.3 $\Rightarrow$ 22.4	79.1 $\Rightarrow$ 23.7	90.3 $\Rightarrow$ 9.4	80.7 $\Rightarrow$ 80.3	52.73	66.07	0.40
Mean	/				53.05	64.90	1.40

Table 16: The accuracy (

\%

) of target-specified CUPI-Domain on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of CUPI-Domain..

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 59.6	79.8 $\Rightarrow$ 7.2	89.8 $\Rightarrow$ 11.2	78.7 $\Rightarrow$ 5.3	29.42	74.87	25.50
Painting	83.8 $\Rightarrow$ 6.1	81.4 $\Rightarrow$ 63.3	89.1 $\Rightarrow$ 18.3	78.4 $\Rightarrow$ 3.6	35.66	74.43	18.10
Real	84.6 $\Rightarrow$ 12.8	80.5 $\Rightarrow$ 19.9	90.6 $\Rightarrow$ 84.1	80.0 $\Rightarrow$ 6.6	52.23	68.60	6.50
Sketch	84.3 $\Rightarrow$ 4.9	79.1 $\Rightarrow$ 3.6	90.3 $\Rightarrow$ 6.3	80.7 $\Rightarrow$ 42.0	17.19	79.63	38.70
Mean	/				33.62	74.38	16.70

Table 17: The accuracy (

\%

) of target-specified HNTL on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of HNTL.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 78.9	79.8 $\Rightarrow$ 25.2	89.8 $\Rightarrow$ 40.0	78.7 $\Rightarrow$ 25.7	36.55	52.47	6.17
Painting	83.8 $\Rightarrow$ 33.7	81.4 $\Rightarrow$ 70.2	89.1 $\Rightarrow$ 65.6	78.4 $\Rightarrow$ 22.9	22.38	43.03	11.17
Real	84.6 $\Rightarrow$ 36.7	80.5 $\Rightarrow$ 42.7	90.6 $\Rightarrow$ 81.4	80.0 $\Rightarrow$ 18.7	32.40	49.00	9.20
Sketch	84.3 $\Rightarrow$ 35.4	79.1 $\Rightarrow$ 30.6	90.3 $\Rightarrow$ 39.4	80.7 $\Rightarrow$ 72.2	29.55	49.43	8.50
Mean	/				30.22	48.48	8.84

Table 18: The accuracy (

\%

) of target-specified SOPHON on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of SOPHON.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 84.8	79.8 $\Rightarrow$ 18.1	89.8 $\Rightarrow$ 24.8	78.7 $\Rightarrow$ 22.4	51.47	61.00	0.30
Painting	83.8 $\Rightarrow$ 9.2	81.4 $\Rightarrow$ 80.9	89.1 $\Rightarrow$ 26.5	78.4 $\Rightarrow$ 14.4	53.85	67.07	0.50
Real	84.6 $\Rightarrow$ 21.9	80.5 $\Rightarrow$ 21.4	90.6 $\Rightarrow$ 90.4	80.0 $\Rightarrow$ 6.0	58.82	65.27	0.20
Sketch	84.3 $\Rightarrow$ 23.2	79.1 $\Rightarrow$ 15.6	90.3 $\Rightarrow$ 9.2	80.7 $\Rightarrow$ 80.2	54.59	68.57	0.50
Mean	/				54.68	65.48	0.33

Table 19: The accuracy (

\%

) of target-specified IP-CLIP on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of IP-CLIP.

Authorized/Unauthorized	Clipart	Painting	Real	Sketch	$W_{u-a}\uparrow$	$Drop_{u}\uparrow$	$Drop_{a}\downarrow$
Clipart	85.1 $\Rightarrow$ 85.1	79.8 $\Rightarrow$ 12.3	89.8 $\Rightarrow$ 16.3	78.7 $\Rightarrow$ 17.0	57.47	67.58	0.02
Painting	83.8 $\Rightarrow$ 10.4	81.4 $\Rightarrow$ 81.0	89.1 $\Rightarrow$ 17.0	78.4 $\Rightarrow$ 9.1	57.60	71.59	0.44
Real	84.6 $\Rightarrow$ 14.6	80.5 $\Rightarrow$ 14.1	90.6 $\Rightarrow$ 90.4	80.0 $\Rightarrow$ 10.7	61.76	68.58	0.23
Sketch	84.3 $\Rightarrow$ 15.9	79.1 $\Rightarrow$ 11.2	90.3 $\Rightarrow$ 12.3	80.7 $\Rightarrow$ 80.3	57.12	71.46	0.36
Mean	/				58.49	69.80	0.26

Table 20: The accuracy (

\%

) of target-specified AoD-IP on Mini-DomainNet. The vertical/horizontal axis denotes the authorized/unauthorized domain. In each task, the left of ’

\Rightarrow

’ shows the test accuracy of SL-CLIP on the unauthorized domain, while the right side presents the accuracy of AoD-IP.

$X_{a}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$X_{e^{2}}$	$A^{ip}_{e^{2}}\rightarrow A^{ip}_{u}$	$R_{a}$ / $R_{e}$ / $R_{u}$
Amazon $\dagger$	68.1 $\rightarrow$ 4.3 / 5.1 / 4.2	Dslr $\dagger$	97.7 $\rightarrow$ 20.6 / 1.2 / 0.3	Webcam $\dagger$	94.7 $\rightarrow$ 28.6 / 3.2 / 9.8	91.5 / 94.8 / 89.4
Dslr $\dagger$	97.8 $\rightarrow$ 22.5 / 1.3 / 0.2	Amazon $\dagger$	67.8 $\rightarrow$ 3.1 / 2.5 / 1.6	Webcam $\dagger$	91.9 $\rightarrow$ 25.4 / 0.8 / 2.4	100.0 / 98.5 / 92.3
Webcam $\dagger$	92.0 $\rightarrow$ 25.8 / 1.2 / 2.4	Amazon $\dagger$	68.5 $\rightarrow$ 0.8 / 1.3 / 0.6	Dslr $\dagger$	97.2 $\rightarrow$ 22.1 / 1.0 / 1.3	96.0 / 96.4 / 93.0

Table 21: The detailed results of authorization application AoD-IP on Office-31.

$X_{a}$	$A^{ip}_{a}\rightarrow A^{ip}_{u}$	$X_{e^{1}}$	$A^{ip}_{e^{1}}\rightarrow A^{ip}_{u}$	$X_{e^{2}}$	$A^{ip}_{e^{2}}\rightarrow A^{ip}_{u}$	$X_{e^{3}}$	$A^{ip}_{e^{3}}\rightarrow A^{ip}_{u}$	$R_{a}$ / $R_{e}$ / $R_{u}$
Cl $\dagger$	77.5 $\rightarrow$ 4.4 / 4.4 / 2.9 / 16.8	Pa $\dagger$	72.4 $\rightarrow$ 4.0 / 2.4 / 3.8 / 12.4	Re $\dagger$	83.4 $\rightarrow$ 1.9 / 1.7 / 1.6 / 11.6	Sk $\dagger$	76.1 $\rightarrow$ 0.8 / 1.0 / 1.6 / 4.0	100.0 / 99.9 / 92.5
Pa $\dagger$	70.0 $\rightarrow$ 6.7 / 7.9 / 4.5 / 13.0	Cl $\dagger$	77.9 $\rightarrow$ 2.9 / 1.6 / 2.9 / 20.0	Re $\dagger$	83.8 $\rightarrow$ 5.7 / 4.6 / 4.6 / 13.5	Sk $\dagger$	77.9 $\rightarrow$ 2.1 / 1.4 / 1.7 / 9.0	100.0 / 100.0 / 91.9
Re $\dagger$	83.2 $\rightarrow$ 5.4 / 2.4 / 1.9 / 13.0	Cl $\dagger$	77.3 $\rightarrow$ 1.6 / 1.0 / 1.4 / 4.4	Pa $\dagger$	72.8 $\rightarrow$ 5.1 / 2.7 / 3.8 / 13.3	Sk $\dagger$	76.0 $\rightarrow$ 1.3 / 0.5 / 1.0 / 2.2	100.0 / 99.5 / 93.8
Sk $\dagger$	75.1 $\rightarrow$ 3.5 / 1.3 / 3.8 / 5.9	Cl $\dagger$	78.9 $\rightarrow$ 2.4 / 1.4 / 3.8 / 7.5	Pa $\dagger$	71.9 $\rightarrow$ 2.5 / 1.1 / 3.7 / 8.4	Re $\dagger$	84.2 $\rightarrow$ 1.6 / 0.5 / 0.8 / 3.8	99.9 / 99.8 / 95.3

Table 22: The detailed results of authorization application AoD-IP on Mini-DomainNet.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	57.5	13.8	41.0	15.67	37.43	62.50
Dslr $\dagger$	78.7	18.5	54.3	39.25	50.50	92.80
Webcam $\dagger$	31.8	17.3	14.8	54.59	21.30	85.30
Mean	/			36.50	36.41	80.20

Table 23: The results of authorization application NTL on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	36.0	7.5	19.0	29.26	20.83	65.50
Dslr $\dagger$	74.0	6.3	29.3	54.47	36.53	94.30
Webcam $\dagger$	57.0	12.5	22.3	40.56	30.60	80.80
Mean	/			41.43	29.32	80.20

Table 24: The results of authorization application CUTI-Domain on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	25.1	7.2	18.5	31.51	16.92	65.23
Dslr $\dagger$	70.1	6.2	23.6	58.62	33.30	95.00
Webcam $\dagger$	45.9	12.3	18.8	49.70	25.68	84.50
Mean	/			46.61	25.30	81.58

Table 25: The results of authorization application CUPI-Domain on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	6.30	7.80	3.10	5.55	5.73	26.60
Dslr $\dagger$	46.90	6.30	10.90	73.19	21.37	96.90
Webcam $\dagger$	34.40	15.60	6.30	78.36	18.77	98.40
Mean	/			52.37	15.29	73.97

Table 26: The results of authorization application HNTL on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	11.20	7.50	10.00	39.11	9.57	67.50
Dslr $\dagger$	7.50	10.00	12.50	85.31	10.00	97.50
Webcam $\dagger$	12.50	11.20	13.50	74.09	12.40	92.50
Mean	/			66.17	10.66	85.83

Table 27: The results of authorization application SOPHON on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	4.5	3.3	2.8	37.46	3.53	63.00
Dslr $\dagger$	27.3	1.5	0.5	82.42	9.77	95.80
Webcam $\dagger$	31.0	4.3	11.3	56.45	15.53	83.30
Mean	/			58.78	9.61	80.70

Table 28: The results of authorization application IP-CLIP on Office-31.

Authorized/Test	Amazon	Dslr	Webcam	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Amazon $\dagger$	2.7	3.0	2.1	44.64	2.61	68.13
Dslr $\dagger$	21.7	1.2	0.6	87.55	7.83	97.57
Webcam $\dagger$	26.6	1.7	4.9	75.96	11.07	92.87
Mean	/			69.39	7.17	86.19

Table 29: The results of authorization application AoD-IP on Office-31.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	21.0	14.3	17.0	29.5	49.47	20.45	81.30
Clipart $\dagger$	21.0	13.0	45.0	31.8	9.74	27.70	48.00
Product $\dagger$	21.3	27.8	36.8	24.0	44.44	27.48	81.80
Real-World $\dagger$	10.3	22.5	27.5	16.5	51.50	19.20	82.00
Mean	/				38.79	23.71	73.28

Table 30: The results of authorization application NTL on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	4.5	5.0	21.0	11.0	54.95	10.38	79.50
Clipart $\dagger$	11.0	16.0	36.5	20.0	16.86	20.88	52.80
Product $\dagger$	18.0	33.5	61.0	29.0	39.53	35.38	83.00
Real-World $\dagger$	7.5	3.5	10.8	9.5	62.87	7.83	83.30
Mean	/				43.55	18.61	74.65

Table 31: The results of authorization application CUTI-Domain on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	4.1	5.2	15.1	9.2	56.53	8.39	79.50
Clipart $\dagger$	8.3	13.4	25.5	18.5	23.03	16.43	56.90
Product $\dagger$	15.1	21.7	56.8	27.7	44.41	30.31	83.50
Real-World $\dagger$	5.3	3.5	8.6	8.8	66.03	6.55	84.60
Mean	/				47.50	15.42	76.13

Table 32: The results of authorization application CUPI-Domain on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	2.1	4.1	2.1	1.6	45.23	2.48	68.50
Clipart $\dagger$	1.6	3.1	2.1	1.3	17.62	2.03	43.00
Product $\dagger$	2.1	4.4	3.4	3.4	28.10	3.33	54.70
Real-World $\dagger$	2.6	3.6	2.1	3.1	71.00	2.85	85.70
Mean	/				40.49	2.67	62.98

Table 33: The results of authorization application HNTL on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	3.8	4.8	7.2	9.2	50.14	6.25	74.00
Clipart $\dagger$	5.5	16.2	13.0	15.2	26.61	12.48	58.20
Product $\dagger$	2.0	3.8	10.2	4.5	64.64	5.13	83.00
Real-World $\dagger$	4.8	4.0	13.2	10.0	59.44	8.00	81.20
Mean	/				50.21	7.96	74.10

Table 34: The results of authorization application SOPHON on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art	1.5	3.3	7.8	3.0	60.12	3.88	79.50
Clipart	4.3	5.3	22.5	9.8	26.52	10.48	57.00
Product	5.8	9.3	12.0	6.5	57.74	8.40	80.30
Real-World	2.3	4.0	8.5	6.0	71.17	5.20	87.00
Mean	/				53.89	6.99	75.95

Table 35: The results of authorization application IP-CLIP on Office-Home-65.

Authorized/Test	Art	Clipart	Product	Real-World	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Art $\dagger$	1.6	1.8	8.6	2.9	64.29	3.11	81.75
Clipart $\dagger$	2.8	4.9	15.7	5.9	28.95	5.28	56.51
Product $\dagger$	1.7	2.6	8.3	2.4	67.55	3.73	84.08
Real-World $\dagger$	1.7	2.6	6.0	2.3	73.77	3.14	87.48
Mean	/				58.64	3.82	77.45

Table 36: The results of authorization application AoD-IP on Office-Home-65.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	13.5	8.7	12.1	35.9	38.45	17.54	71.40
Painting $\dagger$	26.2	15.1	14.3	41.1	32.78	24.18	70.60
Real $\dagger$	41.1	21.8	24.6	64.1	35.66	37.90	81.60
Sketch $\dagger$	16.5	7.8	11.3	30.6	38.66	16.55	71.00
Mean	/				36.39	24.04	73.65

Table 37: The results of authorization application NTL on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	57.9	24.3	31.1	63.0	22.77	44.08	74.60
Painting $\dagger$	46.5	13.8	21.6	57.0	24.48	34.73	69.80
Real $\dagger$	42.4	17.9	21.9	57.1	33.56	34.83	77.90
Sketch $\dagger$	6.0	4.8	6.4	20.6	48.18	9.45	74.30
Mean	/				32.25	30.77	74.15

Table 38: The results of authorization application CUTI-Domain on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	30.5	18.9	26.7	40.7	34.35	29.21	75.00
Painting $\dagger$	30.7	12.1	16.7	18.2	35.64	19.43	70.20
Real $\dagger$	29.4	15.1	18.3	37.5	44.35	25.08	80.30
Sketch $\dagger$	6.5	4.2	5.8	15.6	50.24	8.02	75.00
Mean	/				41.14	20.43	75.13

Table 39: The results of authorization application CUPI-Domain on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	1.8	0.3	1.6	0.8	2.60	1.11	16.70
Painting $\dagger$	0.3	0.3	0.5	0.3	0.47	0.35	7.00
Real $\dagger$	1.0	0.5	1.0	1.3	2.05	0.95	14.80
Sketch $\dagger$	0.5	0.8	1.3	1.3	0.42	0.98	7.00
Mean	/				1.39	0.85	11.38

Table 40: The results of authorization application HNTL on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	24.1	5.1	8.4	15.9	50.55	13.38	78.10
Painting $\dagger$	16.0	10.6	11.4	14.6	41.59	13.15	71.40
Real $\dagger$	11.7	6.3	9.4	4.4	60.10	7.95	81.60
Sketch $\dagger$	10.0	5.9	7.0	3.7	45.15	6.65	70.60
Mean	/				49.35	10.28	75.43

Table 41: The results of authorization application SOPHON on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	4.6	5.1	4.8	14.9	50.88	7.35	75.10
Painting $\dagger$	7.5	4.6	8.4	23.2	40.33	10.93	69.20
Real $\dagger$	17.3	8.4	9.8	38.1	54.06	18.40	83.30
Sketch $\dagger$	4.6	3.8	5.7	18.7	48.27	8.20	73.70
Mean	/				48.39	11.22	75.33

Table 42: The results of authorization application IP-CLIP on Mini-DomainNet.

Authorized/Test	Clipart	Painting	Real	Sketch	$D_{u-a}\uparrow$	$A_{u}\downarrow$	$A_{a}\uparrow$
Clipart $\dagger$	2.8	2.1	2.8	12.2	56.82	4.96	77.90
Painting $\dagger$	4.6	3.5	4.0	11.8	47.24	5.96	71.78
Real $\dagger$	3.7	2.3	2.2	10.5	66.07	4.66	83.65
Sketch $\dagger$	1.9	1.1	2.0	5.3	56.22	2.57	76.28
Mean	/				56.59	4.54	77.40

Table 43: The results of authorization application AoD-IP on Mini-DomainNet.

$\lambda_{1}$	0.01	0.05	0.1	0.2	0.5	1
$X_{a}\rightarrow X_{u}$	84.1 $\rightarrow$ 12.8	83.9 $\rightarrow$ 13.0	84.0 $\rightarrow$ 11.9	82.8 $\rightarrow$ 12.5	83.2 $\rightarrow$ 11.8	93.6 $\rightarrow$ 12.1

Table 44: Ablation study of target-specified AoD-IP ’Cl

\rightarrow

Pa’.

Augmentation	Augmentation
AutoContrast	Applies automatic contrast adjustment to an image.
Brightness	Adjusts the brightness of an image.
Color	Adjusts the color saturation of an image.
Contrast	Adjusts the contrast of an image.
Equalize	Equalizes the histogram of an image.
Identity	Returns the image without any changes.

Table 45: A detailed description of the augmentation method applied in the target-free scenario.

VLM Inference

Vision-language model inference studies how multimodal models turn images and video into answers efficiently, accurately, and reliably.

See all 82 papers

Apr 18, 2026

SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

The public accessibility of large vision-language models (LVLMs) raises serious concerns about unauthorized model reuse and intellectual property infringement. Existing ownership verification methods often rely on semantically abnormal queries or out-of-distribution responses as fingerprints, which can be easily detected and removed by adversaries. We expose this vulnerability through a Semantic Divergence Attack (SDA), which identifies and filters fingerprint queries by measuring semantic divergence between a suspect model and a reference model, showing that existing fingerprints are not semantic-preserving and are therefore easy to detect and bypass. To address these limitations, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework that requires no parameter modification. SIF introduces Semantic-Aligned Fingerprint Distillation (SAFD), which transfers text watermarking signals into the visual modality to produce semantically coherent yet fingerprinted responses. In addition, Robust-Fingerprint Optimization (RFO) enhances robustness by simulating worst-case representation perturbations, making the fingerprints resilient to model modifications such as fine-tuning and quantization. Extensive experiments on LLaVA-1.5 and Qwen2.5-VL demonstrate that SIF achieves strong stealthiness and robustness, providing a practical solution for LVLM copyright protection. Code is available at https://github.com/UCF-ML-Research/SIF-VLM-Fingerprint

Yifei Zhao, Qian Lou, Mengxin Zheng

University of Central Florida

VLM Robustness Language Model Fingerprinting Large Vision-Language Models LLM Watermarking

Ask PDF

Mar 24, 2026

Chain-of-Authorization: Embedding authorization into large language models

Although Large Language Models (LLMs) have evolved from text generators into the cognitive core of modern AI systems, their inherent lack of authorization awareness exposes these systems to catastrophic risks, ranging from unintentional data leakage to unauthorized command execution. Existing defense mechanisms are fundamentally decoupled from internal reasoning, rendering them insufficient for the complex security demands of dynamic AI systems. Here, we propose the Chain-of-Authorization (CoA) framework, a paradigm that internalizes access control as a foundational cognitive capability. By systematically redesigning the input-output format and fine-tuning the model on synthesized data with complex permission topologies, CoA forces the model to generate a structured authorization trajectory as a causal prerequisite for any substantive response or action, thereby enabling LLMs to internalize access boundaries within dynamic reasoning environments. CoA maintains high utility in authorized scenarios while achieving high rejection rates of unauthorized prompts and robust defense against diverse adversarial attacks. By embedding authorization directly into the reasoning process, CoA provides a principled architectural blueprint for deploying secure LLMs as the cognitive cores of modern AI systems.

Yang Li, Yule Liu, Xinlei He +3

Department of Computer Science and Technology, Tsinghua University . · Data Science and Analytic Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou) . · Institude for Math and AI, Wuhan University .

LLM Security LLM Safety Adversarial Robustness of Language Models

Ask PDF

Mar 26, 2026

Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models

On-device Vision-Language Models (VLMs) promise data privacy via local execution. However, we show that the architectural shift toward Dynamic High-Resolution preprocessing (e.g., AnyRes) introduces an inherent algorithmic side-channel. Unlike static models, dynamic preprocessing decomposes images into a variable number of patches based on their aspect ratio, creating workload-dependent inputs. We demonstrate a dual-layer attack framework against local VLMs. In Tier 1, an unprivileged attacker can exploit significant execution-time variations using standard unprivileged OS metrics to reliably fingerprint the input's geometry. In Tier 2, by profiling Last-Level Cache (LLC) contention, the attacker can resolve semantic ambiguity within identical geometries, distinguishing between visually dense (e.g., medical X-rays) and sparse (e.g., text documents) content. By evaluating state-of-the-art models such as LLaVA-NeXT and Qwen2-VL, we show that combining these signals enables reliable inference of privacy-sensitive contexts. Finally, we analyze the security engineering trade-offs of mitigating this vulnerability, reveal substantial performance overhead with constant-work padding, and propose practical design recommendations for secure Edge AI deployments.

Eyal Hadad, Mordechai Guri

Ben-Gurion University of the Negev, Israel Faculty of Computer and Information Science

Privacy Protection in VLMs Adversarial Attacks on VLMs

Ask PDF

Abstract

VLM Inference

Explore similar work

SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

Chain-of-Authorization: Embedding authorization into large language models

Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models