Organizations: The University of Queensland, Brisbane, Australia · University of Wollongong, Wollongong, Australia · City University of Hong Kong, Hong Kong SAR, China
As deep learning inference is increasingly deployed in shared and cloud-based settings, a growing concern is input repurposing, in which data submitted for one task is reused by unauthorized models for another. Existing privacy defenses largely focus on restricting data access, but provide limited control over what downstream uses a released representation can still support. We propose a feature extraction framework that suppresses cross-model transfer while preserving accuracy for a designated classifier. The framework employs a variational latent bottleneck, trained with a task-driven cross-entropy objective and KL regularization, but without any pixel-level reconstruction loss, to encode inputs into a compact latent space. A dynamic binary mask, computed from per-dimension KL divergence and gradient-based saliency with respect to the frozen target model, suppresses latent dimensions that are uninformative for the intended task. Because saliency computation requires gradient access, the encoder is trained in a white-box setting, whereas inference requires only a forward pass through the frozen target model. On CIFAR-100, the processed representations retain strong utility for the designated classifier while reducing the accuracy of all unintended classifiers to below 2%, yielding a suppression ratio exceeding 45 times relative to unintended models. Preliminary experiments on CIFAR-10, Tiny ImageNet, and Pascal VOC provide exploratory evidence that the approach extends across task settings, although further evaluation is needed to assess robustness against adaptive adversaries.
Figures & tables
Figure 1: Overview of the proposed selective-utility inference framework. An input image is encoded into a variational latent representation, filtered by a dynamic binary mask, decoded into a transformed image, and evaluated by a frozen target classifier. During training, gradients from the target-model loss update the encoder and decoder; at inference time, only a forward pass is required.
Figure 2: Our feature filtering pipeline. The encoder produces latent features together with μ and logσ2 . Each latent dimension is scored using KL divergence and gradient-based saliency, the scores are normalized and combined into a unified importance measure, and thresholding yields a binary mask that selects the dimensions retained for decoding.
Figure 3: Information Bottleneck view of the proposed variational latent bottleneck. The objective encourages the representation Z to retain information useful for predicting Y while suppressing information about the input X , with λ controlling the compression-utility trade-off.
Figure 4: The proposed latent masking mechanism. Per-dimension KL divergence and task-loss saliency are combined to estimate latent importance, and a thresholded binary mask suppresses dimensions that contribute little to the intended task.
Model
Accuracy (%)
VGG16
76.62
ResNet152
85.46
DenseNet121
83.61
ConvNeXt-V2
88.34
Table 1: Baseline top-1 accuracy (%) of the four pre-trained classifiers on original CIFAR-100 test images before any transformation is applied.
VGG16
ResNet152
DenseNet121
ConvNeXt-V2
Target Accuracy
67.77
67.44
68.57
70.60
Mean Other Models
1.14
1.23
1.21
1.08
Table 2: Top-1 accuracy (%) on transformed CIFAR-100 test images produced without latent masking. Each column corresponds to a different target model used to supervise the encoder–decoder during training.
λ
ResNet152 (Target)
DenseNet121
ConvNeXt-V2
0.01
68.60
1.00
0.70
0.005
65.35
0.40
1.47
Table 3: Top-1 accuracy (%) on transformed CIFAR-100 test images under KL-only dynamic masking with ResNet152 as the target model. Lower off-target accuracy indicates stronger suppression of cross-model transfer.
Configuration
Target Acc. (%)
Mean Unintended (%)
No masking
67.44
1.23
KL-only masking
68.60
0.85
Integrated masking
70.12
1.01
Table 4: Ablation summary on CIFAR-100 with ResNet152 as target. Target accuracy improves across stages while mean unintended accuracy remains near chance (1%).
Target Model
ResNet152
DenseNet121
ConvNeXt-V2
VGG16
ResNet152
70.12
1.18
0.93
0.93
DenseNet121
1.00
68.57
1.57
1.16
ConvNeXt-V2
1.02
1.00
72.23
0.71
VGG16
1.01
0.78
0.94
67.90
Table 5: Top-1 accuracy (%) on transformed CIFAR-100 test images under integrated dynamic masking. Each row denotes the target model used during training, diagonal entries report target-model accuracy, and off-diagonal entries report the accuracy of unintended classifiers applied to the same transformed inputs.
Figure 5: Classification accuracy (%) on CIFAR-100 under integrated dynamic masking. Diagonal entries show performance of the designated target model on transformed inputs, while off-diagonal entries show the accuracy of unintended classifiers applied to the same inputs. Strong diagonal contrast indicates selective utility and reduced cross-model transfer.
Figure 6: Qualitative cross-task transfer example. From left to right: original input, style reference, stylization of the original image, and stylization of the transformed image produced by the proposed framework. The degraded stylization result on the transformed image suggests reduced retention of features useful for this unintended downstream task.
Institute of Information Engineering, Chinese Academy of Sciences, China, and the School of Cyber Security, University of Chinese Academy of Sciences, China
Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon, Republic of Korea