Organizations: The University of Queensland, Brisbane, Australia · University of Wollongong, Wollongong, Australia · City University of Hong Kong, Hong Kong SAR, China
As deep learning inference is increasingly deployed in shared and cloud-based settings, a growing concern is input repurposing, in which data submitted for one task is reused by unauthorized models for another. Existing privacy defenses largely focus on restricting data access, but provide limited control over what downstream uses a released representation can still support. We propose a feature extraction framework that suppresses cross-model transfer while preserving accuracy for a designated classifier. The framework employs a variational latent bottleneck, trained with a task-driven cross-entropy objective and KL regularization, but without any pixel-level reconstruction loss, to encode inputs into a compact latent space. A dynamic binary mask, computed from per-dimension KL divergence and gradient-based saliency with respect to the frozen target model, suppresses latent dimensions that are uninformative for the intended task. Because saliency computation requires gradient access, the encoder is trained in a white-box setting, whereas inference requires only a forward pass through the frozen target model. On CIFAR-100, the processed representations retain strong utility for the designated classifier while reducing the accuracy of all unintended classifiers to below 2%, yielding a suppression ratio exceeding 45 times relative to unintended models. Preliminary experiments on CIFAR-10, Tiny ImageNet, and Pascal VOC provide exploratory evidence that the approach extends across task settings, although further evaluation is needed to assess robustness against adaptive adversaries.
Figures & tables
Figure 1: Overview of the proposed selective-utility inference framework. An input image is encoded into a variational latent representation, filtered by a dynamic binary mask, decoded into a transformed image, and evaluated by a frozen target classifier. During training, gradients from the target-model loss update the encoder and decoder; at inference time, only a forward pass is required.
Figure 2: Our feature filtering pipeline. The encoder produces latent features together with μ and logσ2 . Each latent dimension is scored using KL divergence and gradient-based saliency, the scores are normalized and combined into a unified importance measure, and thresholding yields a binary mask that selects the dimensions retained for decoding.
Figure 3: Information Bottleneck view of the proposed variational latent bottleneck. The objective encourages the representation Z to retain information useful for predicting Y while suppressing information about the input X , with λ controlling the compression-utility trade-off.
Figure 4: The proposed latent masking mechanism. Per-dimension KL divergence and task-loss saliency are combined to estimate latent importance, and a thresholded binary mask suppresses dimensions that contribute little to the intended task.
Model
Accuracy (%)
VGG16
76.62
ResNet152
85.46
DenseNet121
83.61
ConvNeXt-V2
88.34
Table 1: Baseline top-1 accuracy (%) of the four pre-trained classifiers on original CIFAR-100 test images before any transformation is applied.
VGG16
ResNet152
DenseNet121
ConvNeXt-V2
Target Accuracy
67.77
67.44
68.57
70.60
Mean Other Models
1.14
1.23
1.21
1.08
Table 2: Top-1 accuracy (%) on transformed CIFAR-100 test images produced without latent masking. Each column corresponds to a different target model used to supervise the encoder–decoder during training.
λ
ResNet152 (Target)
DenseNet121
ConvNeXt-V2
0.01
68.60
1.00
0.70
0.005
65.35
0.40
1.47
Table 3: Top-1 accuracy (%) on transformed CIFAR-100 test images under KL-only dynamic masking with ResNet152 as the target model. Lower off-target accuracy indicates stronger suppression of cross-model transfer.
Configuration
Target Acc. (%)
Mean Unintended (%)
No masking
67.44
1.23
KL-only masking
68.60
0.85
Integrated masking
70.12
1.01
Table 4: Ablation summary on CIFAR-100 with ResNet152 as target. Target accuracy improves across stages while mean unintended accuracy remains near chance (1%).
Target Model
ResNet152
DenseNet121
ConvNeXt-V2
VGG16
ResNet152
70.12
1.18
0.93
0.93
DenseNet121
1.00
68.57
1.57
1.16
ConvNeXt-V2
1.02
1.00
72.23
0.71
VGG16
1.01
0.78
0.94
67.90
Table 5: Top-1 accuracy (%) on transformed CIFAR-100 test images under integrated dynamic masking. Each row denotes the target model used during training, diagonal entries report target-model accuracy, and off-diagonal entries report the accuracy of unintended classifiers applied to the same transformed inputs.
Figure 5: Classification accuracy (%) on CIFAR-100 under integrated dynamic masking. Diagonal entries show performance of the designated target model on transformed inputs, while off-diagonal entries show the accuracy of unintended classifiers applied to the same inputs. Strong diagonal contrast indicates selective utility and reduced cross-model transfer.
Figure 6: Qualitative cross-task transfer example. From left to right: original input, style reference, stylization of the original image, and stylization of the transformed image produced by the proposed framework. The degraded stylization result on the transformed image suggests reduced retention of features useful for this unintended downstream task.
Private inference protects both user inputs and server models during neural network inference, but existing solutions remain too slow for practical deployment. This motivates recent efforts to run a public encoder, such as a pretrained backbone, outside the protection boundary and evaluate only a small private predictor cryptographically. While appealing for efficiency, this design is not inherently secure: naively offloading a public encoder may create a feature-space shortcut: an extraction adversary may learn the remaining private predictor's feature-to-output mapping more easily than the original model's input-to-output behavior. We present Gecko, designed to limit this additional risk while retaining a compact encrypted predictor. We leverage a frozen backbone that contributes hierarchical features, fixed Fastfood projections that compress them, and private feature gating that prepares them for prediction. We formalize ideal independence and information-preservation conditions as design guidance, then separately evaluate component-reuse extraction attacks. Across image and audio tasks, Gecko achieves 0.4-2.2 second inference with at most 10.8 MB communication and accuracy comparable to transfer-learning baselines. Under the evaluated attacks, reusing the offloaded public encoder provides no significant advantage to model-extraction adversaries. Source code and a demo are available at https://github.com/CassiniHuy/gecko-infer.
Cheng'an Wei, Kai Chen, Yue Zhao +2
Institute of Information Engineering, Chinese Academy of Sciences, China, and the School of Cyber Security, University of Chinese Academy of Sciences, China
Image compression is a ubiquitous component of modern visual pipelines, routinely applied by social media platforms and resource-constrained systems prior to inference. Despite its prevalence, the impact of compression on adversarial robustness remains poorly understood. We study a previously unexplored adversarial setting in which attacks are applied directly in compressed representations, and show that compression can act as an adversarial amplifier for deep image classifiers. Under identical nominal perturbation budgets, compression-aware attacks are substantially more effective than their pixel-space counterparts. We attribute this effect to decision space reduction, whereby compression induces a non-invertible, information-losing transformation that contracts classification margins and increases sensitivity to perturbations. Extensive experiments across standard benchmarks and architectures support our analysis and reveal a critical vulnerability in compression-in-the-loop deployment settings. Code will be released.
Lewis Evans, Harkrishan Jandu, Zihan Ye +2
University of Nottingham, Nottingham, United Kingdom · University of Chinese Academy of Sciences, Beijing, China · Xiamen University, Xiamen, China +1
As pretrained models are increasingly shared on the web, ensuring that models can forget or delete sensitive, copyrighted, or private information upon request has become crucial. Machine unlearning has been proposed to address this challenge. However, current evaluations for unlearning methods rely on output-based metrics, which cannot verify whether information is completely deleted or merely suppressed at the representation level, where suppression is insufficient for true unlearning. To address this gap, we propose a novel restoration-based analysis framework that uses Sparse Autoencoders to identify class-specific expert features in intermediate layers and applies inference-time steering to quantitatively distinguish between suppression and deletion. Applying our framework to 12 major unlearning methods in image classification tasks, we find that most methods achieve high restoration rates of unlearned information, indicating that they only suppress information at the decision-boundary level, while preserving semantic features in intermediate representations. Notably, even retraining from pretrained checkpoints shows high restoration, revealing that robust semantic features inherited from pretraining are not removed by retraining. These results demonstrate that representation-level retention poses significant risks overlooked by output-based metrics, highlighting the need for new unlearning evaluation criteria. We propose new evaluation guidelines that prioritize representation-level verification, especially for privacy-critical applications in the era of pre-trained models.
Yurim Jang, Jaeung Lee, Dohyun Kim +2
Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon, Republic of Korea