A Privacy Study of Sparse Collaborative Inference

Maximilian Andreas Hoefler; Karsten Mueller; Wojciech Samek

cs.LGAug 17, 2026

A Privacy Study of Sparse Collaborative Inference

Authors: Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek

Organizations: Fraunhofer Heinrich Hertz Institute, Germany · Technical University of Berlin, Germany · Berlin Institute for the Foundations of Learning and Data (BIFOLD)

Abstract

Collaborative inference (CI) splits a model between an edge device and a server, whereby the client computes an intermediate activation, transmits it, and the server completes the computation. This raises two concerns, the communication cost of the transmission and the risk that it reveals private information about the input. Recent work reduces this cost by sparsifying activations and entropy-coding the result. Sparsity has also been argued to improve privacy, on the intuition that transmitting fewer values reveals less about the input. We test this claim by decomposing the sparse activation into the retained values and the set of positions they occupy, and by reconstructing inputs from each component in isolation. We find that sparsification reduces the leakage far less than it reduces the transmission cost, and that the remaining risk shifts to the positions, which prior analyses treat as side information for decoding. Across natural-image and face datasets, the positions alone constitute a serious privacy risk, enabling high-fidelity reconstructions and re-identification of individuals. The leakage from the positions persists even when both the transmission cost and the task utility are low. We conclude that the positions of sparse activations should be treated as sensitive transmitted data and audited carefully in the context of collaborative inference. Code is available at https://github.com/an7123/Privacy-Study-Sparse-CI.

Figures & tables

Figure 1: Decomposition of a sparse activation. Top-

k

selection splits the activation

z

into the transmitted positions, a binary mask over the

k

largest-magnitude coordinates, and the transmitted values at those coordinates. The two components are normally transmitted together, but they can be inverted separately, which is the basis of our probing framework.

positions $\downarrow$ / values $\rightarrow$	No values (mask, $b{=}0$ )	Real values (precision $b$ )
All positions (no sparsity)	–	Dense ( $b{=}16$ , rate $16$ ) ; $b$ -bit quant. ( $b{\in}\{1,2,4,8\}$ , rate $b$ )
True top- $k$ (input-dep.)	Positions only (rate $H_{2}(\rho)$ )	Top- $k$ activation (FP16, rate $H_{2}(\rho){+}16\rho$ )
Block-shuffled ( $4{\times}4$ )	Block-shuffled positions	–
Random (input-indep.)	Random positions	Values on random positions

Table 1: Transmitted representations as a two-factor design (source of the positions

\times

whether values are sent, at precision

b

); rates in bits/dim from Eq. 3 . “–” cells are unused (see Sec. 3.2 ).

Representation	TinyImageNet			Imagenette
Representation	Acc.	SSIM (WB)	SSIM (Aux)	Acc.	SSIM (WB)	SSIM (Aux)
Dense activations	$53.1{\pm}0.8$	$0.436{\pm}0.001$	$0.559{\pm}0.002$	$88.9{\pm}0.4$	$0.496{\pm}0.003$	$0.561{\pm}0.003$
Top- $k$ activation	$52.2{\pm}0.6$	$0.213{\pm}0.008$	$0.435{\pm}0.005$	$86.0{\pm}0.6$	$0.127{\pm}0.008$	$0.420{\pm}0.005$
Positions only	$49.8{\pm}0.5$	$0.059{\pm}0.006$	$0.428{\pm}0.006$	$82.3{\pm}0.3$	$0.013{\pm}0.009$	$0.400{\pm}0.005$
Values on rand. positions	$16.2{\pm}0.4$	$0.038{\pm}0.003$	$0.145{\pm}0.003$	$13.0{\pm}0.4$	$0.041{\pm}0.004$	$0.216{\pm}0.007$
Block-shuffled positions	$18.6{\pm}0.9$	$0.061{\pm}0.003$	$0.295{\pm}0.004$	$64.7{\pm}0.4$	$0.052{\pm}0.003$	$0.300{\pm}0.004$

Table 2: Reconstruction leakage at 95% sparsity on TinyImageNet and Imagenette. For each dataset we report downstream accuracy (Acc., attack-independent) and reconstruction SSIM under the WB audit and the Aux inverse ( Sec. 3.3 ).

Figure 2: Trade-off between rate, utility, and leakage on TinyImageNet.

Representation	Acc.	SSIM		ID cos. (Aux)	R@1/R@5 (Aux)
Representation	Acc.	WB	Aux	ID cos. (Aux)	R@1/R@5 (Aux)
Dense activations	$75.0{\pm}0.8$	$0.516{\pm}0.003$	$0.840{\pm}0.001$	$0.623{\pm}0.004$	$38.4{\pm}0.5$ / $63.5{\pm}0.6$
Top- $k$ activation	$74.0{\pm}0.6$	$0.436{\pm}0.003$	$0.754{\pm}0.008$	$0.514{\pm}0.009$	$24.9{\pm}0.6$ / $51.3{\pm}0.6$
Positions only	$63.0{\pm}0.6$	$0.076{\pm}0.007$	$0.751{\pm}0.002$	$0.510{\pm}0.002$	$24.5{\pm}0.9$ / $49.5{\pm}0.7$
Values on rand. positions	$20.0{\pm}0.7$	$0.154{\pm}0.004$	$0.290{\pm}0.003$	$0.039{\pm}0.008$	$1.1{\pm}0.8$ / $11.6{\pm}0.9$
Block-shuffled positions	$50.8{\pm}0.6$	$0.161{\pm}0.004$	$0.501{\pm}0.002$	$0.021{\pm}0.003$	$1.1{\pm}0.8$ / $12.0{\pm}0.4$

Table 3: FaceScrub leakage at 95% sparsity. Acc. is downstream accuracy, SSIM is reconstruction fidelity under the WB audit and the Aux inverse ( Sec. 3.3 ); ID cos. is FaceNet embedding cosine and R@1/R@5 are rank-1/rank-5 re-identification against an held-out evaluation gallery (one held-out image per identity), both under the Aux inverse.

Figure 3: Reconstructions under the Aux inverse for TinyImageNet (left four columns) and FaceScrub (right four). Rows, top to bottom: original, dense activation, the top- $k$ activation (the retained values at their true positions, labeled “values only” in the figure), positions only, block-shuffled positions, dense $4$ -bit, dense $2$ -bit.

Panel of Figure 4 — Figure 4: Robustness of the values-versus-positions result (TinyImageNet, Aux inverse): (a) across split layers and (b) across backbones.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Probe	Transmitted object	Rate (bits/dim)
Dense	$z$	$b=16$
Top- $k$ activation	$z\odot m$	$H_{2}(\rho)+\rho b$
Positions only	$m\in\{0,1\}^{d}$	$H_{2}(\rho)$
Values on random pos.	values scattered to fresh random positions	$H_{2}(\rho)+\rho b$
Dense $b$ -bit quant.	symmetric $b$ -bit quantization of $z$	$b$
Static mask [ 12 ]	$z\odot m_{\text{fixed}}$	$\rho b$

Appendix

Table A1 : Transmitted object and analytical rate for each probe. The binary mask is fed to the server as literal

0/1

floats occupying the activation tensor, without magnitude rescaling.

Inverse	Params
Shared generative trunk	1.78 M
ConvDecoder, spatial input ( $C{=}128$ )	2.07 M
ConvDecoder, occupancy map ( $C{=}1$ )	1.78 M
ConvDecoder, per-channel counts (vector, $C{=}128$ )	3.89 M
Values MLP-on-sorted ( Sec. 0.A.6 )	40.4 M
Values DeepSets ( Sec. 0.A.6 )	19.4 M

Appendix

Table A2 : Parameter counts, so that attacker capacity is explicit and comparable across input layouts. The generative trunk is identical in every inverse; only the input head changes.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 433 papers

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs
Sep 30, 2026Fahao Chen, Linkang Du, Jinhao Zhou +2LLM Inference Privacy Leakage in Language Models
Shandong University China · Xi’an Jiaotong University China · Waseda University Japan
Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding
Aug 29, 2026Kejia Zhang, Tianyuan Zou, Zixuan GU +1Privacy Leakage in Language Models Privacy-Preserving Language Models
The Hong Kong Polytechnic University · Institute for AI Industry Research, Tsinghua University · School of Software, Tsinghua University
What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference
May 22, 2026Mingyuan Fan, Yu Liu, Fuyi Wang +1Privacy-Preserving Language Models Adversarial Attacks
East China Normal University Shanghai, China · RMIT University Melbourne, Australia

Sep 30, 2026J/K move · Enter open · S save

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs

Fahao Chen, Linkang Du, Jinhao Zhou, Peng Li+1

Shandong University China · Xi’an Jiaotong University China · Waseda University Japan

LLM Inference Privacy Leakage in Language Models LLM Security Sparse Attention

Open paper AskPDF arXiv

Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior introduces previously unexplored privacy risks. We identify a new GPU micro-architectural side channel, termed Sparsity-Induced Memory Access (SIMA), which arises from secret-dependent key-value cache access patterns induced by sparse attention. Based on this observation, we present SparLeak, a phase-aware side-channel attack that extracts SIMA traces during LLM inference and enables two practical privacy extractions: query attribute inference from prefill-phase traces and autoregressive response reconstruction from decoding-phase traces. By reconstructing approximate token-level sparsity profiles from page-level observations and applying profiling-based learning, SparLeak accurately recovers sensitive information, including user-query attributes and private LLM response content. Extensive evaluation across three LLM architectures, three sparse attention mechanisms, and three privacy-sensitive datasets shows that SparLeak achieves average attack success rates of 90.9% for attribute inference and 87.3% for response reconstruction under real-world LLM serving settings, highlighting the significance to account for SIMA leakage when deploying sparse-attention-based LLM systems. We provide anonymized SIMA traces, trained attack models, evaluation scripts, and documentation as artifacts at https://anonymous.4open.science/r/Janus_artifacts/.

Abstract

Privacy-Preserving ML

Explore similar work

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs

Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding

What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference