Instance encoding is a popular empirical technique for privacy enhancement when sharing data to an untrusted server. It transforms sensitive data through an encoding process before sharing, with the hope that the encoding process retains utility but makes it hard to reconstruct the original data. However, most work offers no theoretical guarantee that the encoding process is actually irreversible. A recent work derived a mean-squared error (MSE) bound limiting any adversary's reconstruction accuracy, offering one of the first theoretical results in this domain. This bound, however, has three critical limitations: it is often too loose, only works with randomized encoders (excluding many deterministic encoders practitioners use), and only bounds MSE. We introduce a family of new bounds that (1) are tighter, (2) applicable even to fully deterministic encoders, and (3) can extend beyond MSE to other norm-based similarity metrics, by properly accounting for the encoder's spectral structure. We evaluate our bounds across a range of encoders, datasets, and attacks, showing they hold consistently and improve upon the existing bound.
Figures & tables
Figure 1: The idea of instance encoding and its invertibility.
Figure 2: Our MSE bounds (solid blue line) closely lower-bound the attackers’ achievable MSE for all the attackers. Bound from Maeng et al. [ 35 ] (dotted line) becomes very loose when r<d and is always zero when σ=0 (no encoder randomness).
Figure 3: Our LPIPS bounds (solid blue line) are always met, and tight near m∼d . For m≪d , the bound is unfortunately looser and near-flat, being less informative than the MSE bounds. Still, this is the first working LPIPS bound to our knowledge.
Figure 4: MSE and LPIPS bound with different skewness α .
Figure 5: MSE bound vs. achieved MSE by the attacker.
Figure 6: LPIPS bound vs. achieved LPIPS by the attacker.
Figure 7: S quality check: all the eigenvalues μi of Σ1/2SΣ1/2 must be equal to or larger than 1.
Figure 8: Bound vs. attack MSE/LPIPS with while varying the number of data k used to estimate E[G] . Result from the CIFAR-10 dataset; MNIST dataset shows similar trends and is omitted for space.
Figure 9: LPIPS bounds with and without noise, under various threshold τ that splits between the range space and the null space (Section 6.4.3 ). Empirically, τ=10−4 seems to be a reasonable choice, where the bounds appear the tightest and never violated.
Figure 10: Visualization of the reconstructed images from Figures 2 (b–c) ( σ=10−3 ), along with each encoder’s RTC.
Figure 11: Visualization of the reconstructed images from Figures 2 (e–f) ( σ=0 ), along with each encoder’s RTC.
Figure 12: Same reference images from Figures 10 and 11 , compared with their class-mean images.