Differentially Private Datastore Generation for Retrieval-Augmented Inference

Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD). However, DPSGD suffers from substantial utility degradation and slow convergence due to gradient clipping and noise injection. Prior works have attempted to improve DPSGD from various perspectives; notably, the Differentially Private Selective Update and Release (DPSUR) algorithm has achieved remarkable model utility. However, the privacy accounting in DPSUR overlooks the variation in sampling probability introduced by the selective release mechanism, which compromises the rigor of its privacy guarantees. To address these limitations, we re-evaluate the privacy analysis of the selective release mechanism and propose a novel algorithm: Differentially Private Selective Release based on Clipped Gradients (DPSR-CG). Through a rigorous, newly derived privacy analysis and extensive experiments on multiple datasets (MNIST, CIFAR-10, IMDB, and FMNIST), we demonstrate that our DPSR-CG mechanism maintains strict privacy guarantees while achieving exceptional model performance.

The ever-increasing collection of personal data has created mounting pressure to develop technologies that protect sensitive aspects of individual identity. Differential privacy (DP) provides a principled framework with strong formal guarantees and has already achieved practical success. However, releasing high-dimensional data, such as images, has remained elusive: releasing uncompressed privatized data requires significant storage. At the same time, no effective data compression scheme exists that can compress high-resolution data with privacy guarantees. We address this challenge with DP-DiPP, a compression pipeline that combines stochastic codes with diffusion models. DP-DiPP is highly flexible: the practitioner has direct control over the compression rate-privacy-utility tradeoff. As the theoretical backbone, we extend the Poisson private representation (PPR) to encode the outputs of privacy mechanisms. We then combine it with DiffC, a diffusion-based lossy data compression method, to obtain a differentially private image compressor. Our experiments on privatized image classification on CIFAR-10 demonstrate that DP-DiPP significantly outperforms the baseline, achieving a 10-30 times better compression while retaining comparable privacy guarantees and utility.

This paper investigates the problem of generating synthetic tabular data with differential privacy (DP) guarantees, enabling data sharing in sensitive domains. Despite extensive study, state-of-the-art methods often focus on minimizing low-order marginal query errors and overlook the challenges posed by high-order correlations. To address this gap, we extend the Private Evolution (PE) framework, originally developed for DP-compliant image and text synthesis, to tabular data. We introduce Tab-PE -- an algorithm for synthetic tabular data generation under DP constraints. Tab-PE iteratively improves a candidate dataset via an evolutionary process that leverages tabular-specialized operators to produce variations, privately scores them, and selects the highest-quality samples to retain and propagate. In contrast to the original PE, which relies on large foundation models, Tab-PE employs heuristic operators with significantly lower computational costs, making PE more practical and scalable for tabular data. Through extensive experiments on real-world and simulation datasets, we demonstrate that Tab-PE substantially outperforms prior baselines on datasets exhibiting high-order correlations. Compared to the best baseline -- AIM, Tab-PE improves classification accuracy by up to 10% while running 28 times faster.