Differential Privacy

Follow this topic Ask about this topic

Differential privacy studies algorithms whose outputs reveal limited information about any individual record, even when an attacker knows other data. Common approaches add calibrated noise to statistics or clipped gradients, use randomized response for local privacy, or combine privacy with subsampling, shuffling, and secure aggregation. These methods support private statistical estimation, machine learning, federated learning, and synthetic data generation, including for images, tables, and text; evaluation often measures both task utility and membership-inference risk. A central tension is that stronger guarantees can reduce accuracy or increase computational cost, while task-specific structure and tighter privacy accounting may improve the trade-off without resolving all deployment risks.

Latest papers

285 papers43 papers in the last four weeks, up 54% on the four weeks before. 0.2% of all new papers.

Mar 8, 2026

Beyond Data Splitting: Full-Data Conformal Prediction by Differential Privacy

Privacy protection and uncertainty quantification are increasingly important in data-driven decision making. Conformal prediction provides finite-sample marginal coverage, but existing private approaches often rely on data splitting, reducing the effective sample size. We propose a full-data privacy-preserving conformal prediction framework that avoids splitting. Our framework leverages stability induced by differential privacy to control the gap between in-sample and out-of-sample conformal scores, and pairs this with a conservative private quantile routine designed to prevent under-coverage. We show that a generic differential privacy guarantee yields a universal coverage floor, yet cannot generally recover the nominal

1-α

level. We then provide a refined, mechanism-specific stability analysis and yields asymptotic recovery of the nominal level. Experiments demonstrate sharper prediction sets than the split-based private baseline.

Young Hyun Cho, Jordan Awan

Purdue University · University of Pittsburgh

Conformal Prediction Differential Privacy

Mar 5, 2026

Differentially Private Multimodal In-Context Learning

Vision-language models are increasingly applied to sensitive domains such as medical imaging and personal photographs, yet existing differentially private methods for in-context learning are limited to few-shot, text-only settings because privacy cost scales with the number of tokens processed. We present Differentially Private Multimodal Task Vectors (DP-MTV), the first framework enabling many-shot multimodal in-context learning with formal

(\varepsilon, δ)

-differential privacy by aggregating hundreds of demonstrations into compact task vectors in activation space. DP-MTV partitions private data into disjoint chunks, applies per-layer clipping to bound sensitivity, and adds calibrated noise to the aggregate, requiring only a single noise addition that enables unlimited inference queries. We evaluate on eight benchmarks across three VLM architectures, supporting deployment with or without auxiliary data. At

\varepsilon=1.0

, DP-MTV achieves 50% on VizWiz compared to 55% non-private and 35% zero-shot, preserving most of the gain from in-context learning under meaningful privacy constraints.

Ivoline C. Ngong, Zarreen Reza, Joseph P. Near

University of Vermont, Burlington, VT, USA · Independent Researcher

Privacy Protection in VLMs Multimodal ICL Privacy-Preserving ML Differential Privacy Vision-Language Models

Mar 4, 2026

LDP-Slicing: Local Differential Privacy for Images via Randomized Bit-Plane Slicing

Local Differential Privacy (LDP) is the gold standard trust model for privacy-preserving machine learning by guaranteeing privacy at the data source. However, its application to image data has long been considered impractical due to the high dimensionality of pixel space. Canonical LDP mechanisms are designed for low-dimensional data, resulting in severe utility degradation when applied to high-dimensional pixel spaces. This paper demonstrates that this utility loss is not inherent to LDP, but from its application to an inappropriate data representation. We introduce LDP-Slicing, a lightweight, training-free framework that resolves this domain mismatch. Our key insight is to decompose pixel values into a sequence of binary bit-planes. This transformation allows us to apply the LDP mechanism directly to the bit-level representation. To further strengthen privacy and preserve utility, we integrate a perceptual obfuscation module that mitigates human-perceivable leakage and an optimization-based privacy budget allocation strategy. This pipeline satisfies rigorous pixel-level

\varepsilon

-LDP while producing images that retain high utility for downstream tasks. Extensive experiments on face recognition and image classification demonstrate that LDP-Slicing outperforms existing DP/LDP baselines under comparable privacy budgets, with negligible computational overhead.

Yuanming Cao, Chengqi Li, Wenbo He

McMaster University, Hamilton, Ontario, Canada

Privacy-Preserving ML Differential Privacy

Mar 3, 2026

Adaptive Methods Are Preferable in High Privacy Settings: An SDE Perspective

Differential Privacy (DP) is becoming central to large-scale training as privacy regulations tighten. We revisit how DP noise interacts with adaptivity in optimization through the lens of stochastic differential equations, providing the first SDE-based analysis of private optimizers. Focusing on DP-SGD and DP-SignSGD under per-example clipping, we show a sharp contrast under fixed hyperparameters: DP-SGD converges at a Privacy-Utility Trade-Off of

\mathcal{O}(1/\varepsilon^2)

with speed independent of

\varepsilon

, while DP-SignSGD converges at a speed linear in

\varepsilon

with an

\mathcal{O}(1/\varepsilon)

trade-off, dominating in high-privacy or large batch noise regimes. By contrast, under optimal learning rates, both methods achieve comparable theoretical asymptotic performance; however, the optimal learning rate of DP-SGD scales linearly with

\varepsilon

, while that of DP-SignSGD is essentially

\varepsilon

-independent. This makes adaptive methods far more practical, as their hyperparameters transfer across privacy levels with little or no re-tuning. Empirical results confirm our theory across training and test metrics, and empirically extend from DP-SignSGD to DP-Adam.

Enea Monzio Compagnoni, Alessandro Stanghellini, Rustem Islamov +2

University of Basel, Switzerland · University of Zürich, Switzerland

Differentially Private Stochastic Gradient Descent Differential Privacy Adaptive Gradient Methods

Mar 3, 2026

PrivMedChat: End-to-End Differentially Private RLHF for Medical Dialogue Systems

Large language models are increasingly used for patient-facing medical assistance and clinical decision support, but adapting them to clinical dialogue often requires supervision derived from doctor-patient conversations that may contain sensitive information. Conventional supervised fine-tuning and reinforcement learning from human feedback (RLHF) can amplify memorization, enabling membership inference and disclosure of rare training-set details. We present PrivMedChat (Private Medical Chat), an end-to-end framework for differentially private RLHF (DP-RLHF) for medical dialogue systems. Our approach enforces differential privacy at each training stage that accesses dialogue-derived supervision, combining DP-SGD for supervised fine-tuning and reward model learning from preference pairs, and DP-aware policy optimization for alignment. To avoid costly clinician labeling, we introduce an annotation-free preference construction strategy that pairs physician responses with filtered non-expert generations. We evaluate PrivMedChat across medical dialogue tasks and assess utility, safety, and privacy under consistent privacy accounting, thereby providing a practical pathway to align medical chatbots while offering formal privacy guarantees. We open-source our code at https://github.com/sudip-bhujel/privmedchat.

Sudip Bhujel

Department of Computer Science, University of Kentucky, Lexington, KY, USA

Privacy-Preserving Language Models RL from Human Feedback Differential Privacy

Mar 2, 2026

Noise-Calibrated Inference from Differentially Private Sufficient Statistics in Exponential Families

Many differentially private (DP) data release systems either output DP synthetic data and leave analysts to perform inference as usual, which can lead to severe miscalibration, or output a DP point estimate without a principled way to do uncertainty quantification. This paper develops a clean and tractable middle ground for exponential families: release only DP sufficient statistics, then perform noise-calibrated likelihood-based inference and optional parametric synthetic data generation as post-processing. Our contributions are: (1) a general recipe for approximate-DP release of clipped sufficient statistics under the Gaussian mechanism; (2) asymptotic normality, explicit variance inflation, and valid Wald-style confidence intervals for the plug-in DP MLE; (3) a noise-aware likelihood correction that is first-order equivalent to the plug-in but supports bootstrap-based intervals; and (4) a matching minimax lower bound showing the privacy distortion rate is unavoidable. The resulting theory yields concrete design rules and a practical pipeline for releasing DP synthetic data with principled uncertainty quantification, validated on three exponential families and real census data.

Amir Asiaee, Samhita Pal

Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA

Exponential Family Differential Privacy Confidence Region Estimation

Mar 2, 2026

Accurate, private, secure, federated U-statistics with higher degree

We study the problem of computing a U-statistic with a kernel function f of degree k

\ge

2, i.e., the average of some function f over all k-tuples of instances, in a federated learning setting. Ustatistics of degree 2 include several useful statistics such as Kendall's

τ

coefficient, the Area under the Receiver-Operator Curve and the Gini mean difference. Existing methods provide solutions only under the lower-utility local differential privacy model and/or scale poorly in the size of the domain discretization. In this work, we propose a protocol that securely computes U-statistics of degree k

\ge

2 under central differential privacy by leveraging Multi Party Computation (MPC). Our method substantially improves accuracy when compared to prior solutions. We provide a detailed theoretical analysis of its accuracy, communication and computational properties. We evaluate its performance empirically, obtaining favorable results, e.g., for Kendall's

τ

coefficient, our approach reduces the Mean Squared Error by up to four orders of magnitude over existing baselines.

Quentin Sinh, Jan Ramon

MAGNET · MAGNET, Univ. Lille, INRIA, CNRS, UMR 9189 - CRIStAL, F-59000 Lille, France.

U-Statistics Differentially Private Federated Learning Differential Privacy

Mar 1, 2026

Differential privacy representation geometry for medical image analysis

Differential privacy (DP)'s effect in medical imaging is typically evaluated only through end-to-end performance, leaving the mechanism of privacy-induced utility loss unclear. We introduce Differential Privacy Representation Geometry for Medical Imaging (DP-RGMI), a framework that interprets DP as a structured transformation of representation space and decomposes performance degradation into encoder geometry and task-head utilization. Geometry is quantified by representation displacement from initialization and spectral effective dimension, while utilization is measured as the gap between linear-probe and end-to-end utility. Across over 594,000 images from four chest X-ray datasets and multiple pretrained initializations, we show that DP is consistently associated with a utilization gap even when linear separability is largely preserved. At the same time, displacement and spectral dimension exhibit non-monotonic, initialization- and dataset-dependent reshaping, indicating that DP alters representation anisotropy rather than uniformly collapsing features. Correlation analysis reveals that the association between end-to-end performance and utilization is robust across datasets but can vary by initialization, while geometric quantities capture additional prior- and dataset-conditioned variation. These findings position DP-RGMI as a reproducible framework for diagnosing privacy-induced failure modes and informing privacy model selection.

Soroosh Tayebi Arasteh, Marziyeh Mohammadi, Sven Nebelung +1

Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany · INDA Institute, RWTH Aachen University, Aachen, Germany

Neural Representation Geometry Healthcare Privacy-Preserving ML Differential Privacy Medical Image Representation Learning

Feb 27, 2026

Challenges in Enabling Private Data Valuation

Data valuation methods quantify how individual training examples contribute to a model's behavior, and are increasingly used for dataset curation, auditing, and emerging data markets. As these techniques become operational, they raise serious privacy concerns: valuation scores can reveal whether a person's data was included in training, whether it was unusually influential, or what sensitive patterns exist in proprietary datasets. This motivates the study of privacy-preserving data valuation. However, privacy is fundamentally in tension with valuation utility under differential privacy (DP). DP requires outputs to be insensitive to any single record, while valuation methods are explicitly designed to measure per-record influence. As a result, naive privatization often destroys the fine-grained distinctions needed to rank or attribute value, particularly in heterogeneous datasets where rare examples exert outsized effects. In this work, we analyze the feasibility of DP-compatible data valuation. We identify the core algorithmic primitives across common valuation frameworks that induce prohibitive sensitivity, explaining why straightforward DP mechanisms fail. We further derive design principles for more privacy-amenable valuation procedures and empirically characterize how privacy constraints degrade ranking fidelity across representative methods and datasets. Our results clarify the limits of current approaches and provide a foundation for developing valuation methods that remain useful under rigorous privacy guarantees.

Yiwei Fu, Tianhao Wang, Varun Chandrasekaran

University of Illinois Urbana-Champaign · ∗University of Virginia

Data Valuation Privacy-Utility Tradeoff Differential Privacy

Feb 26, 2026

Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling

Generative models trained on sensitive image datasets risk memorizing and reproducing individual training examples, making strong privacy guarantees essential. While differential privacy (DP) provides a principled framework for such guarantees, standard DP finetuning (e.g., with DP-SGD) often results in severe degradation of image quality, particularly in high-frequency textures, due to the indiscriminate addition of noise across all model parameters. In this work, we propose a spectral DP framework based on the hypothesis that the most privacy-sensitive portions of an image are often low-frequency components in the wavelet space (e.g., facial features and object shapes) while high-frequency components are largely generic and public. Based on this hypothesis, we propose the following two-stage framework for DP image generation with coarse image intermediaries: (1) DP finetune an autoregressive spectral image tokenizer model on the low-resolution wavelet coefficients of the sensitive images, and (2) perform high-resolution upsampling using a publicly pretrained super-resolution model. By restricting the privacy budget to the global structures of the image in the first stage, and leveraging the post-processing property of DP for detail refinement, we achieve promising trade-offs between privacy and utility. Experiments on the MS-COCO and MM-CelebA-HQ datasets show that our method generates images with improved quality and style capture relative to other leading DP image frameworks.

Jasmine Bayrooti, Weiwei Kong, Natalia Ponomareva +3

University of Cambridge

Image Generation Differential Privacy Autoregressive Image Generation

Feb 26, 2026

MAPLE: Metadata Augmented Private Language Evolution

Differentially private (DP) fine-tuning of large language models (LLMs) requires massive compute and full model access, which rules out state-of-the-art proprietary APIs for general users. Generating DP synthetic data offers a practical workaround. This approach also allows for transparent exploratory data analysis and arbitrary reuse across downstream tasks, sidestepping the rigid constraints of a model's parameter space. Private Evolution (PE) provides a promising API-based framework for generating this data, but its success relies heavily on initialization. If the private data distribution falls too far outside the foundation model's pre-training priors -- a common issue in highly specialized domain -- PE struggles to align with the target data. This misalignment causes poor convergence, degraded utility, and wasted API calls. To solve this initialization bottleneck, we introduce Metadata Augmented Private Language Evolution (MAPLE). MAPLE extracts DP tabular metadata and uses in-context learning to firmly ground the initial synthetic distribution in the target domain. Our evaluations on domain-specific text generation tasks show that MAPLE yields a strictly better privacy-utility trade-off, converges significantly faster, and sharply reduces API costs compared to baseline PE methods.

Eli Chien, Yuzheng Hu, Ryan McKenna +3

National Taiwan University · University of Illinois Urbana-Champaign · Google Research +1

Privacy-Preserving Language Models Differentially Private Synthetic Data Generation Synthetic Data Generation Privacy-Preserving ML Differential Privacy Synthetic Data Generation for Language Models

Feb 26, 2026

DPSQL+: A Differentially Private SQL Library with a Minimum Frequency Rule

SQL is the de facto interface for exploratory data analysis; however, releasing exact query results can expose sensitive information through membership or attribute inference attacks. Differential privacy (DP) provides rigorous privacy guarantees, but in practice, DP alone may not satisfy governance requirements such as the minimum frequency rule, which requires each released group (cell) to include contributions from at least

k

distinct individuals. In this paper, we present DPSQL+, a privacy-preserving SQL library that simultaneously enforces user-level

(\varepsilon,δ)

-DP and the minimum frequency rule. DPSQL+ adopts a modular architecture consisting of: (i) a Validator that statically restricts queries to a DP-safe subset of SQL; (ii) an Accountant that consistently tracks cumulative privacy loss across multiple queries; and (iii) a Backend that interfaces with various database engines, ensuring portability and extensibility. Experiments on the TPC-H benchmark demonstrate that DPSQL+ achieves practical accuracy across a wide range of analytical workloads -- from basic aggregates to quadratic statistics and join operations -- and allows substantially more queries under a fixed global privacy budget than prior libraries in our evaluation.

Tomoya Matsumoto, Shokichi Takakura, Shun Takagi +1

LY Corporation

Differential Privacy Relational Databases

Feb 26, 2026

Tackling Privacy Heterogeneity in Differentially Private Federated Learning

Differentially private federated learning (DP-FL) enables clients to collaboratively train machine learning models while preserving the privacy of their local data. However, most existing DP-FL approaches assume that all clients share a uniform privacy budget, an assumption that does not hold in real-world scenarios where privacy requirements vary widely. This privacy heterogeneity poses a significant challenge: conventional client selection strategies, which typically rely on data quantity, cannot distinguish between clients providing high-quality updates and those introducing substantial noise due to strict privacy constraints. To address this gap, we present the first systematic study of privacy-aware client selection in DP-FL. We establish a theoretical foundation by deriving a convergence analysis that quantifies the impact of privacy heterogeneity on training error. Building on this analysis, we propose a privacy-aware client selection strategy, formulated as a convex optimization problem, that adaptively adjusts selection probabilities to minimize training error. Extensive experiments on benchmark datasets demonstrate that our approach achieves up to a 10% improvement in test accuracy on CIFAR-10 compared to existing baselines under heterogeneous privacy budgets. These results highlight the importance of incorporating privacy heterogeneity into client selection for practical and effective federated learning.

Ruichen Xu, Ying-Jun Angela Zhang, Jianwei Huang

Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong · School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network, and CSIJRI Joint Research Centre on Smart Energy Storage, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, P.R. China

Differentially Private Federated Learning Client Selection in Federated Learning Differential Privacy

Feb 25, 2026

Differentially Private Truncation of Unbounded Data via Public Second Moments

Data privacy is important in the AI era, and differential privacy (DP) is one of the golden solutions. However, DP is typically applicable only if data have a bounded underlying distribution. We address this limitation by leveraging second-moment information from a small amount of public data. We propose Public-moment-guided Truncation (PMT), which transforms private data using the public second-moment matrix and applies a principled truncation whose radius depends only on non-private quantities: data dimension and sample size. This transformation yields a well-conditioned second-moment matrix, enabling its inversion with a significantly strengthened ability to resist the DP noise. Furthermore, we demonstrate the applicability of PMT by using penalized and generalized linear regressions. Specifically, we design new loss functions and algorithms, ensuring that solutions in the transformed space can be mapped back to the original domain. We have established improvements in the models' DP estimation through theoretical error bounds, robustness guarantees, and convergence results, attributing the gains to the conditioning effect of PMT. Experiments on synthetic and real datasets confirm that PMT substantially improves the accuracy and stability of DP models.

Zilong Cao, Xuan Bi, Hai Zhang

School of Mathematics, Northwest University · Department of Information and Decision Sciences, Minnesota Carlson, University of Minnesota

Privacy-Preserving ML Differential Privacy

Feb 21, 2026

SLDP: Semi-Local Differential Privacy for Density-Adaptive Analytics

Density-adaptive domain discretization is essential for high-utility privacy-preserving analytics but remains challenging under Local Differential Privacy (LDP) due to the privacy-budget costs associated with iterative refinement. We propose a novel framework, Semi-Local Differential Privacy (SLDP), that assigns a privacy region to each user based on local density and defines adjacency by the potential movement of a point within its privacy region. We present an interactive

(\varepsilon, δ)

-SLDP protocol, orchestrated by an honest-but-curious server over a public channel, to estimate these regions privately. Crucially, our framework decouples the privacy cost from the number of refinement iterations, allowing for high-resolution grids without additional privacy budget cost. We experimentally demonstrate the framework's effectiveness on estimation tasks across synthetic and real-world datasets.

Alexey Kroshnin, Alexandra Suvorikova

Weierstrass Institute for Applied Analysis and Stochastic

Differential Privacy Privacy-Preserving ML

Feb 21, 2026

PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems

Privacy preserving machine learning deployments in sensitive deep learning applications; from medical imaging to autonomous systems; increasingly require combining multiple techniques. Yet, practitioners lack systematic guidance to assess the synergistic and non-additive interactions of these hybrid configurations, relying instead on isolated technique analysis that misses critical system level interactions. We introduce PrivacyBench, a benchmarking framework that reveals striking failures in privacy technique combinations with severe deployment implications. Through systematic evaluation across ResNet18 and ViT models on medical datasets, we uncover that FL + DP combinations exhibit severe convergence failure, with accuracy dropping from 98% to 13% while compute costs and energy consumption substantially increase. In contrast, FL + SMPC maintains near-baseline performance with modest overhead. Our framework provides the first systematic platform for evaluating privacy-utility-cost trade-offs through automated YAML configuration, resource monitoring, and reproducible experimental protocols. PrivacyBench enables practitioners to identify problematic technique interactions before deployment, moving privacy-preserving computer vision from ad-hoc evaluation toward principled systems design. These findings demonstrate that privacy techniques cannot be composed arbitrarily and provide critical guidance for robust deployment in resource-constrained environments.

Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya +2

1ML Collective · 2Friedrich-Alexander-Universität Erlangen-Nürnberg · 3Carnegie Mellon University

Privacy-Preserving ML Differential Privacy Federated Learning Privacy-Utility Tradeoff

Feb 20, 2026

DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning

Differentially private (DP) synthetic data generation plays a pivotal role in developing large language models (LLMs) on private data, where data owners cannot provide eyes-on access to individual examples. Generating DP synthetic data typically involves a difficult trade-off. On one hand, DP finetuning methods train an LLM as a synthetic data generator with formal privacy guarantees, yet it still requires the raw content of private examples for model training. However, methods that avoid direct exposure to private data are bounded by an off-the-shelf, un-finetuned model, whose outputs often lack domain fidelity. Can we train an LLM to generate high-quality synthetic text without eyes-on access to individual private examples? In this work, we introduce Differentially Private Reinforcement Fine-Tuning (DP-RFT), an online reinforcement learning algorithm for synthetic data generation with LLMs. DP-RFT leverages DP-protected nearest-neighbor votes from an eyes-off private corpus as a reward signal for on-policy synthetic samples generated by an LLM. The LLM iteratively learns to generate synthetic data to maximize the expected DP votes through Proximal Policy Optimization (PPO). We evaluate DP-RFT for long-form and domain-specific synthetic data generation, such as news articles, meeting transcripts, and medical article abstracts. Our experiments show that DP-RFT closes the gap between private evolution and DP finetuning methods in terms of the fidelity and downstream utility of the generated synthetic data, while respecting the private data boundary.

Fangyuan Xu, Sihao Chen, Zinan Lin +13

New York University · Microsoft · University of Southern California

Differentially Private Synthetic Data Generation Synthetic Data Generation for Language Models Privacy-Preserving Language Models Privacy-Preserving ML Differential Privacy RL for Generative Models

Feb 19, 2026

JAX-Privacy: A library for differentially private machine learning

JAX-Privacy is a library designed to simplify the deployment of robust and performant mechanisms for differentially private machine learning. Guided by design principles of usability, flexibility, and efficiency, JAX-Privacy serves both researchers requiring deep customization and practitioners who want a more out-of-the-box experience. The library provides verified, modular primitives for critical components for all aspects of the mechanism design including batch selection, gradient clipping, noise addition, accounting, and auditing, and brings together a large body of recent research on differentially private ML.

Ryan McKenna, Galen Andrew, Borja Balle +6

Privacy-Preserving ML Differential Privacy Differentially Private Stochastic Gradient Descent

Feb 19, 2026

Efficient privacy loss accounting for subsampling and random allocation

We consider the privacy amplification properties of a sampling scheme in which a user's data isused in

k

steps chosen randomly and uniformly from a sequence (or set) of

t

steps. This sampling scheme has been recently applied in the context of differentially private optimization (Chua et al., 2024a; Choquette-Choo et al., 2025) and communication-efficient high-dimensional private aggregation (Asi et al., 2026), where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme (Feldman & Shenfeld, 2025; Dong et al., 2025) lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting privacy parameters are not tight due to the approximation steps in the analysis. Second, the computed parameters are either the hockey stick or Renyi divergence, both of which introduce overheads when used in privacy loss accounting. In this work, we demonstrate that the privacy loss distribution (PLD) of random allocation applied to any differentially private algorithm can be computed efficiently. When applied to the Gaussian mechanism, our results demonstrate that the privacy-utility trade-off for random allocation is at least as good as that of Poisson subsampling. In particular, random allocation is better suited for training via DP-SGD. To support these computations, our work develops new tools for general privacy loss accounting based on a notion of PLD realization. This notion allows us to extend accurate privacy loss accounting to subsampling which previously required manual noise-mechanism-specific analysis.

Vitaly Feldman, Moshe Shenfeld

Apple · The Hebrew University of Jerusalem · *Work partially done while author was an intern at Apple.

Privacy Accounting Differential Privacy Privacy-Utility Tradeoff

Feb 18, 2026

Learning with Locally Private Examples by Inverse Weierstrass Private Stochastic Gradient Descent

Releasing data once and for all under noninteractive Local Differential Privacy (LDP) enables complete data reusability, but the resulting noise may create bias in subsequent analyses. In this work, we leverage the Weierstrass transform to characterize this bias in binary classification. We prove that inverting this transform leads to a bias-correction method to compute unbiased estimates of nonlinear functions on examples released under LDP. We then build a novel stochastic gradient descent algorithm called Inverse Weierstrass Private SGD (IWP-SGD). It converges to the true population risk minimizer at a rate of

\mathcal{O}(1/n)

, with

n

the number of examples. We empirically validate IWP-SGD on binary classification tasks using synthetic and real-world datasets.

Jean Dufraiche, Paul Mangold, Michaël Perrot +1

Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, F-59000 Lille, France · CMAP, CNRS, École polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France

Differential Privacy Differentially Private Stochastic Gradient Descent Binary Classification

Feb 18, 2026

Differentially Private Non-convex Distributionally Robust Optimization

Real-world deployments routinely face distribution shifts, group imbalances, and adversarial perturbations, under which the traditional Empirical Risk Minimization (ERM) framework can degrade severely. Distributionally Robust Optimization (DRO) addresses this issue by optimizing the worst-case expected loss over an uncertainty set of distributions, offering a principled approach to robustness. Meanwhile, as training data in DRO always involves sensitive information, safeguarding it against leakage under Differential Privacy (DP) is essential. In contrast to classical DP-ERM, DP-DRO has received much less attention due to its minimax optimization structure with uncertainty constraint. To bridge the gap, we provide a comprehensive study of DP-(finite-sum)-DRO with

ψ

-divergence and non-convex loss. First, we study DRO with general

ψ

-divergence by reformulating it as a minimization problem, and develop a novel

(\varepsilon, δ)

-DP optimization method, called DP Double-Spider, tailored to this structure. Under mild assumptions, we show that it achieves a utility bound of

\mathcal{O}(\frac{1}{\sqrt{n}}+ (\frac{\sqrt{d \log (1/δ)}}{n \varepsilon})^{2/3})

in terms of the gradient norm, where

n

denotes the data size and

d

denotes the model dimension. We further improve the utility rate for specific divergences. In particular, for DP-DRO with KL-divergence, by transforming the problem into a compositional finite-sum optimization problem, we develop a DP Recursive-Spider method and show that it achieves a utility bound of

\mathcal{O}((\frac{\sqrt{d \log(1/δ)}}{n\varepsilon})^{2/3} )

, matching the best-known result for non-convex DP-ERM. Experimentally, we demonstrate that our proposed methods outperform existing approaches for DP minimax optimization.

Difei Xu, Meng Ding, Zebin Ma +4

Provable Responsible AI and Data Analytics (PRADA) Lab · King Abdullah University of Science and Technology · State University of New York at Buffalo +1

Distributionally Robust Optimization Differential Privacy Min-Max Optimization Nonconvex Stochastic Optimization

Feb 17, 2026

Certified Per-Instance Unlearning Using Individual Sensitivity Bounds

Certified machine unlearning can be achieved through noise injection, with differential privacy guarantees obtained by calibrating the noise to worst-case sensitivity. Yet, the influence of individual training points is highly heterogeneous, making such uniform calibration unnecessarily conservative for most deletion requests. We consider certified per-instance unlearning, a data-dependent notion of certified deletion in which the unlearning mechanism and its certificate can adapt to the point being removed. Our framework turns point-specific sensitivity information into formal unlearning guarantees, allowing the amount of noise to reflect the difficulty of the deletion request rather than the worst case. When training is deterministic, we derive explicit per-instance unlearning certificates. For noisy gradient descent, in the case of ridge regression, similar certificates are obtained through high-probability sensitivity bounds. Experiments on NYU-Depth with frozen DINOv2 features show substantial variation in deletion difficulty across points, and demonstrate that per-instance calibration can substantially improve utility over uniform worst-case calibration.

Hanna Benarroch, Jamal Atif, Olivier Cappé

DI-ENS · CMAP

Personalized Differential Privacy Machine Unlearning Differential Privacy

Feb 16, 2026

Differentially Private Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) is a widely used framework for reducing hallucinations in large language models (LLMs) on domain-specific tasks by retrieving relevant documents from a database to support accurate responses. However, when the database contains sensitive corpora, such as medical records or legal documents, RAG poses serious privacy risks by potentially exposing private information through its outputs. Prior work has demonstrated that one can practically craft adversarial prompts that force an LLM to regurgitate the augmented contexts. A promising direction is to integrate differential privacy (DP), a privacy notion that offers strong formal guarantees, into RAG systems. However, naively applying DP mechanisms into existing systems often leads to significant utility degradation. Particularly for RAG systems, DP can reduce the usefulness of the augmented contexts leading to increase risk of hallucination from the LLMs. Motivated by these challenges, we present DP-KSA, a novel privacy-preserving RAG algorithm that integrates DP using the propose-test-release paradigm. DP-KSA follows from a key observation that most question-answering (QA) queries can be sufficiently answered with a few keywords. Hence, DP-KSA first obtains an ensemble of relevant contexts, each of which will be used to generate a response from an LLM. We utilize these responses to obtain the most frequent keywords in a differentially private manner. Lastly, the keywords are augmented into the prompt for the final output. This approach effectively compresses the semantic space while preserving both utility and privacy. We formally show that DP-KSA provides formal DP guarantees on the generated output with respect to the RAG database. We evaluate DP-KSA on two QA benchmarks using three instruction-tuned LLMs, and our empirical results demonstrate that DP-KSA achieves a strong privacy-utility tradeoff.

Tingting Tang, James Flemings, Yongqin Wang +1

University of Southern California Los Angeles, California, USA

Privacy-Preserving RAG Retrieval-Augmented Generation Differential Privacy

Feb 14, 2026

Locally Private Parametric Methods for Change-Point Detection

We study parametric change-point detection under local differential privacy, focusing on the offline localization of a single change point in a sequence of independent observations.~In the non-private setting, we derive improved finite-sample accuracy guarantees for a generalized likelihood-ratio estimator using martingale methods.~We establish finite-sample minimax lower bounds in both the non-private and locally private settings, showing that localization accuracy is fundamentally limited by the Chernoff information between the pre- and post-change distributions.~Guided by this, we design locally private CPD algorithms based on aggregated randomized response and binary mechanisms, which maximize the Chernoff information between the induced output distributions in low- and high-privacy regimes, respectively, and derive their finite-sample guarantees. Together, our upper and lower bounds quantify the statistical cost of privacy in change-point localization.~As part of our analysis, we also show that strong data processing inequality coefficients of Rényi divergence and Jeffreys--Rényi divergence of order infinity can be computed by restricting to pairs of input distributions with a common support of at most two symbols.~We corroborate our results through empirical evaluation on synthetic data.

Anuj Kumar Yadav, Cemre Cadir, Yanina Y. Shkel +1

School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne (EPFL), Vaud, Switzerland.

Change-Point Detection Differential Privacy Minimax Estimation

Feb 2, 2026

On the Inherent Privacy Amplification of Missing Data

Privacy preservation is critical in many high-stakes domains such as medicine and finance, where sensitive data must be analyzed without compromising individual confidentiality. At the same time, these applications often involve datasets with inherent missing values due to non-response or data corruption for example. Missing data is traditionally analyzed through its impact on statistical efficiency and model performance. In fact, it reduces the information available to analysts and can degrade the final utility of the model. In this work, we take an alternative approach and study missing data through the lens of privacy preservation. Intuitively, when features are missing, less information is revealed about individuals, suggesting that data missingness could inherently enhance privacy. We formalize this intuition within a novel framework that integrates missing data into differential privacy. In essence, our approach accounts for the de facto missingness in the data to refine existing privacy guarantees without modifying the underlying mechanism. Using this framework, we show for the first time that missing data can induce inherent privacy amplification for differentially private algorithms, highlighting a previously overlooked interaction between missing data and formal privacy guarantees..

Simon Roburin, Rafa{ë}l Pinot, Erwan Scornet

LPSM · Sorbonne Université, Université Paris Cité, CNRS Laboratoire de Probabilités, Statistique et Modélisation (LPSM) F-75005 Paris, France

Privacy-Preserving ML Differential Privacy

Jan 27, 2026

The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis

Differential privacy limits how much each training image can influence a medical imaging model but lowers its diagnostic accuracy. Pretrained weights reduce this loss. It is unknown whether the pretraining objective or domain improves accuracy more. Here we train chest radiograph classifiers under differential privacy from five starting points on 591,884 radiographs from five datasets. We show that supervised pretraining on chest radiographs ranks first in 24 of 25 combinations of dataset and privacy setting. Measured by the area under the receiver operating characteristic curve, its lead over supervised pretraining on natural images is 2.6% without privacy and 15.3% at the strictest budget. From that starting point, changing the domain improves this area 2.2 to 3.7 times more than changing the objective. Private pretraining on chest radiographs lowers this area by about 5% under privacy but outperforms the three starting points without patient data at the strictest budget. Under privacy, supervised chest radiograph pretraining leads in the worst-performing demographic subgroup.

Soroosh Tayebi Arasteh, Mina Farajiamiri, Mahshad Lotfinia +7

Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany · Department of Urology, Stanford University, Stanford, CA, USA +5

Privacy-Utility Tradeoff Medical Image Classification Differential Privacy Domain-Adaptive Pretraining

Dec 29, 2025

Improved Bounds for Private and Robust Alignment

In this paper, we study the private and robust alignment of language models from a theoretical perspective by establishing upper bounds on the suboptimality gap in both offline and online settings. We consider preference labels subject to privacy constraints and/or adversarial corruption, and analyze two distinct interplays between them: privacy-first and corruption-first. For the privacy-only setting, we show that log loss with an MLE-style algorithm achieves near-optimal rates, in contrast to conventional wisdom. For the joint privacy-and-corruption setting, we first demonstrate that existing offline algorithms in fact provide stronger guarantees -- simultaneously in terms of corruption level and privacy parameters -- than previously known, which further yields improved bounds in the corruption-only regime. In addition, we also present the first set of results for private and robust online alignment. Our results are enabled by new uniform convergence guarantees for log loss and square loss under privacy and corruption, which we believe have broad applicability across learning theory and statistics.

Wenqian Weng, Yi He, Xingyu Zhou

Department of Electrical and Computer Engineering, Wayne State University, Detroit, USA.

Privacy-Preserving Language Models Corruption Robustness Differential Privacy Preference Alignment

Dec 3, 2025

Efficient Public Verification of Private ML via Regularization

Training with differential privacy (DP) guarantees dataset members that they cannot be identified by users of the released model. However, those data providers, and, in general, the public, lack methods to efficiently verify that models trained on their data satisfy DP guarantees. The amount of compute needed to verify DP guarantees for current algorithms scales with the amount of computation required to train the model. In this paper we design the first DP algorithm with near optimal privacy-utility trade-offs but whose DP guarantees can be verified cheaper than training. We focus on DP stochastic convex optimization (DP-SCO), where optimal privacy-utility trade-offs are known. Here we show we can obtain tight privacy-utility trade-offs by privately minimizing a series of regularized objectives and only using the standard DP composition bound. Crucially, this method can be verified with much less compute than training. This leads to the first known DP-SCO algorithm with near optimal privacy-utility whose DP verification scales better than training cost, significantly reducing verification costs on large datasets.

Zoë Ruha Bell, Anvith Thudi, Olive Franzese-McLaughlin +2

Department of Computer Science, University of California, Berkeley, USA · Department of Computer Science, University of Toronto, Toronto, Canada · Vector Institute, Toronto, Canada

Differential Privacy

Nov 17, 2025

On the Gradient Complexity of Private Optimization with Private Oracles

We study the running time, in terms of first order oracle queries, of differentially private empirical/population risk minimization of Lipschitz convex losses. We first consider the setting where the loss is non-smooth and the optimizer interacts with a private proxy oracle, which sends only private messages about a minibatch of gradients. In this setting, we show that expected running time

Ω(\min\{\frac{\sqrt{d}}{α^2}, \frac{d}{\log(1/α)}\})

is necessary to achieve

α

excess risk on problems of dimension

d

when

d \geq 1/α^2

. Upper bounds via DP-SGD show these results are tight when

d>\tildeΩ(1/α^4)

. We further show our lower bound can be strengthened to

Ω(\min\{\frac{d}{\bar{m}α^2}, \frac{d}{\log(1/α)} \})

for algorithms which use minibatches of size at most

\bar{m} < \sqrt{d}

. We next consider smooth losses, where we relax the private oracle assumption and give lower bounds under only the condition that the optimizer is private. Here, we lower bound the expected number of first order oracle calls by

\tildeΩ\big(\frac{\sqrt{d}}α + \min\{\frac{1}{α^2}, n\}\big)

, where

n

is the size of the dataset. Modifications to existing algorithms show this bound is nearly tight. Compared to non-private lower bounds, our results show that differentially private optimizers pay a dimension dependent runtime penalty. Finally, as a natural extension of our proof technique, we show lower bounds in the non-smooth setting for optimizers interacting with information limited oracles. Specifically, if the proxy oracle transmits at most

Γ

-bits of information about the gradients in the minibatch, then

Ω\big(\min\{\frac{d}{α^2Γ}, \frac{d}{\log(1/α)}\}\big)

oracle calls are needed. This result shows fundamental limitations of gradient quantization techniques in optimization.

Michael Menart, Aleksandar Nikolov

Department of Computer Science, University of Toronto, Vector Institute · Department of Computer Science, University of Toronto

Query Complexity Oracle Complexity Differentially Private Stochastic Gradient Descent Differential Privacy Gradient Quantization Nonsmooth Convex Optimization+1

Nov 17, 2025

Beyond One-Size-Fits-All: Neural Networks for Differentially Private Tabular Data Synthesis

In differentially private (DP) tabular data synthesis, the consensus is that statistical models are better than neural network (NN)-based methods. However, we argue that this conclusion is incomplete and overlooks the challenge of densely correlated datasets, where intricate dependencies can overwhelm statistical models. In such complex scenarios, neural networks are more suitable due to their capacity to fit complex distributions by learning directly from samples. Despite this potential, existing NN-based algorithms still suffer from significant limitations. We therefore propose MargNet, incorporating successful algorithmic designs of statistical models into neural networks. MargNet applies an adaptive marginal selection strategy and trains the neural networks to generate data that conforms to the selected marginals. On sparsely correlated datasets, our approach achieves utility close to the best statistical method while offering an average 7

\times

speedup over it. More importantly, on densely correlated datasets, MargNet establishes a new state-of-the-art, reducing fidelity error by up to 26% compared to the previous best. We release our code on GitHub.\footnote{https://github.com/KaiChen9909/margnet}

Kai Chen, Chen Gong, Tianhao Wang

University of Virginia Charlottesville, USA

Synthetic Data Generation Synthetic Tabular Data Generation Differential Privacy Differentially Private Synthetic Data Generation