Private Convex Optimization

Private convex optimization focuses on developing algorithms that solve convex optimization problems while preserving the privacy of the training data, typically using differential privacy techniques. Current research emphasizes improving the efficiency and accuracy of these algorithms, exploring various approaches like differentially private stochastic gradient descent (SGD), second-order methods (e.g., Newton's method), and exponential mechanisms, often applied to models such as neural networks (including ReLU networks) and generalized linear models. These advancements are crucial for enabling the responsible use of machine learning in sensitive domains, offering a balance between model utility and data privacy. The development of efficient and provably private algorithms is driving progress in both theoretical understanding and practical applications of private machine learning.

Papers

July 5, 2024

Differentially Private Convex Approximation of Two-Layer ReLU Networks
Antti Koskela
Private Stochastic Privacy Utility Two Layer ReLU Private Gradient Private Convex Optimization Private Convex

May 21, 2024

Uncertainty quantification by block bootstrap for differentially private stochastic gradient descent
Holger Dette, Carina Graw
Differential Privacy Stochastic Gradient Descent Uncertainty Quantification Local Differential Privacy Private Stochastic Gradient Descent Private Convex Optimization

February 17, 2024

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization
Andrew Lowy, Jonathan Ullman, Stephen J. Wright
Natural Gradient Convex Function Non Convex Loss Function Fast Rate Private Convex Optimization Private Empirical Risk Minimization

December 21, 2023

R\'enyi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration
Clément Pierquin, Aurélien Bellet, Marc Tommasi, Matthieu Boussard
Differential Privacy Privacy Amplification Additive Noise Mechanistic Understanding Private Convex Optimization Pufferfish Privacy

November 22, 2023

Differentially Private Non-Convex Optimization under the KL Condition with Optimal Rates
Michael Menart, Enayat Ullah, Raman Arora, Raef Bassily, Cristóbal Guzmán
Differential Privacy Gradient Descent Optimal Rate Private Convex Optimization Private Empirical Risk Minimization Noisy Gradient Descent

August 17, 2023

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient
Rongfei Fan, Xuming An, Shiyuan Zuo, Han Hu
Data Aggregation Air Federated Learning Air Computation Convex Loss Function Aggregated Gradient Private Convex Optimization Gradient Normalization

May 22, 2023

Faster Differentially Private Convex Optimization via Second-Order Methods
Arun Ganesh, Mahdi Haghifam, Thomas Steinke, Abhradeep Thakurta
Gradient Descent Convex Optimization Second Order Convex Loss Function Second Order Algorithm Private Convex Optimization

July 18, 2022

Private Convex Optimization in General Norms
Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian
Stochastic Convex Optimization Learning Norm Private Optimization Private Convex Optimization Schatten $P$ Norm

June 27, 2022

Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization
Xiaodong Yang, Huishuai Zhang, Wei Chen, Tie-Yan Liu
Differential Privacy Gradient Norm Normalization Dictionary Small Perturbation Clipped Stochastic Gradient Descent Private Convex Optimization

May 19, 2022

Differentially private Riemannian optimization
Andi Han, Bamdev Mishra, Pratik Jawanpuria, Junbin Gao
Riemannian Gradient Riemannian Geometry Gaussian Mechanism Private Convex Optimization Private Empirical Risk Minimization

March 1, 2022

Private Convex Optimization via Exponential Mechanism
Sivakanth Gopi, Yin Tat Lee, Daogao Liu
Exponential Mechanism Log Concave Private Convex Optimization Private Convex

February 9, 2021

Output Perturbation for Differentially Private Convex Optimization: Faster and More General
Andrew Lowy, Meisam Razaviyayn
Private Algorithm Private Gradient Convex Loss Function Private Convex Optimization Private Empirical Risk Minimization

Private Convex Optimization

Papers

Differentially Private Convex Approximation of Two-Layer ReLU Networks

Uncertainty quantification by block bootstrap for differentially private stochastic gradient descent

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

R\'enyi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration

Differentially Private Non-Convex Optimization under the KL Condition with Optimal Rates

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient

Faster Differentially Private Convex Optimization via Second-Order Methods

Private Convex Optimization in General Norms

Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization

Differentially private Riemannian optimization

Private Convex Optimization via Exponential Mechanism

Output Perturbation for Differentially Private Convex Optimization: Faster and More General