Private Optimization

Private optimization aims to train machine learning models while guaranteeing the privacy of individual data points, typically using differential privacy mechanisms. Current research focuses on improving the efficiency and accuracy of differentially private optimizers, such as variants of stochastic gradient descent (SGD), by incorporating techniques like Kalman filtering, low-pass filtering, and adaptive preconditioners to mitigate the negative impact of added noise. These advancements are crucial for deploying machine learning in sensitive domains like healthcare and finance, where data privacy is paramount, and are driving progress in both theoretical understanding and practical applications of differential privacy.

Papers

October 10, 2022

Rieoptax: Riemannian Optimization in JAX
Saiteja Utpala, Andi Han, Pratik Jawanpuria, Bamdev Mishra
Riemannian Manifold Riemannian Optimization State of the Art JAX Private Optimization Geometric Primitive Riemannian Stochastic

September 30, 2022

Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, George Karypis
Large Model Private Stochastic Differentially Private Private Optimization Minimal Cost

September 15, 2022

Differentially Private Estimation of Hawkes Process
Simiao Zuo, Tianyi Liu, Tuo Zhao, Hongyuan Zha
Differential Privacy Hawkes Process Private Estimation Private Optimization

July 21, 2022

Differentially Private Partial Set Cover with Applications to Facility Location
George Z. Li, Dung Nguyen, Anil Vullikanti
Financial Application Private Algorithm Private Optimization Facility Location Problem Facility Location Cover Problem

July 18, 2022

Private Convex Optimization in General Norms
Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian
Stochastic Convex Optimization Learning Norm Private Optimization Private Convex Optimization Schatten $P$ Norm

March 13, 2022

Private Non-Convex Federated Learning Without a Trusted Server
Andrew Lowy, Ali Ghafelebashi, Meisam Razaviyayn
Differential Privacy Private Federated Learning Non Convex Loss Function Private Optimization

February 12, 2022

Private Adaptive Optimization with Side Information
Tian Li, Manzil Zaheer, Sashank J. Reddi, Virginia Smith
Differential Privacy Privacy Guarantee Adaptive Preconditioner Privacy Utility Side Information Private Optimization

January 6, 2022

Learning to be adversarially robust and differentially private
Jamie Hayes, Borja Balle, M. Pawan Kumar
LeArning Abstract Differential Privacy Adversarial Training Adversarial Perturbation Private Stochastic Private Optimization