Distributional Robustness

Distributional robustness in machine learning aims to develop models that perform reliably across diverse data distributions, mitigating the impact of data shifts between training and deployment. Current research focuses on developing algorithms and optimization frameworks, such as those based on Wasserstein distances and minimax formulations, to enhance model robustness against adversarial attacks and subpopulation shifts, often incorporating techniques from Bayesian methods and causal inference. This field is crucial for building trustworthy and reliable AI systems, particularly in high-stakes applications where model performance must be consistent across various demographics and environmental conditions.

Papers

December 20, 2022

Distributional Robustness Bounds Generalization Errors
Shixiong Wang, Haowei Wang
Robust Optimization Distributional Robustness Distributional Uncertainty

October 22, 2022

Exploring The Landscape of Distributional Robustness for Question Answering Models
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
Native Robustness Landscape Image Better Robustness Distributional Robustness Question Answering Model Span Prediction

October 21, 2022

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Yaru Niu, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
Robust Reinforcement Learning Multi Task Reinforcement Learning Distributional Robustness Hierarchical Latent Hierarchical MDP Distributionally Robust Markov Decision Process

October 6, 2022

Distributionally Adaptive Meta Reinforcement Learning
Anurag Ajay, Abhishek Gupta, Dibya Ghosh, Sergey Levine, Pulkit Agrawal
Meta Reinforcement Learning Distributional Robustness Policy Alignment Meta Reinforcement Meta Rl Algorithm

September 30, 2022

Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty
Ariel Neufeld, Julian Sester
Practical Algorithm Markov Decision Process Robust Version Distributional Robustness Stochastic Optimal Control Q$ Learning Algorithm Wasserstein Uncertainty Robust Markov Decision

September 15, 2022

Explicit Tradeoffs between Adversarial and Natural Distributional Robustness
Mazda Moayeri, Kiarash Banihashem, Soheil Feizi
Adversarial Training Adversarial Robustness Adversarial Text Adversarial Vulnerability Distributional Robustness

August 5, 2022

Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
Christian Fröhlich, Robert C. Williamson
Kullback Leibler Divergence Distributional Robustness Risk Minimization Thin Tail Risk Measure Ambiguity Set Divergence Measure

June 16, 2022

Methods for Estimating and Improving Robustness of Language Models
Michal Štefánik
Large Language Model Language Model Native Robustness NCD Method Estimation Task Neural Model Semantic Relation Distributional Robustness

June 10, 2022

Is Self-Supervised Learning More Robust Than Supervised Learning?
Yuanyi Zhong, Haoran Tang, Junkun Chen, Jian Peng, Yu-Xiong Wang
Contrastive Learning Self Supervised Learning Supervised Learning Robust Version Visual Representation Distributional Robustness

June 7, 2022

Enhancing Distributional Stability among Sub-populations
Jiashuo Liu, Jiayun Wu, Jie Peng, Xiaoyu Wu, Yang Zheng, Bo Li, Peng Cui
Distribution Shift Distributional Robustness Invariant Learning Stable Optimization Machine Learning Assumption

May 31, 2022

May 3, 2022

Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)
Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, Ludwig Schmidt
Loss Function Single CLIP Contrastive Language Image Distributional Robustness Enhanced Robustness Language Image

April 7, 2022

What You See is What You Get: Principled Deep Learning via Distributional Generalization
Bogdan Kulynych, Yao-Yuan Yang, Yaodong Yu, Jarosław Błasiok, Preetum Nakkiran
Deep Learning Adversarial Training Distribution Generalization Style Representation Distributional Robustness

March 23, 2022

Wasserstein Distributionally Robust Optimization with Wasserstein Barycenters
Tim Tsz-Kit Lau, Han Liu
Robust Optimization Nonconvex Optimization Distributional Robustness Wasserstein Barycenter Robust Optimization Problem Wasserstein Distributionally Robust

March 1, 2022

Global-Local Regularization Via Distributional Robustness
Hoang Phan, Trung Le, Trung Phung, Tuan Anh Bui, Nhat Ho, Dinh Phung
Loss Function Adversarial Learning Distributional Robustness

February 27, 2022

A Unified Wasserstein Distributional Robustness Framework for Adversarial Training
Tuan Anh Bui, Trung Le, Quan Tran, He Zhao, Dinh Phung
Adversarial Attack Adversarial Example Adversarial Training Distributional Robustness Wasserstein Distributionally Robust

February 3, 2022

Certifying Out-of-Domain Generalization for Blackbox Functions
Maurice Weber, Linyi Li, Boxin Wang, Zhikuan Zhao, Bo Li, Ce Zhang
Native Robustness Loss Function Domain Generalization Black Box Function Distributional Robustness

December 20, 2021

Distributionally Robust Group Backwards Compatibility
Martin Bertran, Natalia Martinez, Alex Oesterling, Guillermo Sapiro
Machine Learning Model Model Performance Distributional Robustness Image Classification Datasets Demographic Fairness

December 12, 2021

Fairness for Robust Learning to Rank
Omid Memarrast, Ashkan Rezaei, Rizal Fathony, Brian Ziebart
Procedural Fairness Group Fairness Robust Learning Stable Rank Fairness Constraint Distributional Robustness Proportional Ranking Minimax Game