Group Distributionally Robust Optimization

Latest papers

9 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Dynamic Minimax Regret Optimization for Robust LLM Post-Training
Oct 5, 2026Chengbo Zang, Haoyu Dong, Mehmet Kerem Turkcan +3Group Distributionally Robust Optimization Dynamic Regret Minimization
Department of Electrical Engineering, Columbia University · Department of Computer Science & Engineering, University of California, Santa Cruz · Department of Civil Engineering, Columbia University
Learning the Robustness Mechanism with Bilevel Optimization
Sep 28, 2026Yiyang Shen, Qihang Lin, Weiran WangDistribution Shift Robustness Group Distributionally Robust Optimization
Department of Informatics University of Iowa · Tippie College of Business University of Iowa · Department of Computer Science University of Iowa
Equitable System-Prompt Selection via Constrained Mixed-Strategy GroupDRO
Aug 5, 2026Mengyu Xu, Qiaoxin Yang, Zhihan Liu +4Group Distributionally Robust Optimization LLM Prompting
1The University of Chicago · 2SynAI Technologies, Inc. · 3Northwestern University +2
Breaking Spurious Correlations via Generative Randomization and Cross-Variant Self-Supervised Learning
Jul 7, 2026Suraj Yadav, Anjaneya Sharma, Siddharth YadavContrastive Learning Group Distributionally Robust Optimization
Indraprastha Institute of Information Technology Delhi
Distributionally Robust Linear Regression With Block Lewis Weights
Jun 30, 2026Naren Sarayu Manoj, Kumar Kshitij PatelRobust Regression Group Distributionally Robust Optimization
TTIC · Institute for Foundations of Data Science, Yale University
Fair and Calibrated Toxicity Detection with Robust Training and Abstention
May 13, 2026Mokshit SuranaProbability Calibration Group Distributionally Robust Optimization
University of Illinois Chicago
Robust Fair Disease Diagnosis in CT Images
Apr 8, 2026Justin Li, Daniel Ding, Asmita Yuki Pritha +3Group Distributionally Robust Optimization Fairness in Medical Image Classification
Carmel High School, Carmel, Indiana, USA · Capstone School Dhaka, Bangladesh · Clarkstown High School South, West Nyack, New York, USA +2
Reproducibility study on how to find Spurious Correlations, Shortcut Learning, Clever Hans or Group-Distributional non-robustness and how to fix them
Apr 6, 2026Ole Delzer, Sidney BenderGroup Distributionally Robust Optimization Spurious Correlation Robustness
Machine Learning Group Technische Universität Berlin
Towards Fair and Robust Volumetric CT Classification via KL-Regularised Group Distributionally Robust Optimisation
Mar 16, 2026Samuel Johnny, Blessed Guda, Goodness Obasi +2Group Distributionally Robust Optimization Medical Image Classification
Carnegie Mellon University Africa, Kigali, Rwanda · Carnegie Mellon University, Pittsburgh, USA

Oct 5, 2026J/K move · Enter open · S save

Dynamic Minimax Regret Optimization for Robust LLM Post-Training

Chengbo Zang, Haoyu Dong, Mehmet Kerem Turkcan, Gil Zussman+2

Department of Electrical Engineering, Columbia University · Department of Computer Science & Engineering, University of California, Santa Cruz · Department of Civil Engineering, Columbia University

Group Distributionally Robust Optimization Dynamic Regret Minimization Distributionally Robust Optimization Robust Multi-Armed Bandits LLM Post-Training LLM Training

Open paper AskPDF arXiv

Modern LLM training increasingly relies on heterogeneous data sources spanning different domains, tasks, preference distributions, and difficulty levels. We study dynamic minimax regret for group-distributionally robust LLM post-training under instantaneous mini-batch-only bandit feedback. The framework views the training as a two-player sampler-optimizer process: a sampler adaptively selects among data sources using bandit feedback, while an optimizer updates the model parameters using stochastic gradients from the selected source. We focus on the practically restrictive setting where source losses evolve with model training but historical data are not re-evaluated, requiring the sampler to track instantaneous worst-sources from stale partial feedback. We propose DUCB-OGD, a simple and scalable algorithm that couples a Discounted Upper-Confidence-Bound sampler with an Online Gradient Descent optimizer. The sampler maintains exponential moving average loss estimates and confidence radii based on discounted effective sample sizes, avoiding costly re-evaluation of past data or intrusive changes to standard training pipelines. For

K

data sources and

T

training steps, we prove that DUCB-OGD achieves a dynamic minimax regret of

\tilde{O}(K^{1/4}T^{3/4})

, which is optimal up to logarithmic factors for the undiscounted objective under our feedback model. Extensive experiments across supervised fine-tuning, preference optimization, and reinforcement learning show that DUCB-OGD integrates seamlessly into modern LLM training pipelines and improves worst-group robustness with negligible computational overhead compared with standard sampling baselines.