Convex Machine Learning

Convex machine learning focuses on developing and analyzing machine learning models with convex objective functions, offering advantages like efficient optimization and strong theoretical guarantees. Current research explores extending these benefits to non-convex domains like deep neural networks, improving training methods (e.g., via implicit updates and Hessian approximations), and developing novel convex-based architectures for tasks such as text generation and classification in non-Euclidean spaces. This work is significant because it enhances model robustness, interpretability, and efficiency, particularly in data-constrained settings and applications requiring privacy-preserving distributed learning.

Papers

October 30, 2024

Tightening convex relaxations of trained neural networks: a unified approach for convex and S-shaped activations
Pablo Carrasco, Gonzalo Muñoz
Neural Network Constructive Approach Convex Relaxation Convex Machine Learning Convex Approximation

August 1, 2024

Towards Certified Unlearning for Deep Neural Networks
Binchi Zhang, Yushun Dong, Tianhao Wang, Jundong Li
Deep Neural Network Machine Unlearning Unlearning Method Non Convex Objective Convex Machine Learning

October 26, 2023

Beyond MLE: Convex Learning for Text Generation
Chenze Shao, Zhengrui Ma, Min Zhang, Yang Feng
Large Language Model Text Generation Generative Language Model Maximum Likelihood Text Generation Model Convex Machine Learning

August 14, 2023

Federated Classification in Hyperbolic Spaces via Secure Aggregation of Convex Hulls
Saurav Prakash, Jin Sima, Chao Pan, Eli Chien, Olgica Milenkovic
Secure Aggregation Hyperbolic Space Convex Hull Euclidean Space Federated Image Classification Convex Machine Learning

July 22, 2023

Implicit Interpretation of Importance Weight Aware Updates
Keyi Chen, Francesco Orabona
Implicit Language Weight Update Follow the Regularized Leader Subgradient Method Convex Machine Learning

February 21, 2023

On Inductive Biases for Machine Learning in Data Constrained Settings
Grégoire Mialon
Machine Learning Machine Learning Model Transfer Learning Inductive Bias Model Interpretability Convex Machine Learning

February 4, 2023

How Many and Which Training Points Would Need to be Removed to Flip this Prediction?
Jinghan Yang, Sarthak Jain, Byron C. Wallace
Native Robustness Training Data Human Prediction Model Prediction Training Point Convex Machine Learning

February 3, 2023

Asymmetric Certified Robustness via Feature-Convex Neural Networks
Samuel Pfrommer, Brendon G. Anderson, Julien Piet, Somayeh Sojoudi
Adversarial Robustness Convex Set Certified Robustness Input Convex Neural Network Convex Neural Network Convex Machine Learning

November 16, 2022

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
Zachary Frangella, Pratik Rathore, Shipu Zhao, Madeleine Udell
Stochastic Gradient Hessian Matrix Stochastic Optimization Approximate Curvature Convex Machine Learning

June 20, 2022

ConvGeN: Convex space learning improves deep-generative oversampling for tabular imbalanced classification on smaller datasets
Kristian Schultz, Saptarshi Bej, Waldemar Hahn, Markus Wolfien, Prashant Srivastava, Olaf Wolkenhauer
Synthetic Data Synthetic Data Generation Table Semantics Sample Generation Synthetic Tabular Data Generation Small Scale Datasets Convex Machine Learning Generative Sampling

May 27, 2022

Don't Explain Noise: Robust Counterfactuals for Randomized Ensembles
Alexandre Forel, Axel Parmentier, Thibaut Vidal
Native Robustness Counterfactual Explanation Industrial Disturbing Noise Ensemble Classifier Convex Machine Learning Robust Counterfactuals Meaningful Counterfactuals