Stochastic Learning

Stochastic learning focuses on developing and analyzing algorithms that learn from data using random sampling and iterative updates, aiming to improve efficiency and robustness compared to deterministic methods. Current research emphasizes the impact of optimization algorithms and model architectures (like convolutional networks and gradient boosted trees) on generalization performance, exploring techniques like Bayesian approaches and preconditioning to enhance training speed and accuracy in various applications, including ranking and federated learning. This field is crucial for advancing machine learning in large-scale and complex settings, offering solutions for handling heterogeneous data, mitigating adversarial attacks, and improving the efficiency of resource-intensive tasks.

Papers

November 1, 2024

ROSS:RObust decentralized Stochastic learning based on Shapley values
Lina Wang, Yunsheng Yuan, Feng Li, Lingjie Duan
Shapley Value Decentralized Learning Stochastic Way Model Gradient Stochastic Learning

June 17, 2024

Just How Flexible are Neural Networks in Practice?
Ravid Shwartz-Ziv, Micah Goldblum, Arpit Bansal, C. Bayan Bruss, Yann LeCun, Andrew Gordon Wilson
Neural Network Convolutional Neural Network Practice Mode Deep Architecture Stochastic Learning Full Batch Gradient Descent

May 23, 2024

Generalization of Hamiltonian algorithms
Andreas Maurer
Strong Generalization Gibbs Algorithm Data Driven Prior PAC Bayesian Bound Stochastic Learning Pseudo Hamiltonian

May 21, 2024

Stochastic Learning of Computational Resource Usage as Graph Structured Multimarginal Schr\"odinger Bridge
Georgiy A. Bondar, Robert Gifford, Linh Thi Xuan Phan, Abhishek Halder
Schr\"odinger Bridge Stochastic Learning

April 18, 2024

Estimating the Hessian Matrix of Ranking Objectives for Stochastic Learning to Rank with Gradient Boosted Trees
Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis
Hessian Matrix Stable Rank Learning to Rank Gradient Boosted Tree Probabilistic Ranking Stochastic Learning

February 13, 2024

Preconditioners for the Stochastic Training of Implicit Neural Representations
Shin-Fang Chng, Hemanth Saratchandran, Simon Lucey
Neural Radiance Field Second Order Adaptive Preconditioner Stochastic Learning Diagonal Preconditioners

November 27, 2023

A Generic Stochastic Hybrid Car-following Model Based on Approximate Bayesian Computation
Jiwan Jiang, Yang Zhou, Xin Wang, Soyoung Ahn
Full Model Vehicle Trajectory Traffic Dynamic Car Following Model Approximate Bayesian Computation Stochastic Learning Generic Stochastic

November 1, 2023

Applications of the Theory of Aggregated Markov Processes in Stochastic Learning Theory
Fangyuan Lin
Financial Application Theoretical Understanding Stochastic Process Markov Model Markov Process Stochastic Learning Q Aggregation

November 16, 2022

Impact of Redundancy on Resilience in Distributed Optimization and Learning
Shuo Liu, Nirupam Gupta, Nitin H. Vaidya
LeArning Abstract Optimization Purpose Global Impact Information Redundancy Tiny Refinement Elicit Resilience Distributed Optimization Stochastic Learning Byzantine Agent Model Resilience

July 12, 2022

Simultaneously Learning Stochastic and Adversarial Bandits under the Position-Based Model
Cheng Chen, Canzhe Zhao, Shuai Li
Follow the Regularized Leader Adversarial Environment Adversarial Bandit Position Prediction Stochastic Learning

June 16, 2022

Simultaneously Learning Stochastic and Adversarial Bandits with General Graph Feedback
Fang Kong, Yichi Zhou, Shuai Li
Adversarial Bandit Feedback Graph Stochastic Learning Stochastic Feedback

May 20, 2022

ClusterEA: Scalable Entity Alignment with Stochastic Training and Normalized Mini-batch Similarities
Yunjun Gao, Xiaoze Liu, Junyang Wu, Tianyi Li, Pengfei Wang, Lu Chen
Batch Normalization Entity Alignment Entity Embeddings Normalization Technique Embedding Based Stochastic Learning Equivalent Entity

March 4, 2022

Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms
Milad Sefidgaran, Amin Gohari, Gaël Richard, Umut Şimşekli
Strong Generalization Generalization Bound Generalization Error Information Theoretic Rate Distortion Information Theoretic Generalization Bound Stochastic Learning

December 24, 2021

Stochastic Learning Equation using Monotone Increasing Resolution of Quantization
Jinwuk Seok, Jeong-Si Kim
Quantization Operator Global Optimization Multiplier Free Quantization Quantization Error Conjunct ReSolution Non Negative Monotone Stochastic Learning Precision Quantization GD Learning

December 17, 2021

Federated Learning with Superquantile Aggregation for Heterogeneous Data
Krishna Pillutla, Yassine Laguel, Jérôme Malick, Zaid Harchaoui
Federated Learning Heterogeneous Data Finite Time Convergence Stochastic Learning Numerical Aggregation Super Learner

Stochastic Learning

Papers

ROSS:RObust decentralized Stochastic learning based on Shapley values

Just How Flexible are Neural Networks in Practice?

Generalization of Hamiltonian algorithms

Stochastic Learning of Computational Resource Usage as Graph Structured Multimarginal Schr\"odinger Bridge

Estimating the Hessian Matrix of Ranking Objectives for Stochastic Learning to Rank with Gradient Boosted Trees

Preconditioners for the Stochastic Training of Implicit Neural Representations

A Generic Stochastic Hybrid Car-following Model Based on Approximate Bayesian Computation

Applications of the Theory of Aggregated Markov Processes in Stochastic Learning Theory

Impact of Redundancy on Resilience in Distributed Optimization and Learning

Simultaneously Learning Stochastic and Adversarial Bandits under the Position-Based Model

Simultaneously Learning Stochastic and Adversarial Bandits with General Graph Feedback

ClusterEA: Scalable Entity Alignment with Stochastic Training and Normalized Mini-batch Similarities

Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms

Stochastic Learning Equation using Monotone Increasing Resolution of Quantization

Federated Learning with Superquantile Aggregation for Heterogeneous Data