Deep Learning

Follow this topic Ask about this topic

Momentum

58 papers in the last four weeks, up 115% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 682

All topics

Universality of Benign Overfitting in Binary Linear Classification
Jan 17, 2025Ichiro Hashimoto, Stanislav Volgushev, Piotr ZwiernikOverfitting Empirical Risk Minimization
Department of Statistical Sciences, University of Toronto
Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning
Jan 13, 2025Thomas ChenGradient Descent Wasserstein Gradient Flows
Network Dynamics-Based Framework for Understanding Deep Neural Networks
Jan 5, 2025Yuchen Lin, Yong Zhang, Sihan Feng +1Neural Network Deep Learning
Department of Physics, Xiamen University, Xiamen 361005, China · 2Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou 730000, China · 3Shenzhen Angel FOF Management Co., Ltd., Shenzhen 518000, China
SPARC: Scalable Path-Specific Counterfactual Fairness via Causal Conditional Independence
Dec 6, 2024Bowei Tian, Yexiao He, Ziyao Wang +3Algorithmic Fairness Causal Inferences
University of Maryland, College Park, MD 20742, USA · Clemson University, Clemson, SC 29634, USA
Contrastive Deep Learning Reveals Age Biomarkers in Histopathological Skin Biopsies
Nov 25, 2024Kaustubh Chakradeo, Pernille Nielsen, Lise Mette Rahbek Gjerdrum +5Biomarker Deep Learning
University of Copenhagen, Section of Epidemiology, Department of Public Health, Copenhagen, Denmark · University of Copenhagen, Section of Epidemiology, Department of Public Health, Copenhagen, 1353, Denmark · Technical University of Denmark, Department of Applied Mathematics and Computer Science, Denmark +7
AMT-APC: Automatic Piano Cover by Fine-Tuning an Automatic Music Transcription Model
Sep 21, 2024Kazuma Komiya, Yoshihisa FukuharaMulti-Instrument Music Transcription Deep Learning
Faculty of Data Science Musashino University Tokyo, Japan
DeepSPoC: A Deep Learning Based Sequential Propagation of Chaos
Aug 29, 2024Kai Du, Yongle Xie, Tao Zhou +1Fokker--Planck Equation Particle Dynamics
Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, 200438, China. · Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, 200433, China. · LSEC, Institute of Computational Mathematics and Scientific/Engineering Computing, AMSS, Chinese Academy of Sciences, Beijing, China.
Beyond Spectral Decomposition: Bayesian Contrastive Learning and its Non-negative Formulation via Factor Analysis
Jul 31, 2024Zhibin Duan, Tiansheng Wen, Yifei Wang +3Exploratory Factor Analysis Contrastive Learning
Xidian University · MIT CSAIL · The University of Texas at Austin
MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network
Jul 30, 2024Yinlong Xu, Zitai Kong, Yixuan Wu +5Arrhythmia Classification Electrocardiography
State Key Laboratory of Transvascular Implantation Devices and TIDRI, Zhejiang University, Hangzhou, China · School of Public Health, Zhejiang University and Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence, Hangzhu, China · State Key Laboratory of Transvascular Implantation Devices of The Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University, Hangzhou, China +3
NGD-SLAM: Towards Real-Time Dynamic SLAM without GPU
May 12, 2024Yuhao Zhang, Mihai Bujanca, Mikel LujánOrb-Slam2 Simultaneous Localization And Mapping
University of Manchester. 2University of Cambridge. · 3Qualcomm Technologies XR Labs, Austria.
Introduction to Transformers: an NLP Perspective
Nov 29, 2023Tong Xiao, Jingbo ZhuTransformer Architectures Natural Language Processing
NLP Lab., Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
Topology-enhanced machine learning for speech signal processing
Nov 26, 2023Pingyao Feng, Qingrui Qu, Haiyu Zhang +4Discrete Speech Representations Speaker
Department of Mathematics, Southern University of Science and Technology, Shenzhen, China. · Department of Mathematics, North Carolina State University, Raleigh, NC, USA. · School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen, China. +1
Data Market Design through Deep Learning
Oct 31, 2023Sai Srivatsa Ravindranath, Yanchen Jiang, David C. ParkesBayesian Experimental Design First-Price Auctions
Harvard John A. Paulson School of Engineering and Applied Sciences
Learning Non-Vacuous Generalization Bounds from Optimization
Jun 9, 2022Chengli Tan, Jiangshe Zhang, Junmin Liu +1Generalization Bounds Deep Learning
Northwestern Polytechnical University, Xi’an, China · Xi’an Jiaotong University, Xi’an, China · SGIT AI Lab, State Grid Corporation of China, Xi’an, China
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning
Sep 6, 2021Yehuda Dar, Vidya Muthukumar, Richard G. BaraniukOverparameterization Bias-Variance Decomposition
Faculty of Computer and Information Science, Ben-Gurion University · Schools of Electrical and Computer Engineering, and Industrial and Systems Engineering, Georgia Institute of Technology · Electrical and Computer Engineering Department, Rice University
Deep learning based numerical approximation algorithms for stochastic partial differential equations
Dec 2, 2020Christian Beck, Sebastian Becker, Patrick Cheridito +2Stochastic Differential Equations Neural Approximations
Department of Mathematics, ETH Zurich · School of Data Science and School of Artificial Intelligence,2025 The Chinese University of Hong Kong, Shenzhen (CUHK-Shenhen) · Applied Mathematics: Institute for Analysis and Numerics, University of Münster +1
Deep Learning for Individual Heterogeneity
Oct 28, 2020Max H. Farrell, Tengyuan Liang, Sanjog MisraHeteroskedasticity Heterogeneity
Department of Economics, UC Santa Barbara · Booth School of Business, University of Chicago
Hybrid Feature Learning for Handwriting Verification
Nov 19, 2018Seyed Mohammad Abuzar Hashemi, Mihir Chauhan, Jun Chu +1Handwriting Feature Learning
The Department of Computer Science and Engineering The State University of New York at Buffalo Buffalo, NY, USA
DNA: Differentially private Neural Augmentation for contact tracing
Date pendingRob Romijnders, Christos Louizos, Yuki M. Asano +1Covid-19 Δ)$-Differential Privacy
University of Amsterdam · Qualcomm AI research
A Dataset and Benchmarks for Atrial Fibrillation Detection from Electrocardiograms of Intensive Care Unit Patients
Date pendingSarah Nassar, Nooshin Maghsoodi, Sophia Mannina +7Arrhythmia Classification Electrocardiogram Foundation Models
Department of Electrical and Computer Engineering at Queen’s University, Kingston, ON, Canada · School of Computing at Queen’s University, Kingston, ON, Canada · Smith School of Business at Queen’s University, Kingston, ON, Canada +5
R $^2$ M: Real-Aware Residual Model Merging for Robust and Generalizable Deepfake Detection
Date pendingJinhee Park, Guisik Kim, Choongsang Cho +1Deepfake Detection Unsupervised Detection
Department of Computer Science and Engineering, Chung-Ang University, Korea · Korea Electronics Technology Institute (KETI), Korea
Deep learning methods for inverse problems using connections between proximal operators and Hamilton-Jacobi equations
Date pendingOluwatosin Akande, Gabriel P. Langlois, Akwum OnwuntaBayesian Inverse Problems Proximal Operators
Industrial and Systems Engineering, Lehigh University, 200 West Packer Avenue, Bethlehem, PA 18015, USA · Department of Mathematics, University of Illinois Urbana-Champaign, Chicago, IL, USA

Jan 17, 2025, cs.LGJ/K move · Enter open · S save

Universality of Benign Overfitting in Binary Linear Classification

Ichiro Hashimoto, Stanislav Volgushev, Piotr Zwiernik

Department of Statistical Sciences, University of Toronto

Overfitting Empirical Risk Minimization Deep Learning Noisy Data Mixture Models

Open paper AskPDF arXiv

The practical success of deep learning has led to the discovery of several surprising phenomena. One of these phenomena, that has spurred intense theoretical research, is ``benign overfitting'': deep neural networks seem to generalize well in the over-parametrized regime even though the networks show a perfect fit to noisy training data. It is now known that benign overfitting also occurs in various classical statistical models. For linear maximum margin classifiers, benign overfitting has been established theoretically in a class of mixture models with very strong assumptions on the covariate distribution. However, even in this simple setting, many questions remain open. For instance, most of the existing literature focuses on the noiseless case where all true class labels are observed without errors, whereas the more interesting noisy case remains poorly understood. We provide a comprehensive study of benign overfitting for linear maximum margin classifiers. We discover a phase transition in test error bounds for the noisy model which was previously unknown and provide some geometric intuition behind it. We further considerably relax the required covariate assumptions in both the noisy and noiseless cases. Our results demonstrate that benign overfitting of maximum margin classifiers holds in a much wider range of scenarios than was previously known and provide new insights into the underlying mechanisms.

Momentum

Related topics

Latest papers 682