Model Selection

Model selection aims to identify the optimal machine learning model for a given task from a potentially vast pool of candidates, balancing performance with resource efficiency. Current research emphasizes efficient algorithms for selecting models, including those leveraging transfer learning, ensembling techniques (like top-k union), and meta-learning approaches to adapt to diverse data distributions and changing objectives. Effective model selection is crucial for improving the accuracy and reliability of machine learning applications across various domains, from natural language processing and computer vision to resource-constrained edge devices and cybersecurity, while also mitigating the environmental impact of computationally intensive model training and deployment.

Papers

May 23, 2023

Clustering Indices based Automatic Classification Model Selection
Sudarsun Santhiappan, Nitin Shravan, Balaraman Ravindran
Model Selection Classification Datasets

May 14, 2023

Unraveling Cold Start Enigmas in Predictive Analytics for OTT Media: Synergistic Meta-Insights and Multimodal Ensemble Mastery
K. Ganguly, A. Patra
Predictive Model Model Selection Prediction Accuracy Ensemble Approach Synergistic Information Cold Start Novel Over the Air Second Multi Model Ensemble

May 3, 2023

fairml: A Statistician's Take on Fair Machine Learning Modelling
Marco Scutari
Model Selection Parameter Estimation Fair Machine Learning Fair Model Statistical Task

April 29, 2023

Limits of Model Selection under Transfer Learning
Steve Hanneke, Samory Kpotufe, Yasaman Mahdaviyeh
Domain Adaptation Transfer Learning Continuum Limit Model Selection Hyperparameter Tuning Hypothesis Class

April 2, 2023

A principled approach to model validation in domain generalization
Boyang Lyu, Thuan Nguyen, Matthias Scheutz, Prakash Ishwar, Shuchin Aeron
Model Selection Design Principle Domain Discrepancy Model Validation Generalization Technique Domain Generalization Datasets

March 15, 2023

Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation
Diego Marcondes, Cláudia Peixoto
Machine Learning LeArning Abstract Domain Knowledge Model Selection Cross Validation Statistical Learning Theory Conformal Risk Control

March 10, 2023

Machine learning for sports betting: should model selection be based on accuracy or calibration?
Conor Walsh, Alok Joshi
New Machine Predictive Model Calibration Performance Model Selection Sport Betting

March 5, 2023

Ensemble Reinforcement Learning: A Survey
Yanjie Song, P. N. Suganthan, Witold Pedrycz, Junwei Ou, Yongming He, Yingwu Chen, Yutong Wu
Reinforcement Learning Timely Survey Ensemble Learning Model Selection Ensemble Deep Reinforcement Learning

March 3, 2023

Online simulator-based experimental design for cognitive model selection
Alexander Aushev, Aini Putkonen, Gregoire Clarte, Suyog Chandramouli, Luigi Acerbi, Samuel Kaski, Andrew Howes
Model Selection Cognitive Science Online Service Computational Model Cognitive Model Cognitive Modeling Mind Task

February 20, 2023

Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift
Kaizheng Wang
Model Selection Covariate Shift Kernel Ridge Regression Imputation Model Regression Function

February 11, 2023

Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP
Xudong Han, Timothy Baldwin, Trevor Cohn
Procedural Fairness NLP Field Model Selection Fairness Research Fair Learning Fairness Property

February 6, 2023

In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation
Alicia Curth, Mihaela van der Schaar
Line by Line Explanation DCU Insight AQ Model Selection Treatment Effect Linear Estimator Heterogeneous Treatment Effect Estimation

December 28, 2022

Choosing the Number of Topics in LDA Models -- A Monte Carlo Comparison of Selection Criteria
Victor Bystrov, Viktoriia Naboka, Anna Staszewska-Bystrova, Peter Winker
Model Selection Numerical Data Monte Carlo Statistical Model Significant Topic Information Criterion Linear Discriminant Analysis

November 20, 2022

MEESO: A Multi-objective End-to-End Self-Optimized Approach for Automatically Building Deep Learning Models
Thanh Phuong Pham
Deep Learning Deep Learning Model Multi Objective Model Selection Self Optimization

November 18, 2022

Understanding the double descent curve in Machine Learning
Luis Sa-Couto, Jose Miguel Ramos, Miguel Almeida, Andreas Wichert
Machine Learning Model Selection Double Descent Parameterized Model

November 4, 2022

Improving the Predictive Performances of $k$ Nearest Neighbors Learning by Efficient Variable Selection
Eddie Pei, Ernest Fokoue
Nearest Neighbor Model Selection Predictive Performance Sequential Selection

November 3, 2022

November 1, 2022

Differentiable Model Selection for Ensemble Learning
James Kotary, Vincenzo Di Vito, Ferdinando Fioretto
Ensemble Learning Model Selection Differentiable Model

Model Selection

Papers

Clustering Indices based Automatic Classification Model Selection

Unraveling Cold Start Enigmas in Predictive Analytics for OTT Media: Synergistic Meta-Insights and Multimodal Ensemble Mastery

fairml: A Statistician's Take on Fair Machine Learning Modelling

Limits of Model Selection under Transfer Learning

A principled approach to model validation in domain generalization

Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation

Machine learning for sports betting: should model selection be based on accuracy or calibration?

Ensemble Reinforcement Learning: A Survey

Online simulator-based experimental design for cognitive model selection

Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift

Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP

In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation

Choosing the Number of Topics in LDA Models -- A Monte Carlo Comparison of Selection Criteria

MEESO: A Multi-objective End-to-End Self-Optimized Approach for Automatically Building Deep Learning Models

Understanding the double descent curve in Machine Learning

Improving the Predictive Performances of $k$ Nearest Neighbors Learning by Efficient Variable Selection

Oracle Inequalities for Model Selection in Offline Reinforcement Learning

Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation

Toward Unsupervised Outlier Model Selection

Differentiable Model Selection for Ensemble Learning