Model Centric Evaluation

Model-centric evaluation focuses on assessing machine learning models' performance and robustness independent of specific datasets, aiming to provide more generalizable insights into model capabilities. Current research emphasizes developing frameworks that evaluate models across a wider range of inputs, including those beyond typical test sets, and analyzing output distributions to identify weaknesses. This approach is particularly relevant for improving model reliability and safety, as well as facilitating more objective comparisons between different model architectures, such as neural networks and tree-based models, leading to better model design and deployment.

Papers

July 2, 2024

A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data
Andrej Tschalzev, Sascha Marton, Stefan Lüdtke, Christian Bartelt, Heiner Stuckenschmidt
Machine Learning Tabular Data Data Centric Feature Engineering Model Centric Evaluation

December 6, 2023

Evaluation of human-model prediction difference on the Internet Scale of Data
Weitang Liu, Ying Wai Li, Yuelei Li, Zihan Wang, Yi-Zhuang You, Jingbo Shang
Machine Learning Model Precision Recall Curve Output Distribution Model Quality Model Centric Evaluation

May 29, 2023

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
Adversarial Attack Robustness Evaluation Robustness Evaluation Framework Model Centric Evaluation

Model Centric Evaluation

Papers

A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data

Evaluation of human-model prediction difference on the Internet Scale of Data

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework