Robustness Measurement

Robustness measurement in machine learning aims to quantify a model's resistance to various perturbations, ensuring reliable performance in real-world scenarios. Current research focuses on developing both local and global robustness metrics, employing techniques like probabilistic verification, contrast set analysis, and comparisons against foundation models to assess model consistency and accuracy under diverse conditions. These efforts are crucial for improving the reliability and trustworthiness of machine learning systems across applications, particularly in high-stakes domains where model failures can have significant consequences. The development of robust models is increasingly important for building reliable and trustworthy AI systems.

Papers

November 14, 2024

BEARD: Benchmarking the Adversarial Robustness for Dataset Distillation
Zheng Zhou, Wenquan Feng, Shuchang Lyu, Guangliang Cheng, Xiaowei Huang, Qi Zhao
Adversarial Attack Adversarial Robustness Dataset Distillation Adversarial Game Robustness Measurement Facial Hair

October 24, 2024

Complexity Matters: Effective Dimensionality as a Measure for Adversarial Robustness
David Khachaturov, Robert Mullins
Native Robustness Adversarial Robustness Complexity Matter Robustness Evaluation Effective Dimension Robustness Measurement

May 31, 2024

Certifying Global Robustness for Deep Neural Networks
You Li, Guannan Zhao, Shuyu Kong, Yunqi He, Hai Zhou
Deep Neural Network Robust Deep Global Robustness Local Robustness Robustness Measurement

October 20, 2023

How Much Consistency Is Your Accuracy Worth?
Jacob K. Johnson, Ana Marasović
Strong Consistency Accuracy Measurement Contrast Set Robustness Measurement

August 21, 2023

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Peiyan Zhang, Haoyang Liu, Chaozhuo Li, Xing Xie, Sunghun Kim, Haohan Wang
Native Robustness Image Datasets Flawed Foundation Robust Model Robustness Evaluation Robustness Measurement

September 16, 2022

Model Predictive Robustness of Signal Temporal Logic Predicates
Yuanfei Lin, Haoxuan Li, Matthias Althoff
Native Robustness Signal Temporal Logic Robustness Measurement

July 11, 2022

How Robust is your Fair Model? Exploring the Robustness of Diverse Fairness Strategies
Edward Small, Wei Shao, Zeliang Zhang, Peihan Liu, Jeffrey Chan, Kacper Sokol, Flora Salim
Native Robustness Fairness Evaluation Fair Model Fairness Method Robustness Measurement Fairness Benchmark Datasets

June 27, 2022

Utilizing Class Separation Distance for the Evaluation of Corruption Robustness of Machine Learning Classifiers
Georg Siedel, Silvia Vock, Andrey Morozov, Stefan Voß
Global Evaluation Machine Learning Classifier Class Separability Corruption Robustness Classifier Robustness Training Robustness Robustness Measurement

January 20, 2022

Steerable Pyramid Transform Enables Robust Left Ventricle Quantification
Xiangyang Zhu, Kede Ma, Wufeng Xue
Ventricle Segmentation Left Ventricular Pyramid Vision Transformer Robust Feature Robust Convolutional Neural Network Robustness Measurement

January 11, 2022

Quantifying Robustness to Adversarial Word Substitutions
Yuting Yang, Pei Huang, FeiFei Ma, Juan Cao, Meishan Zhang, Jian Zhang, Jintao Li
Language Model Native Robustness Robustness Radius Robustness Measurement

December 5, 2021

Beyond Robustness: Resilience Verification of Tree-Based Classifiers
Stefano Calzavara, Lorenzo Cazzaro, Claudio Lucchese, Federico Marcuzzi, Salvatore Orlando
Native Robustness Adversarial Scenario Tree Based Resilience Measure Robustness Measurement