DNN Testing

Deep neural network (DNN) testing aims to identify vulnerabilities and ensure the reliability of these increasingly prevalent models. Current research focuses on developing effective testing methodologies that go beyond simple coverage metrics, emphasizing directed testing strategies that prioritize inputs likely to reveal errors, including those caused by adversarial attacks or backdoors. This work is crucial for improving the robustness and trustworthiness of DNNs across diverse applications, ranging from image recognition to safety-critical systems, by providing methods to identify and mitigate potential failures. Black-box testing approaches, which do not require access to the internal workings of the DNN, are gaining prominence due to their practicality and applicability to real-world scenarios.

Papers

March 25, 2024

DeepKnowledge: Generalisation-Driven Deep Learning Testing
Sondess Missaoui, Simos Gerasimou, Nikolaos Matragkas
Adversarial Attack Depth Information DNN Robustness Deep Learning Testing DNN Testing

May 25, 2023

Rethinking Diversity in Deep Neural Network Testing
Zi Wang, Jihye Choi, Ke Wang, Somesh Jha
Deep Neural Network Diversity Enhancement Diversity Metric Deep Learning Testing Diverse Test DNN Testing

March 8, 2023

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks
Zohreh Aghababaeyan, Manel Abdellatif, Mahboubeh Dadkhah, Lionel Briand
Deep Neural Network Model Retraining DNN Testing Test Selection

October 20, 2022

DNN-ForwardTesting: A New Trading Strategy Validation using Statistical Timeseries Analysis and Deep Neural Networks
Ivan Letteri, Giuseppe Della Penna, Giovanni De Gasperis, Abeer Dyoub
Neural Network Deep Neural Network Time Series Analysis Trading Strategy DNN Testing

March 24, 2022

NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks
Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, Yang Liu
Deep Neural Network Coverage Criterion Neuron Coverage Decision Logic DNN Testing Social NPC

December 24, 2021

CatchBackdoor: Backdoor Testing by Critical Trojan Neural Path Identification via Differential Fuzzing
Haibo Jin, Ruoxi Chen, Jinyin Chen, Yao Cheng, Chong Fu, Ting Wang, Yue Yu, Zhaoyan Ming
Backdoor Attack Fuzz Testing Backdoor Learning Neural Trojan DNN Testing

December 20, 2021

Black-Box Testing of Deep Neural Networks Through Test Case Diversity
Zohreh Aghababaeyan, Manel Abdellatif, Lionel Briand, Ramesh S, Mojtaba Bagherzadeh
Deep Neural Network Diversity Metric Box Testing Neuron Coverage Diverse Test DNN Testing

December 3, 2021

Provably Valid and Diverse Mutations of Real-World Media Data for DNN Testing
Yuanyuan Yuan, Qi Pang, Shuai Wang
High Dimensional Video Dataset Concurrent Validity Single Point Mutation DNN Testing