Deep Learning Testing

Deep learning testing focuses on evaluating the robustness, reliability, and generalizability of deep neural networks (DNNs), aiming to identify vulnerabilities and ensure dependable performance in real-world applications. Current research emphasizes developing novel testing methodologies, including those leveraging generative models to explore input space boundaries, mutation-based techniques to enhance fault detection in large language models, and methods that analyze neuron sensitivity or feature maps to improve test case selection and efficiency. These advancements are crucial for increasing the trustworthiness of DNNs across diverse domains, from image classification and natural language processing to safety-critical systems like autonomous driving.

Papers

August 12, 2024

Deep Learning System Boundary Testing through Latent Space Style Mixing
Amr Abdellatif, Xingcheng Chen, Vincenzo Riccio, Andrea Stocco
Generative Artificial Intelligence Deep Learning Testing MIMICRY Change Optimisation Dynamic Style Based Generative Adversarial Network Style Feature Fusion Network

April 14, 2024

Evaluation and Improvement of Fault Detection for Large Language Models
Qiang Hu, Jin Wen, Maxime Cordy, Yuheng Huang, Wei Ma, Xiaofei Xie, Lei Ma
Large Language Model Fault Detection Open Source LLM Deep Learning Testing Test Selection

March 25, 2024

DeepKnowledge: Generalisation-Driven Deep Learning Testing
Sondess Missaoui, Simos Gerasimou, Nikolaos Matragkas
Adversarial Attack Depth Information Deep Learning Testing DNN Robustness DNN Testing

November 3, 2023

Simulation of acquisition shifts in T2 Flair MR images to stress test AI segmentation networks
Christiane Posselt, Mehmet Yigit Avci, Mehmet Yigitsoy, Patrick Schünke, Christoph Kolbitsch, Tobias Schäffter, Stefanie Remmele
Simulation Study Channel Acquisition Deep Learning Testing Multiple Sclerosis Lesion Segmentation Contrast Affecting Sequence Parameter F1 Score

September 11, 2023

Hazards in Deep Learning Testing: Prevalence, Impact and Recommendations
Salah Ghamizi, Maxime Cordy, Yuejun Guo, Mike Papadakis, And Yves Le Traon
Machine Learning Global Impact Movie Recommendation Sensitivity Analysis Prevalence Threshold Deep Learning Testing Empirical Risk Hazard Triggering Event

September 4, 2023

Hawkeye: Change-targeted Testing for Android Apps based on Deep Reinforcement Learning
Chao Peng, Zhengwei Lv, Jiarong Fu, Jiayuan Liang, Zhao Zhang, Ajitha Rajan, Ping Yang
Deep Reinforcement Learning Level Test Mobile Application Deep Learning Testing Based Testing Systematic Testing

July 21, 2023

Feature Map Testing for Deep Neural Networks
Dong Huang, Qingwen Bu, Yahao Qing, Yichao Fu, Heming Cui
Deep Neural Network Deep Feature Fault Injection Feature Mapping Deep Learning Testing

July 20, 2023

Neuron Sensitivity Guided Test Case Selection for Deep Learning Testing
Dong Huang, Qingwen Bu, Yichao Fu, Yuhao Qing, Bocheng Xiao, Heming Cui
Deep Neural Network DNN Framework Unlabeled Data DNN Model Deep Learning Testing

May 25, 2023

Rethinking Diversity in Deep Neural Network Testing
Zi Wang, Jihye Choi, Ke Wang, Somesh Jha
Deep Neural Network Diversity Enhancement Diversity Metric Deep Learning Testing Diverse Test DNN Testing

February 28, 2023

FacEDiM: A Face Embedding Distribution Model for Few-Shot Biometric Authentication of Cattle
Meshia Cédric Oveneke, Rucha Vaishampayan, Deogratias Lukamba Nsadisa, Jenny Ambukiyenyi Onya
Pre Trained Supervised ImageNet Human Face Biometric Identification Shot Recognition Learned Embeddings Deep Learning Testing Dairy Cattle

January 13, 2023

Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Florian Tambon, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Giuliano Antonio
Deep Reinforcement Learning Post Fault Trajectory Single Point Mutation Mutation Operator Deep Learning Testing Mutation Testing

December 21, 2022

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study
Vincenzo Riccio, Paolo Tonella
Deep Learning Empirical Study Data Generation Deep Learning Testing Test Data Generation Skilled Non Expert Validators

April 1, 2022

Testing Feedforward Neural Networks Training Programs
Houssem Ben Braiek, Foutse Khomh
Deep Neural Network DNN Framework DNN Model DNN Training Deep Learning Testing

February 24, 2022

Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques
Mohit Kumar Ahuja, Arnaud Gotlieb, Helge Spieker
Deep Learning Autonomous Driving Comparative Study Perception System Adversarial Testing Deep Learning Testing Vision Based System

February 12, 2022

Excitement Surfeited Turns to Errors: Deep Learning Testing Framework Based on Excitable Neurons
Haibo Jin, Ruoxi Chen, Haibin Zheng, Jinyin Chen, Yao Cheng, Yue Yu, Xianglong Liu
Deep Neural Network Single Neuron Level Error Feedback Adversarial Input Deep Image Deep Learning Testing Neuron Coverage

January 23, 2022

pvCNN: Privacy-Preserving and Verifiable Convolutional Neural Network Testing
Jiasi Weng, Jian Weng, Gui Tang, Anjia Yang, Ming Li, Jia-Nan Liu
Homomorphic Encryption Zero Knowledge Model Privacy Deep Learning Testing