Mutation Testing

Mutation testing assesses the effectiveness of test suites by introducing artificial faults into software or models, evaluating their detection rate. Current research focuses on adapting this technique to various machine learning domains, including large language models, chatbots, and deep neural networks deployed on hardware accelerators, investigating optimal mutation operators and evaluating the performance of different mutation-based model selection methods against established techniques like cross-validation. This work is significant for improving the reliability and trustworthiness of increasingly complex AI systems across diverse applications, ultimately contributing to more robust and dependable AI deployments.

Papers

October 31, 2024

Metamorphic Malware Evolution: The Potential and Peril of Large Language Models
Pooria Madani
Full Model Full Potential Large Language Malware Analysis Code Transformation Metamorphic Malware Mutation Testing

September 7, 2024

MILE: A Mutation Testing Framework of In-Context Learning Systems
Zeming Wei, Yihao Zhang, Meng Sun
Large Language Model Context Learning Context Demonstration New Embeddings Mutation Testing

January 18, 2024

MutaBot: A Mutation Testing Approach for Chatbots
Michael Ferdinando Urrico, Diego Clerissi, Leonardo Mariani
Chatbot Response Chatbot Model Mutation Testing

November 23, 2023

Empirical Comparison between Cross-Validation and Mutation-Validation in Model Selection
Jinyang Yu, Sami Hamdan, Leonard Sasse, Abigail Morrison, Kaustubh R. Patil
Model Selection Cross Validation Hyperparameter Selection Empirical Comparison Mutation Testing

June 2, 2023

MutateNN: Mutation Testing of Image Recognition Models Deployed on Hardware Accelerators
Nikolaos Louloudakis, Perry Gibson, José Cano, Ajitha Rajan
Deep Neural Network Hardware Accelerator Recognition Model Mutation Testing

January 25, 2023

BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testing
Jiali Wei, Ming Fan, Wenjing Jiao, Wuxia Jin, Ting Liu
Language Model Backdoor Attack Backdoor Detection Backdoor Sample Mutation Testing

January 13, 2023

Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Florian Tambon, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Giuliano Antonio
Deep Reinforcement Learning Post Fault Trajectory Single Point Mutation Mutation Operator Deep Learning Testing Mutation Testing

August 11, 2022

A Probabilistic Framework for Mutation Testing in Deep Neural Networks
Florian Tambon, Foutse Khomh, Giuliano Antoniol
Deep Neural Network Mutation Operator Box Testing Requirement Testability Mutation Testing