DescriptionTesting AI-based systems and especially when they rely on machine learning is considered a challenging task. In this paper, we contribute to this challenge considering testing neural networks utilizing mutation testing. A former paper focused on applying mutation testing to the configuration of neural networks leading to the conclusion that mutation testing can be effectively used. In this paper, we discuss a substantially extended empirical evaluation where we considered different test data and the source code of neural network implementations. In particular, we discuss whether a mutated neural network can be distinguished from the original one after learning, only considering a test evaluation. Unfortunately, this is rarely the case leading to a low mutation score. As a consequence, we see that the testing method, which works well at the configuration level of a neural network, is not sufficient to test neural network libraries requiring substantially more testing effort for assuring quality.
|Period||11 Dec 2020|
|Event title||20th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2020|
|Location||Virtual, Macau, China|
Documents & Links
Activity: Participation in or organisation of › Conference or symposium (Participation in/Organisation of)
Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review