Investigating the Effectiveness of Mutation Testing Tools in the Context of Deep Neural Networks

Nour Chetouane, Lorenz Klampfl, Franz Wotawa*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Verifying the correctness of the implementation of machine learning algorithms like neural networks has become a major topic because – for example – its increasing use in the context of safety critical systems like automated or autonomous vehicles. In contrast to evaluating the learning capabilities of such machine learning algorithms, in verification, and particularly in testing we are interested in finding critical scenarios and in giving some sort of guarantees with respect to the underlying used tests. In this paper, we contribute to the area of testing machine learning algorithms and investigate the effectiveness of traditional mutation tools in the context of Deep Neural Networks testing. In particular, we try to answer the question whether mutated neural networks can be identified considering their learning capabilities when compared to the original network. To answer this question, we performed an empirical study using Java code implementations of such networks and a mutation tool to create mutated neural networks models. As an outcome, we are able to identify some mutations to be more likely to be detected than others.

Original languageEnglish
Title of host publicationAdvances in Computational Intelligence - 15th International Work-Conference on Artificial Neural Networks, IWANN 2019, Proceedings
EditorsGonzalo Joya, Andreu Catala, Ignacio Rojas
PublisherSpringer, Cham
Pages766-777
Number of pages12
ISBN (Print)9783030205201
DOIs
Publication statusPublished - 16 May 2019
Event15th International Work-Conference on Artificial Neural Networks, IWANN 2019 - Gran Canaria, Spain
Duration: 12 Jun 201914 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number11506
VolumeLNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Work-Conference on Artificial Neural Networks, IWANN 2019
CountrySpain
CityGran Canaria
Period12/06/1914/06/19

Keywords

  • Mutation score for neural networks
  • Mutation testing
  • Testing neural networks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Investigating the Effectiveness of Mutation Testing Tools in the Context of Deep Neural Networks'. Together they form a unique fingerprint.

Cite this