Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test

Hermann Felbinger, Franz Wotawa, Mihai Nica

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016
PublisherInstitute of Electrical and Electronics Engineers
Pages23-30
Number of pages8
ISBN (Electronic)9781509037131
DOIs
Publication statusPublished - 21 Sep 2016
Event2nd IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016 - Vienna, Austria
Duration: 1 Aug 20163 Aug 2016

Conference

Conference2nd IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016
CountryAustria
CityVienna
Period1/08/163/08/16

Fingerprint

Software testing
Fault detection
Redundancy

Keywords

  • Coverage
  • Machine learning
  • Mutation Score
  • Redundancy
  • Software testing

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Cite this

Felbinger, H., Wotawa, F., & Nica, M. (2016). Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. In Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016 (pp. 23-30). [7573720] Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/QRS-C.2016.8

Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. / Felbinger, Hermann; Wotawa, Franz; Nica, Mihai.

Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers, 2016. p. 23-30 7573720.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Felbinger, H, Wotawa, F & Nica, M 2016, Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. in Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016., 7573720, Institute of Electrical and Electronics Engineers, pp. 23-30, 2nd IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016, Vienna, Austria, 1/08/16. https://doi.org/10.1109/QRS-C.2016.8
Felbinger H, Wotawa F, Nica M. Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. In Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers. 2016. p. 23-30. 7573720 https://doi.org/10.1109/QRS-C.2016.8
Felbinger, Hermann ; Wotawa, Franz ; Nica, Mihai. / Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers, 2016. pp. 23-30
@inproceedings{7c99730294124464a57e4f18efea15cb,
title = "Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test",
abstract = "Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99{\%} are possible while still keeping coverage and mutation score almost the same.",
keywords = "Coverage, Machine learning, Mutation Score, Redundancy, Software testing",
author = "Hermann Felbinger and Franz Wotawa and Mihai Nica",
year = "2016",
month = "9",
day = "21",
doi = "10.1109/QRS-C.2016.8",
language = "English",
pages = "23--30",
booktitle = "Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016",
publisher = "Institute of Electrical and Electronics Engineers",
address = "United States",

}

TY - GEN

T1 - Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test

AU - Felbinger, Hermann

AU - Wotawa, Franz

AU - Nica, Mihai

PY - 2016/9/21

Y1 - 2016/9/21

N2 - Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

AB - Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

KW - Coverage

KW - Machine learning

KW - Mutation Score

KW - Redundancy

KW - Software testing

UR - http://www.scopus.com/inward/record.url?scp=84991824134&partnerID=8YFLogxK

U2 - 10.1109/QRS-C.2016.8

DO - 10.1109/QRS-C.2016.8

M3 - Conference contribution

SP - 23

EP - 30

BT - Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016

PB - Institute of Electrical and Electronics Engineers

ER -