Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test

Hermann Felbinger, Franz Wotawa, Mihai Nica

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Abstract

Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

Originalspracheenglisch
TitelProceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016
Herausgeber (Verlag)Institute of Electrical and Electronics Engineers
Seiten23-30
Seitenumfang8
ISBN (elektronisch)9781509037131
DOIs
PublikationsstatusVeröffentlicht - 21 Sep 2016
Veranstaltung2nd IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016 - Vienna, Österreich
Dauer: 1 Aug 20163 Aug 2016

Konferenz

Konferenz2nd IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016
LandÖsterreich
OrtVienna
Zeitraum1/08/163/08/16

Fingerprint

Software testing
Fault detection
Redundancy

Schlagwörter

    ASJC Scopus subject areas

    • Software
    • !!Safety, Risk, Reliability and Quality

    Dies zitieren

    Felbinger, H., Wotawa, F., & Nica, M. (2016). Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. in Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016 (S. 23-30). [7573720] Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/QRS-C.2016.8

    Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. / Felbinger, Hermann; Wotawa, Franz; Nica, Mihai.

    Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers, 2016. S. 23-30 7573720.

    Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

    Felbinger, H, Wotawa, F & Nica, M 2016, Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. in Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016., 7573720, Institute of Electrical and Electronics Engineers, S. 23-30, Vienna, Österreich, 1/08/16. https://doi.org/10.1109/QRS-C.2016.8
    Felbinger H, Wotawa F, Nica M. Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. in Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers. 2016. S. 23-30. 7573720 https://doi.org/10.1109/QRS-C.2016.8
    Felbinger, Hermann ; Wotawa, Franz ; Nica, Mihai. / Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test. Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016. Institute of Electrical and Electronics Engineers, 2016. S. 23-30
    @inproceedings{7c99730294124464a57e4f18efea15cb,
    title = "Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test",
    abstract = "Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99{\%} are possible while still keeping coverage and mutation score almost the same.",
    keywords = "Coverage, Machine learning, Mutation Score, Redundancy, Software testing",
    author = "Hermann Felbinger and Franz Wotawa and Mihai Nica",
    year = "2016",
    month = "9",
    day = "21",
    doi = "10.1109/QRS-C.2016.8",
    language = "English",
    pages = "23--30",
    booktitle = "Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016",
    publisher = "Institute of Electrical and Electronics Engineers",
    address = "United States",

    }

    TY - GEN

    T1 - Test-Suite Reduction Does Not Necessarily Require Executing the Program under Test

    AU - Felbinger, Hermann

    AU - Wotawa, Franz

    AU - Nica, Mihai

    PY - 2016/9/21

    Y1 - 2016/9/21

    N2 - Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

    AB - Removing redundancies from test-suites is an important task of software testing in order to keep test-suites as small as possible, but not to harm the test-suite's fault detection capabilities. A straightforward algorithm for test-suite reduction would select elements of the test-suite randomly and remove them if and only if the reduced test-suite fulfills the same or similar coverage or mutation score. Such algorithms rely on the execution of the program and the repeated computation of coverage or mutation score. In this paper, we present an alternative approach that purely relies on a model learned from the original test-suite without requiring the execution of the program under test. The idea is to remove those tests that do not change the learned model. In order to evaluate the approach we carried out an experimental study showing that reductions of 60-99% are possible while still keeping coverage and mutation score almost the same.

    KW - Coverage

    KW - Machine learning

    KW - Mutation Score

    KW - Redundancy

    KW - Software testing

    UR - http://www.scopus.com/inward/record.url?scp=84991824134&partnerID=8YFLogxK

    U2 - 10.1109/QRS-C.2016.8

    DO - 10.1109/QRS-C.2016.8

    M3 - Conference contribution

    SP - 23

    EP - 30

    BT - Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security-Companion, QRS-C 2016

    PB - Institute of Electrical and Electronics Engineers

    ER -