Performance Comparison of Two Search-Based Testing Strategies for ADAS System Validation

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

In this paper, we compare the performance of a genetic algorithm for test parameter optimization with simulated annealing and random testing. Simulated annealing and genetic algorithm both represent search-based testing strategies. In the context of autonomous and automated driving, we apply these methods to iteratively optimize test parameters, to aim at obtaining critical scenarios that form the basis for virtual verification and validation of Advanced Driver Assistant System (ADAS). We consider a test scenario to be critical if the underlying parameter set causes a malfunction of the system equipped with the ADAS function (i.e., near-crash or crash of the vehicle). To assess the criticality of each test scenario we rely on time-to-collision (TTC), which is a well-known and often used time-based safety indicator for recognizing rear-end conflicts. For evaluating the performance of each testing strategy, we set up a simulation framework, where we automatically run simulations for each approach until a predefined minimal TTC threshold is reached or a maximal number of iterations has passed. The genetic algorithm-based approach showed the best performance by generating critical scenarios with the lowest number of required test executions, compared to random testing and simulated annealing.

Original languageEnglish
Title of host publicationTesting Software and Systems - 31st IFIP WG 6.1 International Conference, ICTSS 2019, Proceedings
EditorsChristophe Gaston, Nikolai Kosmatov, Pascale Le Gall
PublisherSpringer
Pages140-156
Number of pages17
ISBN (Print)9783030312794
DOIs
Publication statusPublished - 1 Jan 2019
Externally publishedYes
Event31st IFIP International Conference on Testing Software and Systems, ICTSS 2019 - Paris, France
Duration: 15 Oct 201917 Oct 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11812 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference31st IFIP International Conference on Testing Software and Systems, ICTSS 2019
CountryFrance
CityParis
Period15/10/1917/10/19

    Fingerprint

Keywords

  • Automatic testing
  • Autonomous vehicles
  • Genetic algorithm
  • Simulated annealing
  • System verification

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Klück, F., Zimmermann, M., Wotawa, F., & Nica, M. (2019). Performance Comparison of Two Search-Based Testing Strategies for ADAS System Validation. In C. Gaston, N. Kosmatov, & P. Le Gall (Eds.), Testing Software and Systems - 31st IFIP WG 6.1 International Conference, ICTSS 2019, Proceedings (pp. 140-156). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11812 LNCS). Springer. https://doi.org/10.1007/978-3-030-31280-0_9