Benchmarking Combinations of Learning and Testing Algorithms for Active Automata Learning

Bernhard Aichernig, Martin Tappler*, Felix Wallner

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Active automata learning comprises techniques for learning automata models of black-box systems by testing such systems. While this form of learning enables model-based analysis and verification, it may also require a substantial amount of interactions with considered systems to learn adequate models, which capture the systems’ behaviour. The test cases executed during learning can be divided into two categories: (1) test cases to gain knowledge about a system and (2) test cases to falsify a learned hypothesis automaton. The former are selected by learning algorithms, whereas the latter are selected by conformance-testing algorithms. There exist various options for both types of algorithms and there are dependencies between them. In this paper, we investigate the performance of combinations of four different learning algorithms and seven different testing algorithms. For this purpose, we perform learning experiments using 39 benchmark models. Based on experimental results, we discuss insights regarding the performance of different configurations for various types of systems. These insights may serve as guidance for future users of active automata learning.

Original languageEnglish
Title of host publicationTests and Proofs - 14th International Conference, TAP 2020, held as part of STAF 2020, Proceedings
EditorsWolfgang Ahrendt, Heike Wehrheim
Pages3-22
Number of pages20
DOIs
Publication statusE-pub ahead of print - 20 Jul 2020
Event14th International Conference on Tests and Proofs: TAP 2020, held as part of Software Technologies: Applications and Foundations, STAF 2020 - Bergen, Norway
Duration: 22 Jun 202023 Jun 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12165 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th International Conference on Tests and Proofs
CountryNorway
CityBergen
Period22/06/2023/06/20

Keywords

  • Active automata learning
  • Conformance testing
  • LearnLib
  • Model learning
  • Model-based testing

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fields of Expertise

  • Information, Communication & Computing

Fingerprint Dive into the research topics of 'Benchmarking Combinations of Learning and Testing Algorithms for Active Automata Learning'. Together they form a unique fingerprint.

Cite this