L*-Based Learning of Markov Decision Processes

Martin Tappler, Bernhard Aichernig, Giovanni Bacci, Maria Eichlseder, Kim Guldstrand Larsen

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem Konferenzband


Automata learning techniques automatically generate system models from test observations. These techniques usually fall into two categories: passive and active. Passive learning uses a predetermined data set, e.g., system logs. In contrast, active learning actively queries the system under learning, which is considered more efficient.

An influential active learning technique is Angluin’s L∗
algorithm for regular languages which inspired several generalisations from DFAs to other automata-based modelling formalisms. In this work, we study L∗-based learning of deterministic Markov decision processes, first assuming an ideal setting with perfect information. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling system traces via testing. Experiments with the implementation of our sampling-based algorithm suggest that it achieves better accuracy than state-of-the-art passive learning techniques with the same amount of test data. Unlike existing learning algorithms with predefined states, our algorithm learns the complete model structure including the states.
TitelFormal Methods - The Next 30 Years
Redakteure/-innenMaurice H. ter Beek, Annabelle McIver, José N. Oliveria
Herausgeber (Verlag)Springer
Seiten651 - 669
ISBN (elektronisch)978-3-030-30942-8
ISBN (Print)978-3-030-30941-1
PublikationsstatusVeröffentlicht - 2019
Veranstaltung2019 International Symposium on Formal Methods - Porto, Portugal
Dauer: 7 Okt 201911 Okt 2019


NameLecture Notes in Computer Science


Konferenz2019 International Symposium on Formal Methods
KurztitelFM 2019

Fields of Expertise

  • Information, Communication & Computing

Fingerprint Untersuchen Sie die Forschungsthemen von „L*-Based Learning of Markov Decision Processes“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren