TY - JOUR
T1 - Ontology-based metamorphic testing for chatbots
AU - Božić, Josip
N1 - Funding Information:
The research presented in the paper has been funded in part by the Austrian Research Promotion Agency (FFG). I want to thank the anonymous reviewers for their constructive feedback, which was addressed in the paper. In addition to that, I would like to express my gratitude to this journal’s special issue guest editors for their kind approach.
Publisher Copyright:
© 2021, The Author(s).
PY - 2021
Y1 - 2021
N2 - Modern-day demands for services often require an availability on a 24/7 basis as well as online accessibility around the globe. For this sake, personalized virtual assistants, called chatbots, are implemented. Such systems offer services, goods or information in natural language. These natural language processing (NLP) programs respond to the user in real time and offer an intuitive and simple interface to interact with. Advantages like these make them increasingly popular. Therefore, ensuring correct functionality of chatbots is of increasing importance. However, since different implementations and user behaviour result in unpredictable results, the chatbot’s input and output data are difficult to predict and classify as well. Under such circumstances, test cases can be inferred from the domain of possible inputs of a system under test (SUT). Ontologies are concepts used in AI to provide formal representations of knowledge for a specific domain. Such ontological models contain structured information that is used for test generation. On the other hand, testing of chatbots represents a challenge because of the absence of a test oracle. In this paper, both challenges are addressed by conceptualizing ontologies for input generation and output processing in form of a metamorphic testing approach. In this scenario, both concepts are applied for automated testing of chatbots. The approach is demonstrated on a real system from the tourism domain, thereby discussing the obtained results.
AB - Modern-day demands for services often require an availability on a 24/7 basis as well as online accessibility around the globe. For this sake, personalized virtual assistants, called chatbots, are implemented. Such systems offer services, goods or information in natural language. These natural language processing (NLP) programs respond to the user in real time and offer an intuitive and simple interface to interact with. Advantages like these make them increasingly popular. Therefore, ensuring correct functionality of chatbots is of increasing importance. However, since different implementations and user behaviour result in unpredictable results, the chatbot’s input and output data are difficult to predict and classify as well. Under such circumstances, test cases can be inferred from the domain of possible inputs of a system under test (SUT). Ontologies are concepts used in AI to provide formal representations of knowledge for a specific domain. Such ontological models contain structured information that is used for test generation. On the other hand, testing of chatbots represents a challenge because of the absence of a test oracle. In this paper, both challenges are addressed by conceptualizing ontologies for input generation and output processing in form of a metamorphic testing approach. In this scenario, both concepts are applied for automated testing of chatbots. The approach is demonstrated on a real system from the tourism domain, thereby discussing the obtained results.
KW - Chatbots
KW - Functional testing
KW - Metamorphic testing
KW - Ontologies
UR - http://www.scopus.com/inward/record.url?scp=85105364543&partnerID=8YFLogxK
U2 - 10.1007/s11219-020-09544-9
DO - 10.1007/s11219-020-09544-9
M3 - Article
AN - SCOPUS:85105364543
SN - 0963-9314
JO - Software Quality Journal
JF - Software Quality Journal
ER -