Rebo Junior: Analysis of Dialogue Structure Quality for a Reflection Guidance Chatbot

Irmtraud Maria Wolfbauer*, Viktoria Pammer-Schindler*, Carolyn P. Rosé*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Conversational user interfaces open up new opportunities for reflection guidance. This paper presents a computer-mediated dialogue structure for reflecting on learning tasks, Rebo Junior, and its evaluation in the context of apprenticeship training. We answer three research questions. Firstly, how apprentices react to Rebo Junior; secondly, whether Rebo Junior’s dialogue structure is
apt to lead apprentices in reflective conversations; and thirdly, how user engagement with Rebo Junior develops over time. During three months, 17 apprentices led 153 reflective conversations with Rebo Junior in the context of a training workshop, 117 in phase one and 36 in phase two of the study (five to thirteen
interactions per apprentice). We coded interactions manually for coherence, level of reflectivity, and user engagement. Our results show that apprentices react well to the intervention and that the dialogue structure is successful in leading apprentices through different levels of reflection (114 out of 153 showed observable reflection on the learning experience; 133 out of 153 expressed learning or planned behaviour change for future tasks). Furthermore, the interactions between the apprentices and Rebo Junior result in coherent conversations (149 out of 153 were coherent). Contrary to expectations, engagement did not decrease
over time in either phase. With the present paper, we therefore publish a dialogue structure for reflecting on learning tasks that has worked extremely well despite no adaptivity in the conversational interface. Overall, we interpret the results of
our work as underscoring the importance of dialogue structure quality in conversational agents.

Original languageEnglish
Title of host publicationECTEL 2020 Impact Paper Proceedings
PublisherCEUR Workshop Proceedings
Number of pages14
Volume2676
Publication statusPublished - 1 Jan 2020
Event15th European Conference on Technology-Enhanced Learning: EC-TEL-Impact 2020 - Virtuell, Heidelberg, Germany
Duration: 14 Sep 202018 Sep 2020
Conference number: 15
http://www.ec-tel.eu/index.php?id=957

Publication series

NameCEUR Workshop Proceedings
PublisherRWTH Aachen
ISSN (Print)1613-0073

Conference

Conference15th European Conference on Technology-Enhanced Learning
Abbreviated titleECTEL 2020
CountryGermany
CityVirtuell, Heidelberg
Period14/09/2018/09/20
Internet address

Keywords

  • reflection guidance
  • conversational agent
  • dialogue design
  • field study
  • Reflection guidance chatbot
  • Dialogue structure
  • Reflection guidance
  • Learning technology
  • Levels of reflection
  • Proof of concept evaluation

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Rebo Junior: Analysis of Dialogue Structure Quality for a Reflection Guidance Chatbot'. Together they form a unique fingerprint.

Cite this