Evaluation at scale: An approach to evaluate technology for informal workplace learning across contexts

Stefan Thalmann, Tobias Ley, Ronald Maier, Tamsin Treasure-Jones, Christina Sarigianni, Markus Manhart

Research output: Contribution to journalArticle

Abstract

The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.

LanguageEnglish
Pages289-308
Number of pages20
JournalInternational journal of technology enhanced learning
Volume10
Issue number4
DOIs
StatusPublished - 1 Jan 2018

Fingerprint

workplace
evaluation
learning
large-scale research
informal learning
Planning
planning process
pluralism
research project
stakeholder
scenario
interpretation
User centered design

Keywords

  • Informal learning
  • Participatory design
  • Participatory evaluation
  • Workplace learning

ASJC Scopus subject areas

  • Education
  • Computer Science Applications

Cite this

Evaluation at scale : An approach to evaluate technology for informal workplace learning across contexts. / Thalmann, Stefan; Ley, Tobias; Maier, Ronald; Treasure-Jones, Tamsin; Sarigianni, Christina; Manhart, Markus.

In: International journal of technology enhanced learning, Vol. 10, No. 4, 01.01.2018, p. 289-308.

Research output: Contribution to journalArticle

Thalmann, Stefan ; Ley, Tobias ; Maier, Ronald ; Treasure-Jones, Tamsin ; Sarigianni, Christina ; Manhart, Markus. / Evaluation at scale : An approach to evaluate technology for informal workplace learning across contexts. In: International journal of technology enhanced learning. 2018 ; Vol. 10, No. 4. pp. 289-308
@article{7e19d112728245208b9f40e6ea603e24,
title = "Evaluation at scale: An approach to evaluate technology for informal workplace learning across contexts",
abstract = "The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.",
keywords = "Informal learning, Participatory design, Participatory evaluation, Workplace learning",
author = "Stefan Thalmann and Tobias Ley and Ronald Maier and Tamsin Treasure-Jones and Christina Sarigianni and Markus Manhart",
year = "2018",
month = "1",
day = "1",
doi = "10.1504/IJTEL.2018.095124",
language = "English",
volume = "10",
pages = "289--308",
journal = "International journal of technology enhanced learning",
issn = "1753-5255",
publisher = "Inderscience Publishers",
number = "4",

}

TY - JOUR

T1 - Evaluation at scale

T2 - International journal of technology enhanced learning

AU - Thalmann,Stefan

AU - Ley,Tobias

AU - Maier,Ronald

AU - Treasure-Jones,Tamsin

AU - Sarigianni,Christina

AU - Manhart,Markus

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.

AB - The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.

KW - Informal learning

KW - Participatory design

KW - Participatory evaluation

KW - Workplace learning

UR - http://www.scopus.com/inward/record.url?scp=85054978492&partnerID=8YFLogxK

U2 - 10.1504/IJTEL.2018.095124

DO - 10.1504/IJTEL.2018.095124

M3 - Article

VL - 10

SP - 289

EP - 308

JO - International journal of technology enhanced learning

JF - International journal of technology enhanced learning

SN - 1753-5255

IS - 4

ER -