Evaluation in Pilot-Based Research Projects: A Hierarchical Approach

Thomas Zefferer*

*Korrespondierende/r Autor/in für diese Arbeit

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem Konferenzband


The evaluation of project results is a crucial part of most research projects in the information-technology domain. In particular, this applies to projects that develop solutions for the public sector and test them by means of pilot applications. In these projects, involved stakeholders require meaningful evaluation results to derive lessons learned and to steer future research activities in the right directions. Today, most projects define and apply their own evaluation schemes. This yields evaluation results that are difficult to compare between projects. Sometimes, inconsistent evaluation schemes are even applied within a project. In these cases, even evaluation results of a single project lack comparability. In absence of coherent evaluation schemes, lessons learned from conducted evaluation processes cannot be aggregated to a coherent holistic picture and the overall gain of research projects remains limited. To address this issue, we propose and introduce an evaluation scheme for arbitrary pilot-based research projects targeting the public sector. The proposed scheme follows a hierarchical approach. Concretely, it organizes evaluation criteria on different layers of abstraction. Furthermore, the proposed scheme describes in detail necessary process steps to carry out evaluations using the defined criteria. This way, the proposed scheme enables in-depth evaluations of research projects and their pilot applications. At the same time, it assures that obtained evaluation results remain comparable anytime. The hierarchical evaluation scheme introduced in this article has been successfully applied to the international research project FutureTrust, demonstrating its practical applicability, and showing its potential also for future research projects.

TitelWeb Information Systems and Technologies - 14th International Conference, WEBIST 2018, Revised Selected Papers
Redakteure/-innenMaría José Escalona, Francisco Domínguez Mayo, Tim A. Majchrzak, Valérie Monfort
Herausgeber (Verlag)Springer
ISBN (Print)9783030353292
PublikationsstatusVeröffentlicht - 2020
Extern publiziertJa
Veranstaltung14th International Conference on Web Information Systems and Technologies, WEBIST 2018 - Seville, Spanien
Dauer: 18 Sep 201820 Sep 2018


NameLecture Notes in Business Information Processing
Band372 LNBIP
ISSN (Print)1865-1348
ISSN (elektronisch)1865-1356


Konferenz14th International Conference on Web Information Systems and Technologies, WEBIST 2018

ASJC Scopus subject areas

  • !!Management Information Systems
  • !!Control and Systems Engineering
  • !!Business and International Management
  • Information systems
  • !!Modelling and Simulation
  • !!Information Systems and Management

Fingerprint Untersuchen Sie die Forschungsthemen von „Evaluation in Pilot-Based Research Projects: A Hierarchical Approach“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren