Bringing any Medical 3D Scan onto Augmented Reality Hardware - A Streamlined Data Pipeline

Paul Ganster, Maximilian Weber, Daniel Wild, Christina Gsaxner, Natalie Schellnegger, Jan Egger*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


This contribution presents a streamlined data pipeline to bring medical 3D scans onto Augmented Reality (AR) hardware. When a 3D scan is visualized on a 2D screen, depth information is lost and doctors have to rely on their experience to map the displayed data to the patient. Showing such a scan in AR addresses this problem, as one can view that scan in real 3D. To achieve this, the scan produced by a medical scanner has to be preprocessed by the user and brought onto the AR hardware. Usually, many manual steps are involved in achieving this, which require technical knowledge about the underlying software and hardware components and impede acceptance of this new technology by the target group, medical personnel. This work presents a streamlined pipeline for this process, leading to an enhanced user experience. The core component of the pipeline is a web application, to which a user can upload the direct output of a medical scanner. The scan can be interactively segmented by the user, after which both the scan and segment are stored on a server. Additionally, this paper introduces an AR application, which can be used to browse through patients and view their scans and previously created segments. We evaluate our streamlined data pipeline and AR application in a user study, reporting the results of a system usability questionnaire and a Thinking Aloud test.

Original languageEnglish
Title of host publicationMedical Imaging 2022
Subtitle of host publicationImaging Informatics for Healthcare, Research, and Applications
EditorsThomas M. Deserno, Thomas M. Deserno, Brian J. Park
ISBN (Electronic)9781510649491
Publication statusPublished - 2022
EventMedical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications - Virtual, Online
Duration: 21 Mar 202227 Mar 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications
CityVirtual, Online


  • Augmented Reality
  • Cloud
  • CT
  • DiCOM
  • User Study
  • Workflow

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Bringing any Medical 3D Scan onto Augmented Reality Hardware - A Streamlined Data Pipeline'. Together they form a unique fingerprint.

Cite this