Bringing any Medical 3D Scan onto Augmented Reality Hardware - A Streamlined Data Pipeline

This contribution presents a streamlined data pipeline to bring medical 3D scans onto Augmented Reality (AR) hardware. When a 3D scan is visualized on a 2D screen, depth information is lost and doctors have to rely on their experience to map the displayed data to the patient. Showing such a scan in AR addresses this problem, as one can view that scan in real 3D. To achieve this, the scan produced by a medical scanner has to be preprocessed by the user and brought onto the AR hardware. Usually, many manual steps are involved in achieving this, which require technical knowledge about the underlying software and hardware components and impede acceptance of this new technology by the target group, medical personnel. This work presents a streamlined pipeline for this process, leading to an enhanced user experience. The core component of the pipeline is a web application, to which a user can upload the direct output of a medical scanner. The scan can be interactively segmented by the user, after which both the scan and segment are stored on a server. Additionally, this paper introduces an AR application, which can be used to browse through patients and view their scans and previously created segments. We evaluate our streamlined data pipeline and AR application in a user study, reporting the results of a system usability questionnaire and a Thinking Aloud test.