Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Abstract

In the treatment of head and neck cancer, physicians can benefit from augmented reality in preparing and executing treatment. We present a system allowing a physician wearing an untethered augmented reality headset to see medical visualizations precisely overlaid onto the patient. Our main contribution is a strategy for markerless registration of 3D imaging to the patient’s face. We use a neural network to detect the face using the headset’s depth sensor and register it to computed tomography data. The face registration is seamlessly combined with the headset’s continuous self-localization. We report on registration error and compare our approach to an external, high-precision tracking system.
Originalspracheenglisch
TitelInternational Conference on Medical Image Computing and Computer-Assisted Intervention
Seiten236-244
PublikationsstatusVeröffentlicht - 2019

Fingerprint

Neck
Head
Physicians
Head and Neck Neoplasms
Tomography
Therapeutics

Dies zitieren

Gsaxner, C., Pepe, A., Schmalstieg, D., Egger, J., & Wallner, J. (2019). Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. in International Conference on Medical Image Computing and Computer-Assisted Intervention (S. 236-244)

Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. / Gsaxner, Christina; Pepe, Antonio; Schmalstieg, Dieter; Egger, Jan; Wallner, Jürgen.

International Conference on Medical Image Computing and Computer-Assisted Intervention. 2019. S. 236-244.

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Gsaxner, C, Pepe, A, Schmalstieg, D, Egger, J & Wallner, J 2019, Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. in International Conference on Medical Image Computing and Computer-Assisted Intervention. S. 236-244.
Gsaxner C, Pepe A, Schmalstieg D, Egger J, Wallner J. Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 2019. S. 236-244
Gsaxner, Christina ; Pepe, Antonio ; Schmalstieg, Dieter ; Egger, Jan ; Wallner, Jürgen. / Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2019. S. 236-244
@inproceedings{b0200fdcb7354baab7ff61a52641aaa4,
title = "Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery",
abstract = "In the treatment of head and neck cancer, physicians can benefit from augmented reality in preparing and executing treatment. We present a system allowing a physician wearing an untethered augmented reality headset to see medical visualizations precisely overlaid onto the patient. Our main contribution is a strategy for markerless registration of 3D imaging to the patient’s face. We use a neural network to detect the face using the headset’s depth sensor and register it to computed tomography data. The face registration is seamlessly combined with the headset’s continuous self-localization. We report on registration error and compare our approach to an external, high-precision tracking system.",
author = "Christina Gsaxner and Antonio Pepe and Dieter Schmalstieg and Jan Egger and J{\"u}rgen Wallner",
year = "2019",
language = "English",
pages = "236--244",
booktitle = "International Conference on Medical Image Computing and Computer-Assisted Intervention",

}

TY - GEN

T1 - Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery

AU - Gsaxner, Christina

AU - Pepe, Antonio

AU - Schmalstieg, Dieter

AU - Egger, Jan

AU - Wallner, Jürgen

PY - 2019

Y1 - 2019

N2 - In the treatment of head and neck cancer, physicians can benefit from augmented reality in preparing and executing treatment. We present a system allowing a physician wearing an untethered augmented reality headset to see medical visualizations precisely overlaid onto the patient. Our main contribution is a strategy for markerless registration of 3D imaging to the patient’s face. We use a neural network to detect the face using the headset’s depth sensor and register it to computed tomography data. The face registration is seamlessly combined with the headset’s continuous self-localization. We report on registration error and compare our approach to an external, high-precision tracking system.

AB - In the treatment of head and neck cancer, physicians can benefit from augmented reality in preparing and executing treatment. We present a system allowing a physician wearing an untethered augmented reality headset to see medical visualizations precisely overlaid onto the patient. Our main contribution is a strategy for markerless registration of 3D imaging to the patient’s face. We use a neural network to detect the face using the headset’s depth sensor and register it to computed tomography data. The face registration is seamlessly combined with the headset’s continuous self-localization. We report on registration error and compare our approach to an external, high-precision tracking system.

M3 - Conference contribution

SP - 236

EP - 244

BT - International Conference on Medical Image Computing and Computer-Assisted Intervention

ER -