A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation

Antonio Pepe, Gianpaolo F Trotta, Peter Mohr-Ziak, Christina Gsaxner, Jürgen Wallner, Vitoantonio Bevilacqua, Jan Egger

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Abstract

As of common routine in tumor resections, surgeons rely on local examinations of the removed tissues and on the swiftly made microscopy findings of the pathologist, which are based on intraoperatively taken tissue probes. This approach may imply an extended duration of the operation, increased effort for the medical staff, and longer occupancy of the operating room (OR). Mixed reality technologies, and particularly augmented reality, have already been applied in surgical scenarios with positive initial outcomes. Nonetheless, these methods have used manual or marker-based registration. In this work, we design an application for a marker-less registration of PET-CT information for a patient. The algorithm combines facial landmarks extracted from an RGB video stream, and the so-called Spatial-Mapping API provided by the HMD Microsoft HoloLens. The accuracy of the system is compared with a marker-based approach, and the opinions of field specialists have been collected during a demonstration. A survey based on the standard ISO-9241/110 has been designed for this purpose. The measurements show an average positioning error along the three axes of (x, y, z) = (3.3 ± 2.3, − 4.5 ± 2.9, − 9.3 ± 6.1) mm. Compared with the marker-based approach, this shows an increment of the positioning error of approx. 3 mm along two dimensions (x, y), which might be due to the absence of explicit markers. The application has been positively evaluated by the specialists; they have shown interest in continued further work and contributed to the development process with constructive criticism.
Originalspracheenglisch
Seiten (von - bis)1008-1018
Seitenumfang11
FachzeitschriftJournal of digital imaging
Jahrgang32
Ausgabenummer6
DOIs
PublikationsstatusVeröffentlicht - 4 Sep 2019

Fingerprint

Oral Surgery
Surgery
Tissue
Helmet mounted displays
Operating rooms
Augmented reality
Medical Staff
Operating Rooms
Application programming interfaces (API)
Tumors
Microscopy
Microscopic examination
Demonstrations
Technology
Neoplasms
Surgeons
Surveys and Questionnaires
Pathologists
Positron Emission Tomography Computed Tomography

Dies zitieren

A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation. / Pepe, Antonio; Trotta, Gianpaolo F; Mohr-Ziak, Peter; Gsaxner, Christina; Wallner, Jürgen; Bevilacqua, Vitoantonio; Egger, Jan.

in: Journal of digital imaging, Jahrgang 32, Nr. 6, 04.09.2019, S. 1008-1018.

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Pepe, Antonio ; Trotta, Gianpaolo F ; Mohr-Ziak, Peter ; Gsaxner, Christina ; Wallner, Jürgen ; Bevilacqua, Vitoantonio ; Egger, Jan. / A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation. in: Journal of digital imaging. 2019 ; Jahrgang 32, Nr. 6. S. 1008-1018.
@article{0033c5b9e45448a9a467f3da83127913,
title = "A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation",
abstract = "As of common routine in tumor resections, surgeons rely on local examinations of the removed tissues and on the swiftly made microscopy findings of the pathologist, which are based on intraoperatively taken tissue probes. This approach may imply an extended duration of the operation, increased effort for the medical staff, and longer occupancy of the operating room (OR). Mixed reality technologies, and particularly augmented reality, have already been applied in surgical scenarios with positive initial outcomes. Nonetheless, these methods have used manual or marker-based registration. In this work, we design an application for a marker-less registration of PET-CT information for a patient. The algorithm combines facial landmarks extracted from an RGB video stream, and the so-called Spatial-Mapping API provided by the HMD Microsoft HoloLens. The accuracy of the system is compared with a marker-based approach, and the opinions of field specialists have been collected during a demonstration. A survey based on the standard ISO-9241/110 has been designed for this purpose. The measurements show an average positioning error along the three axes of (x, y, z) = (3.3 ± 2.3, − 4.5 ± 2.9, − 9.3 ± 6.1) mm. Compared with the marker-based approach, this shows an increment of the positioning error of approx. 3 mm along two dimensions (x, y), which might be due to the absence of explicit markers. The application has been positively evaluated by the specialists; they have shown interest in continued further work and contributed to the development process with constructive criticism.",
author = "Antonio Pepe and Trotta, {Gianpaolo F} and Peter Mohr-Ziak and Christina Gsaxner and J{\"u}rgen Wallner and Vitoantonio Bevilacqua and Jan Egger",
year = "2019",
month = "9",
day = "4",
doi = "10.1007/s10278-019-00272-6",
language = "English",
volume = "32",
pages = "1008--1018",
journal = "Journal of digital imaging",
issn = "0897-1889",
publisher = "Springer New York",
number = "6",

}

TY - JOUR

T1 - A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation

AU - Pepe, Antonio

AU - Trotta, Gianpaolo F

AU - Mohr-Ziak, Peter

AU - Gsaxner, Christina

AU - Wallner, Jürgen

AU - Bevilacqua, Vitoantonio

AU - Egger, Jan

PY - 2019/9/4

Y1 - 2019/9/4

N2 - As of common routine in tumor resections, surgeons rely on local examinations of the removed tissues and on the swiftly made microscopy findings of the pathologist, which are based on intraoperatively taken tissue probes. This approach may imply an extended duration of the operation, increased effort for the medical staff, and longer occupancy of the operating room (OR). Mixed reality technologies, and particularly augmented reality, have already been applied in surgical scenarios with positive initial outcomes. Nonetheless, these methods have used manual or marker-based registration. In this work, we design an application for a marker-less registration of PET-CT information for a patient. The algorithm combines facial landmarks extracted from an RGB video stream, and the so-called Spatial-Mapping API provided by the HMD Microsoft HoloLens. The accuracy of the system is compared with a marker-based approach, and the opinions of field specialists have been collected during a demonstration. A survey based on the standard ISO-9241/110 has been designed for this purpose. The measurements show an average positioning error along the three axes of (x, y, z) = (3.3 ± 2.3, − 4.5 ± 2.9, − 9.3 ± 6.1) mm. Compared with the marker-based approach, this shows an increment of the positioning error of approx. 3 mm along two dimensions (x, y), which might be due to the absence of explicit markers. The application has been positively evaluated by the specialists; they have shown interest in continued further work and contributed to the development process with constructive criticism.

AB - As of common routine in tumor resections, surgeons rely on local examinations of the removed tissues and on the swiftly made microscopy findings of the pathologist, which are based on intraoperatively taken tissue probes. This approach may imply an extended duration of the operation, increased effort for the medical staff, and longer occupancy of the operating room (OR). Mixed reality technologies, and particularly augmented reality, have already been applied in surgical scenarios with positive initial outcomes. Nonetheless, these methods have used manual or marker-based registration. In this work, we design an application for a marker-less registration of PET-CT information for a patient. The algorithm combines facial landmarks extracted from an RGB video stream, and the so-called Spatial-Mapping API provided by the HMD Microsoft HoloLens. The accuracy of the system is compared with a marker-based approach, and the opinions of field specialists have been collected during a demonstration. A survey based on the standard ISO-9241/110 has been designed for this purpose. The measurements show an average positioning error along the three axes of (x, y, z) = (3.3 ± 2.3, − 4.5 ± 2.9, − 9.3 ± 6.1) mm. Compared with the marker-based approach, this shows an increment of the positioning error of approx. 3 mm along two dimensions (x, y), which might be due to the absence of explicit markers. The application has been positively evaluated by the specialists; they have shown interest in continued further work and contributed to the development process with constructive criticism.

U2 - 10.1007/s10278-019-00272-6

DO - 10.1007/s10278-019-00272-6

M3 - Article

VL - 32

SP - 1008

EP - 1018

JO - Journal of digital imaging

JF - Journal of digital imaging

SN - 0897-1889

IS - 6

ER -