Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge

Xiahai Zhuang, Lei Li, Christian Payer, Darko Stern, Martin Urschler, Mattias P. Heinrich, Julien Oster, Chunliang Wang, Örjan Smedby, Cheng Bian, Xin Yang, Pheng-Ann Heng, Aliasghar Mortazi, Ulas Bagci, Guanyu Yang, Chenchen Sun, Gaetan Galisot, Jean-Yves Ramel, Thierry Brouard, Qianqian Tong & 14 others Weixin Si, Xiangyun Liao, Guodong Zeng, Zenglin Shi, Guoyan Zheng, Chengjia Wang, Tom MacGillivray, David Newby, Kawal Rhode, Sebastian Ourselin, Raad Mohiaddin, Jennifer Keegan, David Firmin, Guang Yang

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be arduous due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, a set of training data is generally needed for constructing priors or for training. In addition, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets was limited. A number of them also reported poor results in the blinded evaluation, probably due to overfitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The …
Original languageEnglish
JournalarXiv.org e-Print archive
Publication statusPublished - 2019

Fingerprint

Magnetic resonance imaging
Image quality
Deep learning

Cite this

Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge. / Zhuang, Xiahai; Li, Lei; Payer, Christian; Stern, Darko; Urschler, Martin; P. Heinrich, Mattias; Oster, Julien; Wang, Chunliang; Smedby, Örjan; Bian, Cheng; Yang, Xin; Heng, Pheng-Ann; Mortazi, Aliasghar; Bagci, Ulas; Yang, Guanyu; Sun, Chenchen; Galisot, Gaetan; Ramel, Jean-Yves; Brouard, Thierry; Tong, Qianqian; Si, Weixin; Liao, Xiangyun; Zeng, Guodong; Shi, Zenglin; Zheng, Guoyan; Wang, Chengjia; MacGillivray, Tom; Newby, David; Rhode, Kawal; Ourselin, Sebastian; Mohiaddin, Raad; Keegan, Jennifer; Firmin, David; Yang, Guang.

In: arXiv.org e-Print archive, 2019.

Research output: Contribution to journalArticleResearchpeer-review

Zhuang, X, Li, L, Payer, C, Stern, D, Urschler, M, P. Heinrich, M, Oster, J, Wang, C, Smedby, Ö, Bian, C, Yang, X, Heng, P-A, Mortazi, A, Bagci, U, Yang, G, Sun, C, Galisot, G, Ramel, J-Y, Brouard, T, Tong, Q, Si, W, Liao, X, Zeng, G, Shi, Z, Zheng, G, Wang, C, MacGillivray, T, Newby, D, Rhode, K, Ourselin, S, Mohiaddin, R, Keegan, J, Firmin, D & Yang, G 2019, 'Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge' arXiv.org e-Print archive.
Zhuang, Xiahai ; Li, Lei ; Payer, Christian ; Stern, Darko ; Urschler, Martin ; P. Heinrich, Mattias ; Oster, Julien ; Wang, Chunliang ; Smedby, Örjan ; Bian, Cheng ; Yang, Xin ; Heng, Pheng-Ann ; Mortazi, Aliasghar ; Bagci, Ulas ; Yang, Guanyu ; Sun, Chenchen ; Galisot, Gaetan ; Ramel, Jean-Yves ; Brouard, Thierry ; Tong, Qianqian ; Si, Weixin ; Liao, Xiangyun ; Zeng, Guodong ; Shi, Zenglin ; Zheng, Guoyan ; Wang, Chengjia ; MacGillivray, Tom ; Newby, David ; Rhode, Kawal ; Ourselin, Sebastian ; Mohiaddin, Raad ; Keegan, Jennifer ; Firmin, David ; Yang, Guang. / Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge. In: arXiv.org e-Print archive. 2019.
@article{89a29018ae344f48832ba3b3598e060a,
title = "Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge",
abstract = "Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be arduous due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, a set of training data is generally needed for constructing priors or for training. In addition, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets was limited. A number of them also reported poor results in the blinded evaluation, probably due to overfitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The …",
author = "Xiahai Zhuang and Lei Li and Christian Payer and Darko Stern and Martin Urschler and {P. Heinrich}, Mattias and Julien Oster and Chunliang Wang and {\"O}rjan Smedby and Cheng Bian and Xin Yang and Pheng-Ann Heng and Aliasghar Mortazi and Ulas Bagci and Guanyu Yang and Chenchen Sun and Gaetan Galisot and Jean-Yves Ramel and Thierry Brouard and Qianqian Tong and Weixin Si and Xiangyun Liao and Guodong Zeng and Zenglin Shi and Guoyan Zheng and Chengjia Wang and Tom MacGillivray and David Newby and Kawal Rhode and Sebastian Ourselin and Raad Mohiaddin and Jennifer Keegan and David Firmin and Guang Yang",
year = "2019",
language = "English",
journal = "arXiv.org e-Print archive",
publisher = "Cornell University Library",

}

TY - JOUR

T1 - Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge

AU - Zhuang, Xiahai

AU - Li, Lei

AU - Payer, Christian

AU - Stern, Darko

AU - Urschler, Martin

AU - P. Heinrich, Mattias

AU - Oster, Julien

AU - Wang, Chunliang

AU - Smedby, Örjan

AU - Bian, Cheng

AU - Yang, Xin

AU - Heng, Pheng-Ann

AU - Mortazi, Aliasghar

AU - Bagci, Ulas

AU - Yang, Guanyu

AU - Sun, Chenchen

AU - Galisot, Gaetan

AU - Ramel, Jean-Yves

AU - Brouard, Thierry

AU - Tong, Qianqian

AU - Si, Weixin

AU - Liao, Xiangyun

AU - Zeng, Guodong

AU - Shi, Zenglin

AU - Zheng, Guoyan

AU - Wang, Chengjia

AU - MacGillivray, Tom

AU - Newby, David

AU - Rhode, Kawal

AU - Ourselin, Sebastian

AU - Mohiaddin, Raad

AU - Keegan, Jennifer

AU - Firmin, David

AU - Yang, Guang

PY - 2019

Y1 - 2019

N2 - Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be arduous due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, a set of training data is generally needed for constructing priors or for training. In addition, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets was limited. A number of them also reported poor results in the blinded evaluation, probably due to overfitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The …

AB - Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be arduous due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, a set of training data is generally needed for constructing priors or for training. In addition, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets was limited. A number of them also reported poor results in the blinded evaluation, probably due to overfitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The …

M3 - Article

JO - arXiv.org e-Print archive

JF - arXiv.org e-Print archive

ER -