Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network

Savinien Bonheur, Darko Stern, Christian Payer, Michael Pienn, Horst Olschewski, Martin Urschler

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

Despite some design limitations, CNNs have been largely adopted by the computer vision community due to their efficacy and versatility. Introduced by Sabour et al. to circumvent some limitations of CNNs, capsules replace scalars with vectors to encode appearance feature representation, allowing better preservation of spatial relationships between whole objects and its parts. They also introduced the dynamic routing mechanism, which allows to weight the contributions of parts to a whole object differently at each inference step. Recently, Hinton et al. have proposed to solely encode pose information to model such part-whole relationships. Additionally, they used a matrix instead of a vector encoding in the capsules framework. In this work, we introduce several improvements to the capsules framework, allowing it to be applied for multi-label semantic segmentation. More specifically, we combine pose and appearance information encoded as matrices into a new type of capsule, i.e. Matwo-Caps. Additionally, we propose a novel routing mechanism, i.e. Dual Routing, which effectively combines these two kinds of information. We evaluate our resulting Matwo-CapsNet on the JSRT chest X-ray dataset by comparing it to SegCaps, a capsule based network for binary segmentation, as well as to other CNN based state-of-the-art segmentation methods, where we show that our Matwo-CapsNet achieves competitive results, while requiring only a fraction of the parameters of other previously proposed methods.
Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2019
Place of PublicationCham
PublisherSpringer
Pages664-672
ISBN (Electronic)978-3-030-32254-0
ISBN (Print)978-3-030-32253-3
DOIs
Publication statusPublished - 2019
EventMICCAI 2019: 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention - Shenzen, China
Duration: 13 Oct 201917 Nov 2019

Publication series

NameLecture Notes in Computer Science
Volume11768

Conference

ConferenceMICCAI 2019
CountryChina
CityShenzen
Period13/10/1917/11/19

Fingerprint

Labels
Semantics
Computer vision
X rays

Cite this

Bonheur, S., Stern, D., Payer, C., Pienn, M., Olschewski, H., & Urschler, M. (2019). Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 (pp. 664-672). (Lecture Notes in Computer Science; Vol. 11768). Cham: Springer. https://doi.org/10.1007/978-3-030-32254-0_74

Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network. / Bonheur, Savinien; Stern, Darko; Payer, Christian; Pienn, Michael; Olschewski, Horst; Urschler, Martin.

Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham : Springer, 2019. p. 664-672 (Lecture Notes in Computer Science; Vol. 11768).

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Bonheur, S, Stern, D, Payer, C, Pienn, M, Olschewski, H & Urschler, M 2019, Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Lecture Notes in Computer Science, vol. 11768, Springer, Cham, pp. 664-672, MICCAI 2019, Shenzen, China, 13/10/19. https://doi.org/10.1007/978-3-030-32254-0_74
Bonheur S, Stern D, Payer C, Pienn M, Olschewski H, Urschler M. Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham: Springer. 2019. p. 664-672. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-32254-0_74
Bonheur, Savinien ; Stern, Darko ; Payer, Christian ; Pienn, Michael ; Olschewski, Horst ; Urschler, Martin. / Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham : Springer, 2019. pp. 664-672 (Lecture Notes in Computer Science).
@inproceedings{274708457a694f85a998245791e01214,
title = "Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network",
abstract = "Despite some design limitations, CNNs have been largely adopted by the computer vision community due to their efficacy and versatility. Introduced by Sabour et al. to circumvent some limitations of CNNs, capsules replace scalars with vectors to encode appearance feature representation, allowing better preservation of spatial relationships between whole objects and its parts. They also introduced the dynamic routing mechanism, which allows to weight the contributions of parts to a whole object differently at each inference step. Recently, Hinton et al. have proposed to solely encode pose information to model such part-whole relationships. Additionally, they used a matrix instead of a vector encoding in the capsules framework. In this work, we introduce several improvements to the capsules framework, allowing it to be applied for multi-label semantic segmentation. More specifically, we combine pose and appearance information encoded as matrices into a new type of capsule, i.e. Matwo-Caps. Additionally, we propose a novel routing mechanism, i.e. Dual Routing, which effectively combines these two kinds of information. We evaluate our resulting Matwo-CapsNet on the JSRT chest X-ray dataset by comparing it to SegCaps, a capsule based network for binary segmentation, as well as to other CNN based state-of-the-art segmentation methods, where we show that our Matwo-CapsNet achieves competitive results, while requiring only a fraction of the parameters of other previously proposed methods.",
author = "Savinien Bonheur and Darko Stern and Christian Payer and Michael Pienn and Horst Olschewski and Martin Urschler",
year = "2019",
doi = "10.1007/978-3-030-32254-0_74",
language = "English",
isbn = "978-3-030-32253-3",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "664--672",
booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2019",

}

TY - GEN

T1 - Matwo-CapsNet: A Multi-label Semantic Segmentation Capsules Network

AU - Bonheur, Savinien

AU - Stern, Darko

AU - Payer, Christian

AU - Pienn, Michael

AU - Olschewski, Horst

AU - Urschler, Martin

PY - 2019

Y1 - 2019

N2 - Despite some design limitations, CNNs have been largely adopted by the computer vision community due to their efficacy and versatility. Introduced by Sabour et al. to circumvent some limitations of CNNs, capsules replace scalars with vectors to encode appearance feature representation, allowing better preservation of spatial relationships between whole objects and its parts. They also introduced the dynamic routing mechanism, which allows to weight the contributions of parts to a whole object differently at each inference step. Recently, Hinton et al. have proposed to solely encode pose information to model such part-whole relationships. Additionally, they used a matrix instead of a vector encoding in the capsules framework. In this work, we introduce several improvements to the capsules framework, allowing it to be applied for multi-label semantic segmentation. More specifically, we combine pose and appearance information encoded as matrices into a new type of capsule, i.e. Matwo-Caps. Additionally, we propose a novel routing mechanism, i.e. Dual Routing, which effectively combines these two kinds of information. We evaluate our resulting Matwo-CapsNet on the JSRT chest X-ray dataset by comparing it to SegCaps, a capsule based network for binary segmentation, as well as to other CNN based state-of-the-art segmentation methods, where we show that our Matwo-CapsNet achieves competitive results, while requiring only a fraction of the parameters of other previously proposed methods.

AB - Despite some design limitations, CNNs have been largely adopted by the computer vision community due to their efficacy and versatility. Introduced by Sabour et al. to circumvent some limitations of CNNs, capsules replace scalars with vectors to encode appearance feature representation, allowing better preservation of spatial relationships between whole objects and its parts. They also introduced the dynamic routing mechanism, which allows to weight the contributions of parts to a whole object differently at each inference step. Recently, Hinton et al. have proposed to solely encode pose information to model such part-whole relationships. Additionally, they used a matrix instead of a vector encoding in the capsules framework. In this work, we introduce several improvements to the capsules framework, allowing it to be applied for multi-label semantic segmentation. More specifically, we combine pose and appearance information encoded as matrices into a new type of capsule, i.e. Matwo-Caps. Additionally, we propose a novel routing mechanism, i.e. Dual Routing, which effectively combines these two kinds of information. We evaluate our resulting Matwo-CapsNet on the JSRT chest X-ray dataset by comparing it to SegCaps, a capsule based network for binary segmentation, as well as to other CNN based state-of-the-art segmentation methods, where we show that our Matwo-CapsNet achieves competitive results, while requiring only a fraction of the parameters of other previously proposed methods.

U2 - 10.1007/978-3-030-32254-0_74

DO - 10.1007/978-3-030-32254-0_74

M3 - Conference contribution

SN - 978-3-030-32253-3

T3 - Lecture Notes in Computer Science

SP - 664

EP - 672

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2019

PB - Springer

CY - Cham

ER -