Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks

Christian Payer*, Darko Štern, Thomas Neff, Horst Bischof, Martin Urschler

*Korrespondierende/r Autor/in für diese Arbeit

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem Konferenzband

Abstract

Different to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time. The network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal video information. Furthermore, we train the network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos. Afterwards, these embeddings are clustered among subsequent video frames to create the final tracked instance segmentations. We evaluate the recurrent hourglass network by segmenting left ventricles in MR videos of the heart, where it outperforms a network that does not incorporate video information. Furthermore, we show applicability of the cosine embedding loss for segmenting leaf instances on still images of plants. Finally, we evaluate the framework for instance segmentation and tracking on six datasets of the ISBI celltracking challenge, where it shows state-of-the-art performance.

Originalspracheenglisch
TitelMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
Herausgeber (Verlag)Springer Verlag Heidelberg
Seiten3-11
Seitenumfang9
ISBN (Print)9783030009335
DOIs
PublikationsstatusVeröffentlicht - 16 Sep 2018
Veranstaltung21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spanien
Dauer: 16 Sep 201820 Sep 2018

Publikationsreihe

NameLecture Notes in Computer Science
Band11071
ISSN (Print)0302-9743
ISSN (elektronisch)1611-3349

Konferenz

Konferenz21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
LandSpanien
OrtGranada
Zeitraum16/09/1820/09/18

ASJC Scopus subject areas

  • !!Theoretical Computer Science
  • !!Computer Science(all)

Fields of Expertise

  • Information, Communication & Computing

Kooperationen

  • BioTechMed-Graz

Fingerprint Untersuchen Sie die Forschungsthemen von „Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks“. Zusammen bilden sie einen einzigartigen Fingerprint.

  • Projekte

    • 1 Abschlussdatum

    FWF - FAME - Vollautomatische MRT-basierte Altersschätzung von Jugendlichen

    Bischof, H. & Urschler, M.

    1/07/1531/12/18

    Projekt: Foschungsprojekt

    Prämien

    Student Travel Award

    Christian Payer (Empfänger/-in), 2018

    Auszeichnung: Preise / Medaillen / Ehrungen

    Dieses zitieren

    Payer, C., Štern, D., Neff, T., Bischof, H., & Urschler, M. (2018). Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings (S. 3-11). (Lecture Notes in Computer Science; Band 11071 ). Springer Verlag Heidelberg. https://doi.org/10.1007/978-3-030-00934-2_1