Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks

Horst Bischof, Christian Payer, Darko Stern, Marlies Feiner, Martin Urschler

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells. To create the final tracked instance segmentations, the pixel-wise embeddings are clustered among subsequent video frames …
Original languageEnglish
Pages (from-to)106-119
JournalMedical image analysis
Publication statusPublished - 2019

Cite this

Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks. / Bischof, Horst; Payer, Christian; Stern, Darko; Feiner, Marlies; Urschler, Martin.

In: Medical image analysis, 2019, p. 106-119.

Research output: Contribution to journalArticleResearchpeer-review

@article{75ac171496584dc1bf09f1a24642ec0d,
title = "Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks",
abstract = "Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells. To create the final tracked instance segmentations, the pixel-wise embeddings are clustered among subsequent video frames …",
author = "Horst Bischof and Christian Payer and Darko Stern and Marlies Feiner and Martin Urschler",
year = "2019",
language = "English",
pages = "106--119",
journal = "Medical image analysis",
issn = "1361-8415",
publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks

AU - Bischof, Horst

AU - Payer, Christian

AU - Stern, Darko

AU - Feiner, Marlies

AU - Urschler, Martin

PY - 2019

Y1 - 2019

N2 - Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells. To create the final tracked instance segmentations, the pixel-wise embeddings are clustered among subsequent video frames …

AB - Differently to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same object class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time, which is highly relevant, e.g., in biomedical applications involving cell growth and migration. Our network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal information, e.g., from microscopy videos. Moreover, we train our network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos, even in the presence of dynamic structural changes due to mitosis of cells. To create the final tracked instance segmentations, the pixel-wise embeddings are clustered among subsequent video frames …

M3 - Article

SP - 106

EP - 119

JO - Medical image analysis

JF - Medical image analysis

SN - 1361-8415

ER -