Casting Geometric Constraints in Semantic Segmentation as Semi-Supervised Learning

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

We propose a simple yet effective method to learn to segment new indoor scenes from an RGB-D sequence: State-of-the-art methods trained on one dataset, even as large as SUNRGB-D dataset, can perform poorly when applied to images that are not part of the dataset, because of the dataset bias, a common phenomenon in computer vision. To make semantic segmentation more useful in practice, we learn to segment new indoor scenes from sequences without manual annotations by exploiting geometric constraints and readily available training data from SUNRGB-D. As a result, we can then robustly segment new images of these scenes from color information only. To efficiently exploit geometric constraints for our purpose, we propose to cast these constraints as semi-supervised terms, which enforce the fact that the same class should be predicted for the projections of the same 3D location in different images. We show that this approach results in a simple yet very powerful method, which can annotate sequences of ScanNet and our own sequences using only annotations from SUNRGB-D.
Original languageEnglish
Title of host publicationarXiv preprint arXiv:1904.12534
Publication statusPublished - 2019

Fingerprint

Supervised learning
Computer vision
Casting
Semantics
Color

Cite this

Casting Geometric Constraints in Semantic Segmentation as Semi-Supervised Learning. / Stekovic, Sinisa; Fraundorfer, Friedrich; Lepetit, Vincent.

arXiv preprint arXiv:1904.12534. 2019.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

@inproceedings{45f1224e9be245049b04ebfbc4739281,
title = "Casting Geometric Constraints in Semantic Segmentation as Semi-Supervised Learning",
abstract = "We propose a simple yet effective method to learn to segment new indoor scenes from an RGB-D sequence: State-of-the-art methods trained on one dataset, even as large as SUNRGB-D dataset, can perform poorly when applied to images that are not part of the dataset, because of the dataset bias, a common phenomenon in computer vision. To make semantic segmentation more useful in practice, we learn to segment new indoor scenes from sequences without manual annotations by exploiting geometric constraints and readily available training data from SUNRGB-D. As a result, we can then robustly segment new images of these scenes from color information only. To efficiently exploit geometric constraints for our purpose, we propose to cast these constraints as semi-supervised terms, which enforce the fact that the same class should be predicted for the projections of the same 3D location in different images. We show that this approach results in a simple yet very powerful method, which can annotate sequences of ScanNet and our own sequences using only annotations from SUNRGB-D.",
author = "Sinisa Stekovic and Friedrich Fraundorfer and Vincent Lepetit",
year = "2019",
language = "English",
booktitle = "arXiv preprint arXiv:1904.12534",

}

TY - GEN

T1 - Casting Geometric Constraints in Semantic Segmentation as Semi-Supervised Learning

AU - Stekovic, Sinisa

AU - Fraundorfer, Friedrich

AU - Lepetit, Vincent

PY - 2019

Y1 - 2019

N2 - We propose a simple yet effective method to learn to segment new indoor scenes from an RGB-D sequence: State-of-the-art methods trained on one dataset, even as large as SUNRGB-D dataset, can perform poorly when applied to images that are not part of the dataset, because of the dataset bias, a common phenomenon in computer vision. To make semantic segmentation more useful in practice, we learn to segment new indoor scenes from sequences without manual annotations by exploiting geometric constraints and readily available training data from SUNRGB-D. As a result, we can then robustly segment new images of these scenes from color information only. To efficiently exploit geometric constraints for our purpose, we propose to cast these constraints as semi-supervised terms, which enforce the fact that the same class should be predicted for the projections of the same 3D location in different images. We show that this approach results in a simple yet very powerful method, which can annotate sequences of ScanNet and our own sequences using only annotations from SUNRGB-D.

AB - We propose a simple yet effective method to learn to segment new indoor scenes from an RGB-D sequence: State-of-the-art methods trained on one dataset, even as large as SUNRGB-D dataset, can perform poorly when applied to images that are not part of the dataset, because of the dataset bias, a common phenomenon in computer vision. To make semantic segmentation more useful in practice, we learn to segment new indoor scenes from sequences without manual annotations by exploiting geometric constraints and readily available training data from SUNRGB-D. As a result, we can then robustly segment new images of these scenes from color information only. To efficiently exploit geometric constraints for our purpose, we propose to cast these constraints as semi-supervised terms, which enforce the fact that the same class should be predicted for the projections of the same 3D location in different images. We show that this approach results in a simple yet very powerful method, which can annotate sequences of ScanNet and our own sequences using only annotations from SUNRGB-D.

M3 - Conference contribution

BT - arXiv preprint arXiv:1904.12534

ER -