DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM

Ryo Hachiuma, Christian Pirchheim, Dieter Schmalstieg, Hideo Saito

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Abstract

We present DetectFusion, an RGB-D SLAM system that runs in real time and can robustly handle semantically known and unknown objects that can move dynamically in the scene. Our system detects, segments and assigns semantic class labels to known objects in the scene, while tracking and reconstructing them even when they move independently in front of the monocular camera. In contrast to related work, we achieve real-time computational performance on semantic instance segmentation with a novel method combining 2D object detection and 3D geometric segmentation. In addition, we propose a method for detecting and segmenting the motion of semantically unknown objects, thus further improving the accuracy of camera tracking and map reconstruction. We show that our method performs on par or better than previous work in terms of localization and object reconstruction accuracy, while achieving about 20 FPS even if the objects are segmented in each frame.
Originalspracheenglisch
TitelProceedings British Machine Vision Conference (BMVC)
PublikationsstatusAngenommen/In Druck - 2019
VeranstaltungBritish Machine Vision Conference - Cardiff University, Cardiff , Großbritannien / Vereinigtes Königreich
Dauer: 9 Sep 201912 Sep 2019
https://bmvc2019.org/

Konferenz

KonferenzBritish Machine Vision Conference
KurztitelBMVC
LandGroßbritannien / Vereinigtes Königreich
OrtCardiff
Zeitraum9/09/1912/09/19
Internetadresse

Fingerprint

Semantics
Cameras
Labels
Object detection

Dies zitieren

Hachiuma, R., Pirchheim, C., Schmalstieg, D., & Saito, H. (Angenommen/Im Druck). DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM. in Proceedings British Machine Vision Conference (BMVC)

DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM. / Hachiuma, Ryo; Pirchheim, Christian; Schmalstieg, Dieter; Saito, Hideo.

Proceedings British Machine Vision Conference (BMVC). 2019.

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Hachiuma, R, Pirchheim, C, Schmalstieg, D & Saito, H 2019, DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM. in Proceedings British Machine Vision Conference (BMVC)., Cardiff , Großbritannien / Vereinigtes Königreich, 9/09/19.
Hachiuma R, Pirchheim C, Schmalstieg D, Saito H. DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM. in Proceedings British Machine Vision Conference (BMVC). 2019
@inproceedings{2dfa8bb25b704e39a1e024387c0fc14c,
title = "DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM",
abstract = "We present DetectFusion, an RGB-D SLAM system that runs in real time and can robustly handle semantically known and unknown objects that can move dynamically in the scene. Our system detects, segments and assigns semantic class labels to known objects in the scene, while tracking and reconstructing them even when they move independently in front of the monocular camera. In contrast to related work, we achieve real-time computational performance on semantic instance segmentation with a novel method combining 2D object detection and 3D geometric segmentation. In addition, we propose a method for detecting and segmenting the motion of semantically unknown objects, thus further improving the accuracy of camera tracking and map reconstruction. We show that our method performs on par or better than previous work in terms of localization and object reconstruction accuracy, while achieving about 20 FPS even if the objects are segmented in each frame.",
author = "Ryo Hachiuma and Christian Pirchheim and Dieter Schmalstieg and Hideo Saito",
year = "2019",
language = "English",
booktitle = "Proceedings British Machine Vision Conference (BMVC)",

}

TY - GEN

T1 - DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM

AU - Hachiuma, Ryo

AU - Pirchheim, Christian

AU - Schmalstieg, Dieter

AU - Saito, Hideo

PY - 2019

Y1 - 2019

N2 - We present DetectFusion, an RGB-D SLAM system that runs in real time and can robustly handle semantically known and unknown objects that can move dynamically in the scene. Our system detects, segments and assigns semantic class labels to known objects in the scene, while tracking and reconstructing them even when they move independently in front of the monocular camera. In contrast to related work, we achieve real-time computational performance on semantic instance segmentation with a novel method combining 2D object detection and 3D geometric segmentation. In addition, we propose a method for detecting and segmenting the motion of semantically unknown objects, thus further improving the accuracy of camera tracking and map reconstruction. We show that our method performs on par or better than previous work in terms of localization and object reconstruction accuracy, while achieving about 20 FPS even if the objects are segmented in each frame.

AB - We present DetectFusion, an RGB-D SLAM system that runs in real time and can robustly handle semantically known and unknown objects that can move dynamically in the scene. Our system detects, segments and assigns semantic class labels to known objects in the scene, while tracking and reconstructing them even when they move independently in front of the monocular camera. In contrast to related work, we achieve real-time computational performance on semantic instance segmentation with a novel method combining 2D object detection and 3D geometric segmentation. In addition, we propose a method for detecting and segmenting the motion of semantically unknown objects, thus further improving the accuracy of camera tracking and map reconstruction. We show that our method performs on par or better than previous work in terms of localization and object reconstruction accuracy, while achieving about 20 FPS even if the objects are segmented in each frame.

UR - https://arxiv.org/abs/1907.09127

UR - https://www.youtube.com/watch?v=Ys3FXEP3A_4

M3 - Conference contribution

BT - Proceedings British Machine Vision Conference (BMVC)

ER -