Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation

Research output: Contribution to conferencePaperResearchpeer-review

Abstract

We propose a jointly trainable model for semantic
segmentation and disparity map estimation. In this work we
utilize the fact that the two tasks have complementary strength
and weaknesses. Traditional depth prediction algorithms rely on
low-level features and often have problems at large textureless
regions, while for semantic segmentation these regions are
easier to capture. We propose a CNN-based architecture, where
both tasks are tightly interconnected to each other. The model
consists of an encoding stage which computes features for both
tasks, semantic segmentation and disparity estimation. In the
decoding stage we explicitly add the semantic predictions to
the disparity decoding branch and we additionally allow to exchange information in the intermediate feature representations.
Furthermore, we set the focus on efficiency, which we achieve
by the usage of previously introduced ESP building blocks. We
evaluate the model on the commonly used KITTI dataset.
Original languageEnglish
Pages147
Number of pages152
Publication statusPublished - 9 May 2019
EventARW & OAGM Workshop 2019: Austrian Robotics Workshop and OAGM Workshop 2019 - Steyr, Austria
Duration: 9 May 201910 May 2019

Conference

ConferenceARW & OAGM Workshop 2019
CountryAustria
CitySteyr
Period9/05/1910/05/19

Fingerprint

Semantics
Decoding

Keywords

  • Stereo vision
  • Semantic Segmentation
  • Multi-Taks-Learning

Cite this

Harb, R., & Knöbelreiter, P. (2019). Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation. 147. Paper presented at ARW & OAGM Workshop 2019, Steyr, Austria.

Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation. / Harb, Robert; Knöbelreiter, Patrick.

2019. 147 Paper presented at ARW & OAGM Workshop 2019, Steyr, Austria.

Research output: Contribution to conferencePaperResearchpeer-review

Harb, R & Knöbelreiter, P 2019, 'Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation' Paper presented at ARW & OAGM Workshop 2019, Steyr, Austria, 9/05/19 - 10/05/19, pp. 147.
Harb R, Knöbelreiter P. Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation. 2019. Paper presented at ARW & OAGM Workshop 2019, Steyr, Austria.
Harb, Robert ; Knöbelreiter, Patrick. / Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation. Paper presented at ARW & OAGM Workshop 2019, Steyr, Austria.152 p.
@conference{699e7092b799481b887dca14bc402d06,
title = "Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation",
abstract = "We propose a jointly trainable model for semanticsegmentation and disparity map estimation. In this work weutilize the fact that the two tasks have complementary strengthand weaknesses. Traditional depth prediction algorithms rely onlow-level features and often have problems at large texturelessregions, while for semantic segmentation these regions areeasier to capture. We propose a CNN-based architecture, whereboth tasks are tightly interconnected to each other. The modelconsists of an encoding stage which computes features for bothtasks, semantic segmentation and disparity estimation. In thedecoding stage we explicitly add the semantic predictions tothe disparity decoding branch and we additionally allow to exchange information in the intermediate feature representations.Furthermore, we set the focus on efficiency, which we achieveby the usage of previously introduced ESP building blocks. Weevaluate the model on the commonly used KITTI dataset.",
keywords = "Stereo vision, Semantic Segmentation, Multi-Taks-Learning",
author = "Robert Harb and Patrick Kn{\"o}belreiter",
year = "2019",
month = "5",
day = "9",
language = "English",
pages = "147",
note = "ARW & OAGM Workshop 2019 : Austrian Robotics Workshop and OAGM Workshop 2019 ; Conference date: 09-05-2019 Through 10-05-2019",

}

TY - CONF

T1 - Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation

AU - Harb, Robert

AU - Knöbelreiter, Patrick

PY - 2019/5/9

Y1 - 2019/5/9

N2 - We propose a jointly trainable model for semanticsegmentation and disparity map estimation. In this work weutilize the fact that the two tasks have complementary strengthand weaknesses. Traditional depth prediction algorithms rely onlow-level features and often have problems at large texturelessregions, while for semantic segmentation these regions areeasier to capture. We propose a CNN-based architecture, whereboth tasks are tightly interconnected to each other. The modelconsists of an encoding stage which computes features for bothtasks, semantic segmentation and disparity estimation. In thedecoding stage we explicitly add the semantic predictions tothe disparity decoding branch and we additionally allow to exchange information in the intermediate feature representations.Furthermore, we set the focus on efficiency, which we achieveby the usage of previously introduced ESP building blocks. Weevaluate the model on the commonly used KITTI dataset.

AB - We propose a jointly trainable model for semanticsegmentation and disparity map estimation. In this work weutilize the fact that the two tasks have complementary strengthand weaknesses. Traditional depth prediction algorithms rely onlow-level features and often have problems at large texturelessregions, while for semantic segmentation these regions areeasier to capture. We propose a CNN-based architecture, whereboth tasks are tightly interconnected to each other. The modelconsists of an encoding stage which computes features for bothtasks, semantic segmentation and disparity estimation. In thedecoding stage we explicitly add the semantic predictions tothe disparity decoding branch and we additionally allow to exchange information in the intermediate feature representations.Furthermore, we set the focus on efficiency, which we achieveby the usage of previously introduced ESP building blocks. Weevaluate the model on the commonly used KITTI dataset.

KW - Stereo vision

KW - Semantic Segmentation

KW - Multi-Taks-Learning

M3 - Paper

SP - 147

ER -