Multi-Task cGAN for Simultaneous Spaceborne DSM Refinement and Roof-Type Classification

Ksenia Bittner, Marco Körner, Friedrich Fraundorfer, Peter Reinartz

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Abstract

Various deep learning applications benefit from multi-task learning with multiple regression and classification objectives by taking advantage of the similarities between individual tasks. This can result in improved learning efficiency and prediction accuracy for the task-specific models compared to separately trained models. In this paper, we make an observation of such influences for important remote sensing applications like elevation model generation and semantic segmentation tasks from the stereo half-meter resolution satellite digital surface models (DSMs). Mainly, we aim to generate good-quality DSMs with complete, as well as accurate level of detail (LoD) 2-like building forms and to assign an object class label to each pixel in the DSMs. For the label assignment task, we select the roof type classification problem to distinguish between flat, non-flat, and background pixels. To realize those tasks, we train a conditional generative adversarial network (cGAN) with an objective function based on least squares residuals and an auxiliary term based on normal vectors for further roof surface refinement. Besides, we investigate recently published deep learning architectures for both tasks and develop the final end-to-end network, which combines different models, as using them first separately, they provide the best results for their individual tasks.
Originalspracheenglisch
Aufsatznummer1262
FachzeitschriftRemote Sensing
Jahrgang11
Ausgabenummer11
DOIs
PublikationsstatusVeröffentlicht - 2019

Fingerprint

roof
learning
pixel
segmentation
multiple regression
train
remote sensing
prediction

Dies zitieren

Multi-Task cGAN for Simultaneous Spaceborne DSM Refinement and Roof-Type Classification. / Bittner, Ksenia; Körner, Marco; Fraundorfer, Friedrich; Reinartz, Peter.

in: Remote Sensing , Jahrgang 11, Nr. 11, 1262, 2019.

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Bittner, Ksenia ; Körner, Marco ; Fraundorfer, Friedrich ; Reinartz, Peter. / Multi-Task cGAN for Simultaneous Spaceborne DSM Refinement and Roof-Type Classification. in: Remote Sensing . 2019 ; Jahrgang 11, Nr. 11.
@article{2552f17f7e5d4b3795633d495cc3d8a1,
title = "Multi-Task cGAN for Simultaneous Spaceborne DSM Refinement and Roof-Type Classification",
abstract = "Various deep learning applications benefit from multi-task learning with multiple regression and classification objectives by taking advantage of the similarities between individual tasks. This can result in improved learning efficiency and prediction accuracy for the task-specific models compared to separately trained models. In this paper, we make an observation of such influences for important remote sensing applications like elevation model generation and semantic segmentation tasks from the stereo half-meter resolution satellite digital surface models (DSMs). Mainly, we aim to generate good-quality DSMs with complete, as well as accurate level of detail (LoD) 2-like building forms and to assign an object class label to each pixel in the DSMs. For the label assignment task, we select the roof type classification problem to distinguish between flat, non-flat, and background pixels. To realize those tasks, we train a conditional generative adversarial network (cGAN) with an objective function based on least squares residuals and an auxiliary term based on normal vectors for further roof surface refinement. Besides, we investigate recently published deep learning architectures for both tasks and develop the final end-to-end network, which combines different models, as using them first separately, they provide the best results for their individual tasks.",
author = "Ksenia Bittner and Marco K{\"o}rner and Friedrich Fraundorfer and Peter Reinartz",
year = "2019",
doi = "10.3390/rs11111262",
language = "English",
volume = "11",
journal = "Remote Sensing",
issn = "2072-4292",
publisher = "MDPI AG",
number = "11",

}

TY - JOUR

T1 - Multi-Task cGAN for Simultaneous Spaceborne DSM Refinement and Roof-Type Classification

AU - Bittner, Ksenia

AU - Körner, Marco

AU - Fraundorfer, Friedrich

AU - Reinartz, Peter

PY - 2019

Y1 - 2019

N2 - Various deep learning applications benefit from multi-task learning with multiple regression and classification objectives by taking advantage of the similarities between individual tasks. This can result in improved learning efficiency and prediction accuracy for the task-specific models compared to separately trained models. In this paper, we make an observation of such influences for important remote sensing applications like elevation model generation and semantic segmentation tasks from the stereo half-meter resolution satellite digital surface models (DSMs). Mainly, we aim to generate good-quality DSMs with complete, as well as accurate level of detail (LoD) 2-like building forms and to assign an object class label to each pixel in the DSMs. For the label assignment task, we select the roof type classification problem to distinguish between flat, non-flat, and background pixels. To realize those tasks, we train a conditional generative adversarial network (cGAN) with an objective function based on least squares residuals and an auxiliary term based on normal vectors for further roof surface refinement. Besides, we investigate recently published deep learning architectures for both tasks and develop the final end-to-end network, which combines different models, as using them first separately, they provide the best results for their individual tasks.

AB - Various deep learning applications benefit from multi-task learning with multiple regression and classification objectives by taking advantage of the similarities between individual tasks. This can result in improved learning efficiency and prediction accuracy for the task-specific models compared to separately trained models. In this paper, we make an observation of such influences for important remote sensing applications like elevation model generation and semantic segmentation tasks from the stereo half-meter resolution satellite digital surface models (DSMs). Mainly, we aim to generate good-quality DSMs with complete, as well as accurate level of detail (LoD) 2-like building forms and to assign an object class label to each pixel in the DSMs. For the label assignment task, we select the roof type classification problem to distinguish between flat, non-flat, and background pixels. To realize those tasks, we train a conditional generative adversarial network (cGAN) with an objective function based on least squares residuals and an auxiliary term based on normal vectors for further roof surface refinement. Besides, we investigate recently published deep learning architectures for both tasks and develop the final end-to-end network, which combines different models, as using them first separately, they provide the best results for their individual tasks.

U2 - 10.3390/rs11111262

DO - 10.3390/rs11111262

M3 - Article

VL - 11

JO - Remote Sensing

JF - Remote Sensing

SN - 2072-4292

IS - 11

M1 - 1262

ER -