ATGV-Net: Accurate Depth Super-Resolution

Gernot Riegler, Matthias Rüther, Horst Bischof

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

In this work we present a novel approach for single depth map super-resolution. Modern consumer depth sensors, especially Time-of-Flight sensors, produce dense depth measurements, but are affected by noise and have a low lateral resolution. We propose a method that combines the benefits of recent advances in machine learning based single image super-resolution, i.e. deep convolutional networks, with a variational method to recover accurate high-resolution depth maps. In particular, we integrate a variational method that models the piecewise affine structures apparent in depth data via an anisotropic total generalized variation regularization term on top of a deep network. We call our method ATGV-Net and train it end-to-end by unrolling the optimization procedure of the variational method. To train deep networks, a large corpus of training data with accurate ground-truth is required. We demonstrate that it is feasible to train our method solely on synthetic data that we generate in large quantities for this task. Our evaluations show that we achieve state-of-the-art results on three different benchmarks, as well as on a challenging Time-of-Flight dataset, all without utilizing an additional intensity image as guidance.
Original languageEnglish
Title of host publicationEuropean Conference on Computer Vision
PublisherSpringer International Publishing AG
Publication statusPublished - 2016

Fingerprint

Sensors
Learning systems

Cite this

Riegler, G., Rüther, M., & Bischof, H. (2016). ATGV-Net: Accurate Depth Super-Resolution. In European Conference on Computer Vision Springer International Publishing AG .

ATGV-Net: Accurate Depth Super-Resolution. / Riegler, Gernot; Rüther, Matthias; Bischof, Horst.

European Conference on Computer Vision. Springer International Publishing AG , 2016.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Riegler, G, Rüther, M & Bischof, H 2016, ATGV-Net: Accurate Depth Super-Resolution. in European Conference on Computer Vision. Springer International Publishing AG .
Riegler G, Rüther M, Bischof H. ATGV-Net: Accurate Depth Super-Resolution. In European Conference on Computer Vision. Springer International Publishing AG . 2016
Riegler, Gernot ; Rüther, Matthias ; Bischof, Horst. / ATGV-Net: Accurate Depth Super-Resolution. European Conference on Computer Vision. Springer International Publishing AG , 2016.
@inproceedings{47637fee70af431285d695fba479eb9c,
title = "ATGV-Net: Accurate Depth Super-Resolution",
abstract = "In this work we present a novel approach for single depth map super-resolution. Modern consumer depth sensors, especially Time-of-Flight sensors, produce dense depth measurements, but are affected by noise and have a low lateral resolution. We propose a method that combines the benefits of recent advances in machine learning based single image super-resolution, i.e. deep convolutional networks, with a variational method to recover accurate high-resolution depth maps. In particular, we integrate a variational method that models the piecewise affine structures apparent in depth data via an anisotropic total generalized variation regularization term on top of a deep network. We call our method ATGV-Net and train it end-to-end by unrolling the optimization procedure of the variational method. To train deep networks, a large corpus of training data with accurate ground-truth is required. We demonstrate that it is feasible to train our method solely on synthetic data that we generate in large quantities for this task. Our evaluations show that we achieve state-of-the-art results on three different benchmarks, as well as on a challenging Time-of-Flight dataset, all without utilizing an additional intensity image as guidance.",
author = "Gernot Riegler and Matthias R{\"u}ther and Horst Bischof",
year = "2016",
language = "English",
booktitle = "European Conference on Computer Vision",
publisher = "Springer International Publishing AG",
address = "Switzerland",

}

TY - GEN

T1 - ATGV-Net: Accurate Depth Super-Resolution

AU - Riegler, Gernot

AU - Rüther, Matthias

AU - Bischof, Horst

PY - 2016

Y1 - 2016

N2 - In this work we present a novel approach for single depth map super-resolution. Modern consumer depth sensors, especially Time-of-Flight sensors, produce dense depth measurements, but are affected by noise and have a low lateral resolution. We propose a method that combines the benefits of recent advances in machine learning based single image super-resolution, i.e. deep convolutional networks, with a variational method to recover accurate high-resolution depth maps. In particular, we integrate a variational method that models the piecewise affine structures apparent in depth data via an anisotropic total generalized variation regularization term on top of a deep network. We call our method ATGV-Net and train it end-to-end by unrolling the optimization procedure of the variational method. To train deep networks, a large corpus of training data with accurate ground-truth is required. We demonstrate that it is feasible to train our method solely on synthetic data that we generate in large quantities for this task. Our evaluations show that we achieve state-of-the-art results on three different benchmarks, as well as on a challenging Time-of-Flight dataset, all without utilizing an additional intensity image as guidance.

AB - In this work we present a novel approach for single depth map super-resolution. Modern consumer depth sensors, especially Time-of-Flight sensors, produce dense depth measurements, but are affected by noise and have a low lateral resolution. We propose a method that combines the benefits of recent advances in machine learning based single image super-resolution, i.e. deep convolutional networks, with a variational method to recover accurate high-resolution depth maps. In particular, we integrate a variational method that models the piecewise affine structures apparent in depth data via an anisotropic total generalized variation regularization term on top of a deep network. We call our method ATGV-Net and train it end-to-end by unrolling the optimization procedure of the variational method. To train deep networks, a large corpus of training data with accurate ground-truth is required. We demonstrate that it is feasible to train our method solely on synthetic data that we generate in large quantities for this task. Our evaluations show that we achieve state-of-the-art results on three different benchmarks, as well as on a challenging Time-of-Flight dataset, all without utilizing an additional intensity image as guidance.

M3 - Conference contribution

BT - European Conference on Computer Vision

PB - Springer International Publishing AG

ER -