Scalable Full Flow with Learned Binary Descriptors

Gottfried Munda, Alexander Shekhovtsov, Patrick Knöbelreiter, Thomas Pock

Publikation: KonferenzbeitragPaperForschungBegutachtung

Abstract

We propose a method for large displacement optical flow in
which local matching costs are learned by a convolutional neural network
(CNN) and a smoothness prior is imposed by a conditional random
field (CRF). We tackle the computation- and memory-intensive operations
on the 4D cost volume by a min-projection which reduces memory
complexity from quadratic to linear and binary descriptors for efficient
matching. This enables evaluation of the cost on the fly and allows to
perform learning and CRF inference on high resolution images without
ever storing the 4D cost volume. To address the problem of learning binary
descriptors we propose a new hybrid learning scheme. In contrast
to current state of the art approaches for learning binary CNNs we can
compute the exact non-zero gradient within our model. We compare several
methods for training binary descriptors and show results on public
available benchmarks.
Originalspracheenglisch
PublikationsstatusVeröffentlicht - 13 Sep 2017
Veranstaltung39th German Conference on Pattern Recognition - Basel, Schweiz
Dauer: 13 Sep 201615 Sep 2017
https://gcpr2017.dmi.unibas.ch/en/

Konferenz

Konferenz39th German Conference on Pattern Recognition
Kurztitel GCPR 2017
LandSchweiz
OrtBasel
Zeitraum13/09/1615/09/17
Internetadresse

Fingerprint

Costs
Optical flows
Image resolution
Neural networks
Data storage equipment

Dies zitieren

Munda, G., Shekhovtsov, A., Knöbelreiter, P., & Pock, T. (2017). Scalable Full Flow with Learned Binary Descriptors. Beitrag in 39th German Conference on Pattern Recognition, Basel, Schweiz.

Scalable Full Flow with Learned Binary Descriptors. / Munda, Gottfried; Shekhovtsov, Alexander; Knöbelreiter, Patrick; Pock, Thomas.

2017. Beitrag in 39th German Conference on Pattern Recognition, Basel, Schweiz.

Publikation: KonferenzbeitragPaperForschungBegutachtung

Munda, G, Shekhovtsov, A, Knöbelreiter, P & Pock, T 2017, 'Scalable Full Flow with Learned Binary Descriptors' Beitrag in, Basel, Schweiz, 13/09/16 - 15/09/17, .
Munda G, Shekhovtsov A, Knöbelreiter P, Pock T. Scalable Full Flow with Learned Binary Descriptors. 2017. Beitrag in 39th German Conference on Pattern Recognition, Basel, Schweiz.
Munda, Gottfried ; Shekhovtsov, Alexander ; Knöbelreiter, Patrick ; Pock, Thomas. / Scalable Full Flow with Learned Binary Descriptors. Beitrag in 39th German Conference on Pattern Recognition, Basel, Schweiz.
@conference{719f5dfb3aab41e786b48d6951cde53e,
title = "Scalable Full Flow with Learned Binary Descriptors",
abstract = "We propose a method for large displacement optical flow inwhich local matching costs are learned by a convolutional neural network(CNN) and a smoothness prior is imposed by a conditional randomfield (CRF). We tackle the computation- and memory-intensive operationson the 4D cost volume by a min-projection which reduces memorycomplexity from quadratic to linear and binary descriptors for efficientmatching. This enables evaluation of the cost on the fly and allows toperform learning and CRF inference on high resolution images withoutever storing the 4D cost volume. To address the problem of learning binarydescriptors we propose a new hybrid learning scheme. In contrastto current state of the art approaches for learning binary CNNs we cancompute the exact non-zero gradient within our model. We compare severalmethods for training binary descriptors and show results on publicavailable benchmarks.",
author = "Gottfried Munda and Alexander Shekhovtsov and Patrick Kn{\"o}belreiter and Thomas Pock",
year = "2017",
month = "9",
day = "13",
language = "English",
note = "39th German Conference on Pattern Recognition, GCPR 2017 ; Conference date: 13-09-2016 Through 15-09-2017",
url = "https://gcpr2017.dmi.unibas.ch/en/",

}

TY - CONF

T1 - Scalable Full Flow with Learned Binary Descriptors

AU - Munda, Gottfried

AU - Shekhovtsov, Alexander

AU - Knöbelreiter, Patrick

AU - Pock, Thomas

PY - 2017/9/13

Y1 - 2017/9/13

N2 - We propose a method for large displacement optical flow inwhich local matching costs are learned by a convolutional neural network(CNN) and a smoothness prior is imposed by a conditional randomfield (CRF). We tackle the computation- and memory-intensive operationson the 4D cost volume by a min-projection which reduces memorycomplexity from quadratic to linear and binary descriptors for efficientmatching. This enables evaluation of the cost on the fly and allows toperform learning and CRF inference on high resolution images withoutever storing the 4D cost volume. To address the problem of learning binarydescriptors we propose a new hybrid learning scheme. In contrastto current state of the art approaches for learning binary CNNs we cancompute the exact non-zero gradient within our model. We compare severalmethods for training binary descriptors and show results on publicavailable benchmarks.

AB - We propose a method for large displacement optical flow inwhich local matching costs are learned by a convolutional neural network(CNN) and a smoothness prior is imposed by a conditional randomfield (CRF). We tackle the computation- and memory-intensive operationson the 4D cost volume by a min-projection which reduces memorycomplexity from quadratic to linear and binary descriptors for efficientmatching. This enables evaluation of the cost on the fly and allows toperform learning and CRF inference on high resolution images withoutever storing the 4D cost volume. To address the problem of learning binarydescriptors we propose a new hybrid learning scheme. In contrastto current state of the art approaches for learning binary CNNs we cancompute the exact non-zero gradient within our model. We compare severalmethods for training binary descriptors and show results on publicavailable benchmarks.

UR - https://arxiv.org/pdf/1707.06427.pdf

M3 - Paper

ER -