End-to-End Training of Hybrid CNN-CRF Models for Stereo

Patrick Knöbelreiter, Christian Reinbacher, Alexander Shekhovtsov, Thomas Pock

Research output: Contribution to journalArticleResearchpeer-review

Abstract

We propose a novel method for stereo estimation, combining advantages of convolutional neural networks (CNNs) and optimization-based approaches. The optimization, posed as a conditional random field (CRF), takes local matching costs and consistency-enforcing (smoothness) costs as inputs, both estimated by CNN blocks. To perform the inference in the CRF we use an approach based on linear programming relaxation with a fixed number of iterations. We address the challenging problem of training this hybrid model end-to-end. We show that in the discriminative formulation (structured support vector machine) the training is practically feasible. The trained hybrid model with shallow CNNs is comparable to state-of-the-art deep models in both time and performance. The optimization part efficiently replaces sophisticated and not jointly trainable (but commonly applied) post-processing steps by a trainable, well-understood model.
Original languageEnglish
JournalarXiv.org e-Print archive
Publication statusPublished - 30 Nov 2016

Keywords

  • cs.CV

Fingerprint Dive into the research topics of 'End-to-End Training of Hybrid CNN-CRF Models for Stereo'. Together they form a unique fingerprint.

  • Activities

    • 1 Poster presentation

    End-to-End Training of Hybrid CNN-CRF Models for Stereo

    Patrick Knöbelreiter (Speaker)
    21 Jul 2017

    Activity: Talk or presentationPoster presentationScience to science

    Cite this