Learned Collaborative Stereo Refinement

Patrick Knöbelreiter*, Thomas Pock

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


In this work, we propose a learning-based method to denoise and refine disparity maps. The proposed variational network arises naturally from unrolling the iterates of a proximal gradient method applied to a variational energy defined in a joint disparity, color, and confidence image space. Our method allows to learn a robust collaborative regularizer leveraging the joint statistics of the color image, the confidence map and the disparity map. Due to the variational structure of our method, the individual steps can be easily visualized, thus enabling interpretability of the method. We can therefore provide interesting insights into how our method refines and denoises disparity maps. To this end, we can visualize and interpret the learned filters and activation functions and prove the increased reliability of the predicted pixel-wise confidence maps. Furthermore, the optimization based structure of our refinement module allows us to compute eigen disparity maps, which reveal structural properties of our refinement module. The efficiency of our method is demonstrated on the publicly available stereo benchmarks Middlebury 2014 and Kitti 2015.

Original languageEnglish
Pages (from-to)2565-2582
Number of pages18
JournalInternational Journal of Computer Vision
Issue number9
Publication statusPublished - Sep 2021


  • Deep learning
  • Interpretable AI
  • Optimization
  • Refinement
  • Stereo

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Learned Collaborative Stereo Refinement'. Together they form a unique fingerprint.

Cite this