Learning to Find Good Correspondences

Kwang Moo Yi, Eduard Trulls, Yuki Ono, Vincent Lepetit, Mathieu Salzmann, Pascal Fua

Research output: Contribution to journalArticle

Abstract

We develop a deep architecture to learn to find good correspondences for wide-baseline stereo. Given a set of putative sparse matches and the camera intrinsics, we train our network in an end-to-end fashion to label the correspondences as inliers or outliers, while simultaneously using them to recover the relative pose, as encoded by the essential matrix. Our architecture is based on a multi-layer perceptron operating on pixel coordinates rather than directly on the image, and is thus simple and small. We introduce a novel normalization technique, called Context Normalization, which allows us to process each data point separately while imbuing it with global information, and also makes the network invariant to the order of the correspondences. Our experiments on multiple challenging datasets demonstrate that our method is able to drastically improve the state of the art with little training data.

Keywords

  • cs.CV

Cite this

Yi, K. M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., & Fua, P. (2018). Learning to Find Good Correspondences. arXiv.org e-Print archive.

Learning to Find Good Correspondences. / Yi, Kwang Moo; Trulls, Eduard; Ono, Yuki; Lepetit, Vincent; Salzmann, Mathieu; Fua, Pascal.

In: arXiv.org e-Print archive, 2018.

Research output: Contribution to journalArticle

Yi, KM, Trulls, E, Ono, Y, Lepetit, V, Salzmann, M & Fua, P 2018, 'Learning to Find Good Correspondences' arXiv.org e-Print archive.
Yi KM, Trulls E, Ono Y, Lepetit V, Salzmann M, Fua P. Learning to Find Good Correspondences. arXiv.org e-Print archive. 2018.
Yi, Kwang Moo ; Trulls, Eduard ; Ono, Yuki ; Lepetit, Vincent ; Salzmann, Mathieu ; Fua, Pascal. / Learning to Find Good Correspondences. In: arXiv.org e-Print archive. 2018
@article{e07bf7c4bf294215be954d5a759eac61,
title = "Learning to Find Good Correspondences",
abstract = "We develop a deep architecture to learn to find good correspondences for wide-baseline stereo. Given a set of putative sparse matches and the camera intrinsics, we train our network in an end-to-end fashion to label the correspondences as inliers or outliers, while simultaneously using them to recover the relative pose, as encoded by the essential matrix. Our architecture is based on a multi-layer perceptron operating on pixel coordinates rather than directly on the image, and is thus simple and small. We introduce a novel normalization technique, called Context Normalization, which allows us to process each data point separately while imbuing it with global information, and also makes the network invariant to the order of the correspondences. Our experiments on multiple challenging datasets demonstrate that our method is able to drastically improve the state of the art with little training data.",
keywords = "cs.CV",
author = "Yi, {Kwang Moo} and Eduard Trulls and Yuki Ono and Vincent Lepetit and Mathieu Salzmann and Pascal Fua",
note = "CVPR 2018 (Oral)",
year = "2018",
language = "English",
journal = "arXiv.org e-Print archive",
publisher = "Cornell University Library",

}

TY - JOUR

T1 - Learning to Find Good Correspondences

AU - Yi,Kwang Moo

AU - Trulls,Eduard

AU - Ono,Yuki

AU - Lepetit,Vincent

AU - Salzmann,Mathieu

AU - Fua,Pascal

N1 - CVPR 2018 (Oral)

PY - 2018

Y1 - 2018

N2 - We develop a deep architecture to learn to find good correspondences for wide-baseline stereo. Given a set of putative sparse matches and the camera intrinsics, we train our network in an end-to-end fashion to label the correspondences as inliers or outliers, while simultaneously using them to recover the relative pose, as encoded by the essential matrix. Our architecture is based on a multi-layer perceptron operating on pixel coordinates rather than directly on the image, and is thus simple and small. We introduce a novel normalization technique, called Context Normalization, which allows us to process each data point separately while imbuing it with global information, and also makes the network invariant to the order of the correspondences. Our experiments on multiple challenging datasets demonstrate that our method is able to drastically improve the state of the art with little training data.

AB - We develop a deep architecture to learn to find good correspondences for wide-baseline stereo. Given a set of putative sparse matches and the camera intrinsics, we train our network in an end-to-end fashion to label the correspondences as inliers or outliers, while simultaneously using them to recover the relative pose, as encoded by the essential matrix. Our architecture is based on a multi-layer perceptron operating on pixel coordinates rather than directly on the image, and is thus simple and small. We introduce a novel normalization technique, called Context Normalization, which allows us to process each data point separately while imbuing it with global information, and also makes the network invariant to the order of the correspondences. Our experiments on multiple challenging datasets demonstrate that our method is able to drastically improve the state of the art with little training data.

KW - cs.CV

M3 - Article

JO - arXiv.org e-Print archive

T2 - arXiv.org e-Print archive

JF - arXiv.org e-Print archive

ER -