3D Pose Estimation and 3D Model Retrieval for Objects in the Wild

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a scalable, efficient and accurate approach to retrieve 3D models for objects in the wild. Our contribution is twofold. We first present a 3D pose estimation approach for object categories which significantly outperforms the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior to retrieve 3D models which accurately represent the geometry of objects in RGB images. For this purpose, we render depth images from 3D models under our predicted pose and match learned image descriptors of RGB images against those of rendered depth images using a CNN-based multi-view metric learning approach. In this way, we are the first to report quantitative results for 3D model retrieval on Pascal3D+, where our method chooses the same models as human annotators for 50% of the validation images on average. In addition, we show that our method, which was trained purely on Pascal3D+, retrieves rich and accurate 3D models from ShapeNet given RGB images of objects in the wild.
LanguageEnglish
Title of host publicationProceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Number of pages10
StatusPublished - 2018

Fingerprint

Geometry

Cite this

Grabner, A., Roth, P. M., & Lepetit, V. (2018). 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. / Grabner, Alexander; Roth, Peter M.; Lepetit, Vincent.

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Grabner, A, Roth, PM & Lepetit, V 2018, 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Grabner A, Roth PM, Lepetit V. 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
Grabner, Alexander ; Roth, Peter M. ; Lepetit, Vincent. / 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
@inproceedings{5370adf3708a41978b68c5a00598047b,
title = "3D Pose Estimation and 3D Model Retrieval for Objects in the Wild",
abstract = "We propose a scalable, efficient and accurate approach to retrieve 3D models for objects in the wild. Our contribution is twofold. We first present a 3D pose estimation approach for object categories which significantly outperforms the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior to retrieve 3D models which accurately represent the geometry of objects in RGB images. For this purpose, we render depth images from 3D models under our predicted pose and match learned image descriptors of RGB images against those of rendered depth images using a CNN-based multi-view metric learning approach. In this way, we are the first to report quantitative results for 3D model retrieval on Pascal3D+, where our method chooses the same models as human annotators for 50{\%} of the validation images on average. In addition, we show that our method, which was trained purely on Pascal3D+, retrieves rich and accurate 3D models from ShapeNet given RGB images of objects in the wild.",
author = "Alexander Grabner and Roth, {Peter M.} and Vincent Lepetit",
year = "2018",
language = "English",
booktitle = "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition",

}

TY - GEN

T1 - 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild

AU - Grabner,Alexander

AU - Roth,Peter M.

AU - Lepetit,Vincent

PY - 2018

Y1 - 2018

N2 - We propose a scalable, efficient and accurate approach to retrieve 3D models for objects in the wild. Our contribution is twofold. We first present a 3D pose estimation approach for object categories which significantly outperforms the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior to retrieve 3D models which accurately represent the geometry of objects in RGB images. For this purpose, we render depth images from 3D models under our predicted pose and match learned image descriptors of RGB images against those of rendered depth images using a CNN-based multi-view metric learning approach. In this way, we are the first to report quantitative results for 3D model retrieval on Pascal3D+, where our method chooses the same models as human annotators for 50% of the validation images on average. In addition, we show that our method, which was trained purely on Pascal3D+, retrieves rich and accurate 3D models from ShapeNet given RGB images of objects in the wild.

AB - We propose a scalable, efficient and accurate approach to retrieve 3D models for objects in the wild. Our contribution is twofold. We first present a 3D pose estimation approach for object categories which significantly outperforms the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior to retrieve 3D models which accurately represent the geometry of objects in RGB images. For this purpose, we render depth images from 3D models under our predicted pose and match learned image descriptors of RGB images against those of rendered depth images using a CNN-based multi-view metric learning approach. In this way, we are the first to report quantitative results for 3D model retrieval on Pascal3D+, where our method chooses the same models as human annotators for 50% of the validation images on average. In addition, we show that our method, which was trained purely on Pascal3D+, retrieves rich and accurate 3D models from ShapeNet given RGB images of objects in the wild.

M3 - Conference contribution

BT - Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

ER -