Generalized Feedback Loop for Joint Hand-Object Pose Estimation

Markus Oberweger, Paul Wohlhart, Vincent Lepetit

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Abstract

We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.
Originalspracheenglisch
TitelIEEE transactions on pattern analysis and machine intelligence
PublikationsstatusVeröffentlicht - 2019

Fingerprint

Feedback
Neural networks
Graphics processing unit

Dies zitieren

Oberweger, M., Wohlhart, P., & Lepetit, V. (2019). Generalized Feedback Loop for Joint Hand-Object Pose Estimation. in IEEE transactions on pattern analysis and machine intelligence

Generalized Feedback Loop for Joint Hand-Object Pose Estimation. / Oberweger, Markus; Wohlhart, Paul; Lepetit, Vincent.

IEEE transactions on pattern analysis and machine intelligence. 2019.

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Oberweger, M, Wohlhart, P & Lepetit, V 2019, Generalized Feedback Loop for Joint Hand-Object Pose Estimation. in IEEE transactions on pattern analysis and machine intelligence.
Oberweger M, Wohlhart P, Lepetit V. Generalized Feedback Loop for Joint Hand-Object Pose Estimation. in IEEE transactions on pattern analysis and machine intelligence. 2019
Oberweger, Markus ; Wohlhart, Paul ; Lepetit, Vincent. / Generalized Feedback Loop for Joint Hand-Object Pose Estimation. IEEE transactions on pattern analysis and machine intelligence. 2019.
@inproceedings{1aa376a3559543bdbc58cd2355581527,
title = "Generalized Feedback Loop for Joint Hand-Object Pose Estimation",
abstract = "We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.",
author = "Markus Oberweger and Paul Wohlhart and Vincent Lepetit",
year = "2019",
language = "English",
booktitle = "IEEE transactions on pattern analysis and machine intelligence",

}

TY - GEN

T1 - Generalized Feedback Loop for Joint Hand-Object Pose Estimation

AU - Oberweger, Markus

AU - Wohlhart, Paul

AU - Lepetit, Vincent

PY - 2019

Y1 - 2019

N2 - We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.

AB - We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.

M3 - Conference contribution

BT - IEEE transactions on pattern analysis and machine intelligence

ER -