Learning Lightprobes for Mixed Reality Illumination

David Mandl, Kwang Moo Yi, Peter Mohr-Ziak, Peter M. Roth, Pascal Fua, Vincent Lepetit, Dieter Schmalstieg, Denis Kalkofen

Research output: Contribution to conferencePaper

Abstract

This paper presents the first photometric registration pipeline for
Mixed Reality based on high quality illumination estimation using
convolutional neural networks (CNNs). For easy adaptation and deployment
of the system, we train the CNNs using purely synthetic
images and apply them to real image data. To keep the pipeline accurate
and efficient, we propose to fuse the light estimation results
from multiple CNN instances and show an approach for caching
estimates over time. For optimal performance, we furthermore explore
multiple strategies for the CNN training. Experimental results
show that the proposed method yields highly accurate estimates for
photo-realistic augmentations.
Original languageEnglish
Number of pages8
DOIs
Publication statusPublished - 9 Oct 2017
EventInternational Symposium on Mixed and Augmented Reality 2017: ISMAR 2017 - La Cité, Nantes, France
Duration: 9 Oct 201713 Oct 2017
https://ismar2017.sciencesconf.org/

Conference

ConferenceInternational Symposium on Mixed and Augmented Reality 2017
Abbreviated titleISMAR 2017
CountryFrance
CityNantes
Period9/10/1713/10/17
Internet address

Fingerprint Dive into the research topics of 'Learning Lightprobes for Mixed Reality Illumination'. Together they form a unique fingerprint.

Cite this