Neural Cameras: Learning Camera Characteristics for Coherent Mixed Reality Rendering

David Mandl, Peter M. Roth, Tobias Langlotz, Christoph Ebner, Shohei Mori, Stefanie Zollmann, Peter Mohr, Denis Kalkofen

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandBegutachtung

Abstract

Coherent rendering is important for generating plausible Mixed Reality presentations of virtual objects within a user’s real-world environment. Besides photo-realistic rendering and correct lighting, visual coherence requires simulating the imaging system that is used to capture the real environment. While existing approaches either focus on a specific camera or a specific component of the imaging system, we introduce Neural Cameras, the first approach that jointly simulates all major components of an arbitrary modern camera using neural networks. Our system allows for adding new cameras to the framework by learning the visual properties from a database of images that has been captured using the physical camera. We present qualitative and quantitative results and discuss future direction for research that emerge from using Neural Cameras.
Originalsprachedeutsch
Titel2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
Herausgeber (Verlag)IEEE Publications
Seiten508-516
Seitenumfang9
ISBN (Print)978-1-7281-9777-7
DOIs
PublikationsstatusVeröffentlicht - 8 Okt 2021
Veranstaltung2021 IEEE International Symposium on Mixed and Augmented Reality: ISMAR 2021 - Bari, Italien
Dauer: 4 Okt 20218 Okt 2021

Konferenz

Konferenz2021 IEEE International Symposium on Mixed and Augmented Reality
Land/GebietItalien
OrtBari
Zeitraum4/10/218/10/21

Schlagwörter

  • Visualization
  • Runtime
  • Three-dimensional displays
  • Pipelines
  • Mixed reality
  • Coherence
  • Tools

Dieses zitieren