AdaNeRF: Adaptive Sampling for Real-Time Rendering of Neural Radiance Fields

Andreas Kurz*, Thomas Neff, Zhaoyang Lv, Michael Zollhöfer, Markus Steinberger

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Novel view synthesis has recently been revolutionized by learning neural radiance fields directly from sparse observations. However, rendering images with this new paradigm is slow due to the fact that an accurate quadrature of the volume rendering equation requires a large number of samples for each ray. Previous work has mainly focused on speeding up the network evaluations that are associated with each sample point, e.g., via caching of radiance values into explicit spatial data structures, but this comes at the expense of model compactness. In this paper, we propose a novel dual-network architecture that takes an orthogonal direction by learning how to best reduce the number of required sample points. To this end, we split our network into a sampling and shading network that are jointly trained. Our training scheme employs fixed sample positions along each ray, and incrementally introduces sparsity throughout training to achieve high quality even at low sample counts. After fine-tuning with the target number of samples, the resulting compact neural representation can be rendered in real-time. Our experiments demonstrate that our approach outperforms concurrent compact neural representations in terms of quality and frame rate and performs on par with highly efficient hybrid representations. Code and supplementary material is available at https://thomasneff.github.io/adanerf.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022 - 17th European Conference, Proceedings
EditorsShai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
Place of PublicationCham
PublisherSpringer Nature Switzerland AG
Pages254-270
Number of pages17
ISBN (Print)9783031197895
DOIs
Publication statusPublished - 2022
Event2022 European Conference on Computer Vision: ECCV 2022 - Hybrider Event, Israel
Duration: 23 Oct 202227 Oct 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13677 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference2022 European Conference on Computer Vision
Abbreviated titleECCV 2022
Country/TerritoryIsrael
CityHybrider Event
Period23/10/2227/10/22

Keywords

  • Neural radiance fields
  • Neural rendering
  • View synthesis

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'AdaNeRF: Adaptive Sampling for Real-Time Rendering of Neural Radiance Fields'. Together they form a unique fingerprint.

Cite this