Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Demosaicing is an important first step for color image acquisition. For practical reasons, demosaicing algorithms have to be both efficient and yield high quality results in the presence of noise. The demosaicing problem poses several challenges, e.g. zippering and false color artifacts as well as edge blur. In this work, we introduce a novel learning based method that can overcome these challenges. We formulate demosaicing as an image restoration problem and propose to learn efficient regularization inspired by a variational energy minimization framework that can be trained for different sensor layouts. Our algorithm performs joint demosaicing and denoising in close relation to the real physical mosaicing process on a camera sensor. This is achieved by learning a sequence of energy minimization problems composed of a set of RGB filters and corresponding activation functions. We evaluate our algorithm on the Microsoft Demosaicing data set in terms of peak signal to noise ratio (PSNR) and structured similarity index (SSIM). Our algorithm is highly efficient both in image quality and run time. We achieve an improvement of up to 2.6 dB over recent state-of-the-art algorithms.
LanguageEnglish
Title of host publicationIEEE International Conference on Computational Photography (ICCP)
DOIs
StatusPublished - 13 May 2016
EventInternational Conference of Computational Photography - Evanston, IL, United States
Duration: 13 May 201615 May 2016

Conference

ConferenceInternational Conference of Computational Photography
CountryUnited States
CityEvanston, IL
Period13/05/1615/05/16

Fingerprint

Color
Image acquisition
Sensors
Image reconstruction
Image quality
Signal to noise ratio
Chemical activation
Cameras

Cite this

Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization. / Klatzer, Teresa; Hammernik, Kerstin; Knöbelreiter, Patrick; Pock, Thomas.

IEEE International Conference on Computational Photography (ICCP). 2016.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Klatzer, T, Hammernik, K, Knöbelreiter, P & Pock, T 2016, Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization. in IEEE International Conference on Computational Photography (ICCP). International Conference of Computational Photography, Evanston, IL, United States, 13/05/16. DOI: 10.1109/ICCPHOT.2016.7492871
Klatzer T, Hammernik K, Knöbelreiter P, Pock T. Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization. In IEEE International Conference on Computational Photography (ICCP). 2016. Available from, DOI: 10.1109/ICCPHOT.2016.7492871
@inproceedings{54218a4143a3421bbe60a78e5eef4814,
title = "Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization",
abstract = "Demosaicing is an important first step for color image acquisition. For practical reasons, demosaicing algorithms have to be both efficient and yield high quality results in the presence of noise. The demosaicing problem poses several challenges, e.g. zippering and false color artifacts as well as edge blur. In this work, we introduce a novel learning based method that can overcome these challenges. We formulate demosaicing as an image restoration problem and propose to learn efficient regularization inspired by a variational energy minimization framework that can be trained for different sensor layouts. Our algorithm performs joint demosaicing and denoising in close relation to the real physical mosaicing process on a camera sensor. This is achieved by learning a sequence of energy minimization problems composed of a set of RGB filters and corresponding activation functions. We evaluate our algorithm on the Microsoft Demosaicing data set in terms of peak signal to noise ratio (PSNR) and structured similarity index (SSIM). Our algorithm is highly efficient both in image quality and run time. We achieve an improvement of up to 2.6 dB over recent state-of-the-art algorithms.",
author = "Teresa Klatzer and Kerstin Hammernik and Patrick Kn{\"o}belreiter and Thomas Pock",
year = "2016",
month = "5",
day = "13",
doi = "10.1109/ICCPHOT.2016.7492871",
language = "English",
booktitle = "IEEE International Conference on Computational Photography (ICCP)",

}

TY - GEN

T1 - Learning Joint Demosaicing and Denoising Based on Sequential Energy Minimization

AU - Klatzer,Teresa

AU - Hammernik,Kerstin

AU - Knöbelreiter,Patrick

AU - Pock,Thomas

PY - 2016/5/13

Y1 - 2016/5/13

N2 - Demosaicing is an important first step for color image acquisition. For practical reasons, demosaicing algorithms have to be both efficient and yield high quality results in the presence of noise. The demosaicing problem poses several challenges, e.g. zippering and false color artifacts as well as edge blur. In this work, we introduce a novel learning based method that can overcome these challenges. We formulate demosaicing as an image restoration problem and propose to learn efficient regularization inspired by a variational energy minimization framework that can be trained for different sensor layouts. Our algorithm performs joint demosaicing and denoising in close relation to the real physical mosaicing process on a camera sensor. This is achieved by learning a sequence of energy minimization problems composed of a set of RGB filters and corresponding activation functions. We evaluate our algorithm on the Microsoft Demosaicing data set in terms of peak signal to noise ratio (PSNR) and structured similarity index (SSIM). Our algorithm is highly efficient both in image quality and run time. We achieve an improvement of up to 2.6 dB over recent state-of-the-art algorithms.

AB - Demosaicing is an important first step for color image acquisition. For practical reasons, demosaicing algorithms have to be both efficient and yield high quality results in the presence of noise. The demosaicing problem poses several challenges, e.g. zippering and false color artifacts as well as edge blur. In this work, we introduce a novel learning based method that can overcome these challenges. We formulate demosaicing as an image restoration problem and propose to learn efficient regularization inspired by a variational energy minimization framework that can be trained for different sensor layouts. Our algorithm performs joint demosaicing and denoising in close relation to the real physical mosaicing process on a camera sensor. This is achieved by learning a sequence of energy minimization problems composed of a set of RGB filters and corresponding activation functions. We evaluate our algorithm on the Microsoft Demosaicing data set in terms of peak signal to noise ratio (PSNR) and structured similarity index (SSIM). Our algorithm is highly efficient both in image quality and run time. We achieve an improvement of up to 2.6 dB over recent state-of-the-art algorithms.

U2 - 10.1109/ICCPHOT.2016.7492871

DO - 10.1109/ICCPHOT.2016.7492871

M3 - Conference contribution

BT - IEEE International Conference on Computational Photography (ICCP)

ER -