Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Yunjin Chen, Thomas Pock

Research output: Contribution to journalArticle

Abstract

Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD - Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.

LanguageEnglish
Article number7527621
Pages1256-1272
Number of pages17
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume39
Issue number6
DOIs
StatusPublished - 1 Jun 2017

Fingerprint

Image Restoration
Reaction-diffusion
Image reconstruction
Nonlinear Model
Diffusion Model
Nonlinear Diffusion
Influence Function
Reaction-diffusion Model
Linear Filter
Image Denoising
Super-resolution
Parallel Computation
Computer Vision
Simplicity
Image denoising
Framework
Filter
Computer vision
Demonstrate
Experiment

Keywords

  • image denoising
  • image super resolution
  • JPEG deblocking
  • loss specific training
  • Nonlinear reaction diffusion

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

Trainable Nonlinear Reaction Diffusion : A Flexible Framework for Fast and Effective Image Restoration. / Chen, Yunjin; Pock, Thomas.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, 7527621, 01.06.2017, p. 1256-1272.

Research output: Contribution to journalArticle

@article{eea2fd68955b4fe39a994e61ce15d060,
title = "Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration",
abstract = "Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD - Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.",
keywords = "image denoising, image super resolution, JPEG deblocking, loss specific training, Nonlinear reaction diffusion",
author = "Yunjin Chen and Thomas Pock",
year = "2017",
month = "6",
day = "1",
doi = "10.1109/TPAMI.2016.2596743",
language = "English",
volume = "39",
pages = "1256--1272",
journal = "IEEE transactions on pattern analysis and machine intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "6",

}

TY - JOUR

T1 - Trainable Nonlinear Reaction Diffusion

T2 - IEEE transactions on pattern analysis and machine intelligence

AU - Chen,Yunjin

AU - Pock,Thomas

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD - Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.

AB - Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD - Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.

KW - image denoising

KW - image super resolution

KW - JPEG deblocking

KW - loss specific training

KW - Nonlinear reaction diffusion

UR - http://www.scopus.com/inward/record.url?scp=85019248552&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2016.2596743

DO - 10.1109/TPAMI.2016.2596743

M3 - Article

VL - 39

SP - 1256

EP - 1272

JO - IEEE transactions on pattern analysis and machine intelligence

JF - IEEE transactions on pattern analysis and machine intelligence

SN - 0162-8828

IS - 6

M1 - 7527621

ER -