Time Discrete Geodesics in Deep Feature Spaces for Image Morphing

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Abstract

Image morphing in computer vision amounts to computing a visually appealing transition of two images. A prominent model for image morphing originally proposed by Trouvé, Younes and coworkers is image metamorphosis. Here, the space of images is endowed with a Riemannian metric that separately quantifies the contributions due to transport and image intensity variations along a transport path. Geodesic curves in this Riemannian space of images give rise to morphing transitions. The classical metamorphosis model considers images as square-integrable functions on some image domain and thus is non-sensitive to image features such as sharp interfaces or fine texture patterns. To resolve this drawback, we treat images not as intensity maps, but rather as maps into some feature space that encode local structure information. In the simplest case, color intensities are such feature vectors. To appropriately treat local structures and semantic information, deep convolutional neural network features are investigated. The resulting model is formulated directly in terms of a variational time discretization developed for the classical metamorphosis model by Berkels, Effland and Rumpf. The key ingredient is a mismatch energy that locally approximates the squared Riemannian distance and consists of a regularization energy of the time discrete flow and a dissimilarity energy that measures the feature vector modulation along discrete transport paths. The spatial discretization is based on a finite difference and a stable spline interpolation. A variety of numerical examples demonstrates the robustness and versatility of the proposed method for real images using a variant of the iPALM algorithm for the minimization of the fully discrete energy functional.
Originalspracheenglisch
TitelScale Space and Variational Methods in Computer Vision
UntertitelSSVM 2019
Redakteure/-innenJ. Lellmann, M. Burger, J. Modersitzki
ErscheinungsortCham
Herausgeber (Verlag)Springer
Seiten171-182
ISBN (elektronisch)978-3-030-22368-7
ISBN (Print)978-3-030-22367-0
DOIs
PublikationsstatusVeröffentlicht - 2019
Veranstaltung7th International Conference on Scale Space and Variational Methods in Computer Vision - Hofgeismar, Deutschland
Dauer: 30 Jun 20194 Jul 2019

Publikationsreihe

NameLecture Notes in Computer Science
Band11603

Konferenz

Konferenz7th International Conference on Scale Space and Variational Methods in Computer Vision
KurztitelSSVM 2019
LandDeutschland
OrtHofgeismar
Zeitraum30/06/194/07/19

Fingerprint

Splines
Computer vision
Interpolation
Textures
Semantics
Modulation
Color
Neural networks

Dies zitieren

Effland, A., Kobler, E., Pock, T., & Rumpf, M. (2019). Time Discrete Geodesics in Deep Feature Spaces for Image Morphing. in J. Lellmann, M. Burger, & J. Modersitzki (Hrsg.), Scale Space and Variational Methods in Computer Vision: SSVM 2019 (S. 171-182). (Lecture Notes in Computer Science; Band 11603). Cham: Springer. https://doi.org/10.1007/978-3-030-22368-7_14

Time Discrete Geodesics in Deep Feature Spaces for Image Morphing. / Effland, Alexander; Kobler, Erich; Pock, Thomas; Rumpf, Martin.

Scale Space and Variational Methods in Computer Vision: SSVM 2019. Hrsg. / J. Lellmann; M. Burger; J. Modersitzki. Cham : Springer, 2019. S. 171-182 (Lecture Notes in Computer Science; Band 11603).

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandForschungBegutachtung

Effland, A, Kobler, E, Pock, T & Rumpf, M 2019, Time Discrete Geodesics in Deep Feature Spaces for Image Morphing. in J Lellmann, M Burger & J Modersitzki (Hrsg.), Scale Space and Variational Methods in Computer Vision: SSVM 2019. Lecture Notes in Computer Science, Bd. 11603, Springer, Cham, S. 171-182, Hofgeismar, Deutschland, 30/06/19. https://doi.org/10.1007/978-3-030-22368-7_14
Effland A, Kobler E, Pock T, Rumpf M. Time Discrete Geodesics in Deep Feature Spaces for Image Morphing. in Lellmann J, Burger M, Modersitzki J, Hrsg., Scale Space and Variational Methods in Computer Vision: SSVM 2019. Cham: Springer. 2019. S. 171-182. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-22368-7_14
Effland, Alexander ; Kobler, Erich ; Pock, Thomas ; Rumpf, Martin. / Time Discrete Geodesics in Deep Feature Spaces for Image Morphing. Scale Space and Variational Methods in Computer Vision: SSVM 2019. Hrsg. / J. Lellmann ; M. Burger ; J. Modersitzki. Cham : Springer, 2019. S. 171-182 (Lecture Notes in Computer Science).
@inproceedings{85c82c06eec04869a25dfa05a6f74a73,
title = "Time Discrete Geodesics in Deep Feature Spaces for Image Morphing",
abstract = "Image morphing in computer vision amounts to computing a visually appealing transition of two images. A prominent model for image morphing originally proposed by Trouv{\'e}, Younes and coworkers is image metamorphosis. Here, the space of images is endowed with a Riemannian metric that separately quantifies the contributions due to transport and image intensity variations along a transport path. Geodesic curves in this Riemannian space of images give rise to morphing transitions. The classical metamorphosis model considers images as square-integrable functions on some image domain and thus is non-sensitive to image features such as sharp interfaces or fine texture patterns. To resolve this drawback, we treat images not as intensity maps, but rather as maps into some feature space that encode local structure information. In the simplest case, color intensities are such feature vectors. To appropriately treat local structures and semantic information, deep convolutional neural network features are investigated. The resulting model is formulated directly in terms of a variational time discretization developed for the classical metamorphosis model by Berkels, Effland and Rumpf. The key ingredient is a mismatch energy that locally approximates the squared Riemannian distance and consists of a regularization energy of the time discrete flow and a dissimilarity energy that measures the feature vector modulation along discrete transport paths. The spatial discretization is based on a finite difference and a stable spline interpolation. A variety of numerical examples demonstrates the robustness and versatility of the proposed method for real images using a variant of the iPALM algorithm for the minimization of the fully discrete energy functional.",
author = "Alexander Effland and Erich Kobler and Thomas Pock and Martin Rumpf",
year = "2019",
doi = "10.1007/978-3-030-22368-7_14",
language = "English",
isbn = "978-3-030-22367-0",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "171--182",
editor = "J. Lellmann and M. Burger and J. Modersitzki",
booktitle = "Scale Space and Variational Methods in Computer Vision",

}

TY - GEN

T1 - Time Discrete Geodesics in Deep Feature Spaces for Image Morphing

AU - Effland, Alexander

AU - Kobler, Erich

AU - Pock, Thomas

AU - Rumpf, Martin

PY - 2019

Y1 - 2019

N2 - Image morphing in computer vision amounts to computing a visually appealing transition of two images. A prominent model for image morphing originally proposed by Trouvé, Younes and coworkers is image metamorphosis. Here, the space of images is endowed with a Riemannian metric that separately quantifies the contributions due to transport and image intensity variations along a transport path. Geodesic curves in this Riemannian space of images give rise to morphing transitions. The classical metamorphosis model considers images as square-integrable functions on some image domain and thus is non-sensitive to image features such as sharp interfaces or fine texture patterns. To resolve this drawback, we treat images not as intensity maps, but rather as maps into some feature space that encode local structure information. In the simplest case, color intensities are such feature vectors. To appropriately treat local structures and semantic information, deep convolutional neural network features are investigated. The resulting model is formulated directly in terms of a variational time discretization developed for the classical metamorphosis model by Berkels, Effland and Rumpf. The key ingredient is a mismatch energy that locally approximates the squared Riemannian distance and consists of a regularization energy of the time discrete flow and a dissimilarity energy that measures the feature vector modulation along discrete transport paths. The spatial discretization is based on a finite difference and a stable spline interpolation. A variety of numerical examples demonstrates the robustness and versatility of the proposed method for real images using a variant of the iPALM algorithm for the minimization of the fully discrete energy functional.

AB - Image morphing in computer vision amounts to computing a visually appealing transition of two images. A prominent model for image morphing originally proposed by Trouvé, Younes and coworkers is image metamorphosis. Here, the space of images is endowed with a Riemannian metric that separately quantifies the contributions due to transport and image intensity variations along a transport path. Geodesic curves in this Riemannian space of images give rise to morphing transitions. The classical metamorphosis model considers images as square-integrable functions on some image domain and thus is non-sensitive to image features such as sharp interfaces or fine texture patterns. To resolve this drawback, we treat images not as intensity maps, but rather as maps into some feature space that encode local structure information. In the simplest case, color intensities are such feature vectors. To appropriately treat local structures and semantic information, deep convolutional neural network features are investigated. The resulting model is formulated directly in terms of a variational time discretization developed for the classical metamorphosis model by Berkels, Effland and Rumpf. The key ingredient is a mismatch energy that locally approximates the squared Riemannian distance and consists of a regularization energy of the time discrete flow and a dissimilarity energy that measures the feature vector modulation along discrete transport paths. The spatial discretization is based on a finite difference and a stable spline interpolation. A variety of numerical examples demonstrates the robustness and versatility of the proposed method for real images using a variant of the iPALM algorithm for the minimization of the fully discrete energy functional.

U2 - 10.1007/978-3-030-22368-7_14

DO - 10.1007/978-3-030-22368-7_14

M3 - Conference contribution

SN - 978-3-030-22367-0

T3 - Lecture Notes in Computer Science

SP - 171

EP - 182

BT - Scale Space and Variational Methods in Computer Vision

A2 - Lellmann, J.

A2 - Burger, M.

A2 - Modersitzki, J.

PB - Springer

CY - Cham

ER -