Image Morphing in Deep Feature Spaces: Theory and Applications

Alexander Effland, Erich Kobler, Thomas Pock, Marko Rajkovic, Martin Rumpf

Research output: Contribution to journalArticleResearchpeer-review

Abstract

This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller, Trouvé, Younes and coworkers. For this model a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space, the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.
Original languageEnglish
JournalarXiv.org e-Print archive
Publication statusPublished - 2019

Fingerprint

Splines
Semantics
Experiments

Cite this

Image Morphing in Deep Feature Spaces: Theory and Applications. / Effland, Alexander; Kobler, Erich; Pock, Thomas; Rajkovic, Marko; Rumpf, Martin.

In: arXiv.org e-Print archive, 2019.

Research output: Contribution to journalArticleResearchpeer-review

@article{17d1a3133af74164a33ac1e4dd6df0d7,
title = "Image Morphing in Deep Feature Spaces: Theory and Applications",
abstract = "This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller, Trouv{\'e}, Younes and coworkers. For this model a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space, the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.",
author = "Alexander Effland and Erich Kobler and Thomas Pock and Marko Rajkovic and Martin Rumpf",
year = "2019",
language = "English",
journal = "arXiv.org e-Print archive",
publisher = "Cornell University Library",

}

TY - JOUR

T1 - Image Morphing in Deep Feature Spaces: Theory and Applications

AU - Effland, Alexander

AU - Kobler, Erich

AU - Pock, Thomas

AU - Rajkovic, Marko

AU - Rumpf, Martin

PY - 2019

Y1 - 2019

N2 - This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller, Trouvé, Younes and coworkers. For this model a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space, the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.

AB - This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller, Trouvé, Younes and coworkers. For this model a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space, the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.

M3 - Article

JO - arXiv.org e-Print archive

JF - arXiv.org e-Print archive

ER -