L*ReLU: Piece-wise linear activation functions for deep fine-grained visual categorization

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem Konferenzband

Abstract

Deep neural networks paved the way for significant improvements in image visual categorization during the last years. However, even though the tasks are highly varying, differing in complexity and difficulty, existing solutions mostly build on the same architectural decisions. This also applies to the selection of activation functions (AFs), where most approaches build on Rectified Linear Units (ReLUs). In this paper, however, we show that the choice of a proper AF has a significant impact on the classification accuracy, in particular, if fine, subtle details are of relevance. Therefore, we propose to model the degree of absence and the degree presence of features via the AF by using piece-wise linear functions, which we refer to as L*ReLU. In this way, we can ensure the required properties, while still inheriting the benefits in terms of computational efficiency from ReLUs. We demonstrate our approach for the task of Fine-grained Visual Categorization (FGVC), running experiments on seven different benchmark datasets. The results do not only demonstrate superior results but also that for different tasks, having different characteristics, different AFs are selected.
Originalspracheenglisch
TitelProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
Seiten1207-1216
Seitenumfang10
ISBN (elektronisch)978-1-7281-6553-0
DOIs
PublikationsstatusVeröffentlicht - 1 Mär 2020
Veranstaltung2020 IEEE/CVF Winter Conference on Applications of Computer Vision: WACV 2020 - Snowmass Village, USA / Vereinigte Staaten
Dauer: 1 Mär 20205 Mär 2020

Publikationsreihe

NameProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Konferenz

Konferenz2020 IEEE/CVF Winter Conference on Applications of Computer Vision
KurztitelWACV 2020
LandUSA / Vereinigte Staaten
OrtSnowmass Village
Zeitraum1/03/205/03/20

Schlagwörter

  • neural networks
  • Fine-grained Visual Categorizatio
  • activation function
  • L*ReLU

ASJC Scopus subject areas

  • !!Computer Vision and Pattern Recognition
  • !!Computer Science Applications

Fields of Expertise

  • Information, Communication & Computing

Dieses zitieren