Applying Deep Neural Networks for Multi-level Classification of Driver Drowsiness Using Vehicle-based Measures

Sadegh Arefnezhad, Sajjad Samiee, Arno Eichberger, Matthias Frühwirth, Clemens Kaufmann, Emma Klotz

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Accurate and reliable detection of drivers’ drowsiness is significantly important to prevent drowsiness-related accidents. In order to design a non-obtrusive drowsiness detection system, vehicle-based measures play a vital role. This paper, presents a novel method based on deep neural networks for drowsiness detection in drivers using vehicle-based measures. The proposed method is based on a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN). Five vehicle-based measures including lateral deviation from road centerline, lateral acceleration, yaw rate, steering wheel angle and steering wheel velocity are exploited as network inputs. Long-short term memory (LSTM) and gated recurrent unit (GRU) layers are used as RNN in the structure of the designed deep network and Adam optimization method is applied to train the network. The level of drowsiness is classified in three different classes including awake, moderately drowsy and extremely drowsy. The performance of the proposed method is evaluated on experimental data that were collected from 20 driving sessions in a fixed-base driving simulator. Results show that the designed deep networks outperform the classification accuracy of classical classifiers like support vector machine and k-nearest neighbors. Combination of CNN and LSTM (CNN-LSTM) shows the highest accuracy for detection of these three classes which is equal to 96.0%.
Original languageEnglish
JournalExpert Systems with Applications
Publication statusSubmitted - 23 Jul 2019

Fingerprint

Recurrent neural networks
Wheels
Neural networks
Support vector machines
Accidents
Classifiers
Simulators
Deep neural networks
Long short-term memory

Keywords

  • Deep learning
  • driver drowsiness detection
  • recurrent convolutional networks
  • vehicle-based data

Fields of Expertise

  • Mobility & Production

Cite this

Applying Deep Neural Networks for Multi-level Classification of Driver Drowsiness Using Vehicle-based Measures. / Arefnezhad, Sadegh; Samiee, Sajjad; Eichberger, Arno; Frühwirth, Matthias; Kaufmann, Clemens; Klotz, Emma.

In: Expert Systems with Applications, 23.07.2019.

Research output: Contribution to journalArticleResearchpeer-review

@article{66731c05f30440b49a98f9b384f66e42,
title = "Applying Deep Neural Networks for Multi-level Classification of Driver Drowsiness Using Vehicle-based Measures",
abstract = "Accurate and reliable detection of drivers’ drowsiness is significantly important to prevent drowsiness-related accidents. In order to design a non-obtrusive drowsiness detection system, vehicle-based measures play a vital role. This paper, presents a novel method based on deep neural networks for drowsiness detection in drivers using vehicle-based measures. The proposed method is based on a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN). Five vehicle-based measures including lateral deviation from road centerline, lateral acceleration, yaw rate, steering wheel angle and steering wheel velocity are exploited as network inputs. Long-short term memory (LSTM) and gated recurrent unit (GRU) layers are used as RNN in the structure of the designed deep network and Adam optimization method is applied to train the network. The level of drowsiness is classified in three different classes including awake, moderately drowsy and extremely drowsy. The performance of the proposed method is evaluated on experimental data that were collected from 20 driving sessions in a fixed-base driving simulator. Results show that the designed deep networks outperform the classification accuracy of classical classifiers like support vector machine and k-nearest neighbors. Combination of CNN and LSTM (CNN-LSTM) shows the highest accuracy for detection of these three classes which is equal to 96.0{\%}.",
keywords = "Deep learning, driver drowsiness detection, recurrent convolutional networks, vehicle-based data",
author = "Sadegh Arefnezhad and Sajjad Samiee and Arno Eichberger and Matthias Fr{\"u}hwirth and Clemens Kaufmann and Emma Klotz",
year = "2019",
month = "7",
day = "23",
language = "English",
journal = "Expert Systems with Applications",
issn = "0957-4174",
publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Applying Deep Neural Networks for Multi-level Classification of Driver Drowsiness Using Vehicle-based Measures

AU - Arefnezhad, Sadegh

AU - Samiee, Sajjad

AU - Eichberger, Arno

AU - Frühwirth, Matthias

AU - Kaufmann, Clemens

AU - Klotz, Emma

PY - 2019/7/23

Y1 - 2019/7/23

N2 - Accurate and reliable detection of drivers’ drowsiness is significantly important to prevent drowsiness-related accidents. In order to design a non-obtrusive drowsiness detection system, vehicle-based measures play a vital role. This paper, presents a novel method based on deep neural networks for drowsiness detection in drivers using vehicle-based measures. The proposed method is based on a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN). Five vehicle-based measures including lateral deviation from road centerline, lateral acceleration, yaw rate, steering wheel angle and steering wheel velocity are exploited as network inputs. Long-short term memory (LSTM) and gated recurrent unit (GRU) layers are used as RNN in the structure of the designed deep network and Adam optimization method is applied to train the network. The level of drowsiness is classified in three different classes including awake, moderately drowsy and extremely drowsy. The performance of the proposed method is evaluated on experimental data that were collected from 20 driving sessions in a fixed-base driving simulator. Results show that the designed deep networks outperform the classification accuracy of classical classifiers like support vector machine and k-nearest neighbors. Combination of CNN and LSTM (CNN-LSTM) shows the highest accuracy for detection of these three classes which is equal to 96.0%.

AB - Accurate and reliable detection of drivers’ drowsiness is significantly important to prevent drowsiness-related accidents. In order to design a non-obtrusive drowsiness detection system, vehicle-based measures play a vital role. This paper, presents a novel method based on deep neural networks for drowsiness detection in drivers using vehicle-based measures. The proposed method is based on a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN). Five vehicle-based measures including lateral deviation from road centerline, lateral acceleration, yaw rate, steering wheel angle and steering wheel velocity are exploited as network inputs. Long-short term memory (LSTM) and gated recurrent unit (GRU) layers are used as RNN in the structure of the designed deep network and Adam optimization method is applied to train the network. The level of drowsiness is classified in three different classes including awake, moderately drowsy and extremely drowsy. The performance of the proposed method is evaluated on experimental data that were collected from 20 driving sessions in a fixed-base driving simulator. Results show that the designed deep networks outperform the classification accuracy of classical classifiers like support vector machine and k-nearest neighbors. Combination of CNN and LSTM (CNN-LSTM) shows the highest accuracy for detection of these three classes which is equal to 96.0%.

KW - Deep learning

KW - driver drowsiness detection

KW - recurrent convolutional networks

KW - vehicle-based data

M3 - Article

JO - Expert Systems with Applications

JF - Expert Systems with Applications

SN - 0957-4174

ER -