Driver Drowsiness Detection using CNN

Photo by Isaac on Dribbble

Drowsy Driving is a deadly combination of driving and sleepiness. The number of road accidents due to Drowsy Driving is increasing at an alarming rate worldwide. Not having a proper sleep is the main reason behind drowsiness while driving. However, other reasons like sleep disorders, medication, alcohol consumption, or driving during night shifts can also cause drowsiness while driving.

According to a report of AIIMS Neurology India —

Sleep Disorders became the reason behind around more than 20% of all road accidents and around 23% of truck drivers have sleep deprivations.

A separate report by National Highway Traffic Safety Administration, USA, states that —

Drowsy Driving was responsible for around 72,000 crashes, 44,000 injuries and 800 deaths in 2013.

Whatever be the reason for drowsiness, the fatalities due to drowsy driving are increasing every year.

This article presents a solution for driver drowsiness detection using a Convolutional Neural Network. The implementation of the project uses a custom CNN architecture with less than 250K trainable parameters for easy deployment on edge or computationally less efficient devices. As a result, the driver can be alerted at the right time if the system detects that the driver has fallen asleep before anything dangerous happens.

The Cainvas Platform is used for implementation, which provides seamless execution of python notebooks for building AI systems that can eventually be deployed on edge (i.e. an embedded system such as compact MCUs).
The notebook can be found here.

The flow of the article is as follows: –

  • Description of the Problem Statement
  • CNN Model Architecture
  • Drowsiness Detection Dataset
  • Training the Model
  • Performance of the Model
  • Testing the Model
  • Building Pipeline for Predictions on Full Face Images
  • Conclusion

Description of the Problem Statement

The project aims at detecting drowsiness while driving to alert the driver at the right time to prevent any mishappening. The project uses a CNN model to predict whether a person feels drowsy or not based on whether the eyes are closed or open. The project’s main objective was to limit the number of trainable parameters of the CNN model to under 250K so that the system can be deployed on edge or computationally less efficient devices. The project has a direct application in the automobile industry, makes drive safer, and reduces the death toll caused by drowsy driving.

Building the CNN Model

The implementation uses a custom-designed Convolutional Neural Network that has the following characteristics —

  • Three Convolution Blocks having 2, 3, and 3 convolutional layers, respectively.
  • A BatchNormalization Layer follows each Convolution Layer.
  • A Dropout Layer follows each Convolution Block for avoiding overfitting and a MaxPool Layer.
  • 3 Fully Connected Layers follow convolution Layers for classification.

Defining the model using Keras —

https://gist.github.com/Yuvnish017/fd0b30d6d66ce8d51beb71341bfbe6c0

Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1 (Conv2D) (None, 32, 32, 32) 896
_________________________________________________________________
batch_normalization (BatchNo (None, 32, 32, 32) 128
_________________________________________________________________
conv2 (Conv2D) (None, 32, 32, 32) 9248
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 32) 128
_________________________________________________________________
dropout (Dropout) (None, 32, 32, 32) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 16, 16, 32) 0
_________________________________________________________________
conv3 (Conv2D) (None, 16, 16, 64) 18496
_________________________________________________________________
batch_normalization_2 (Batch (None, 16, 16, 64) 256
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 8, 8, 64) 0
_________________________________________________________________
conv4 (Conv2D) (None, 8, 8, 64) 36928
_________________________________________________________________
batch_normalization_3 (Batch (None, 8, 8, 64) 256
_________________________________________________________________
dropout_1 (Dropout) (None, 8, 8, 64) 0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 64) 0
_________________________________________________________________
conv5 (Conv2D) (None, 4, 4, 64) 36928
_________________________________________________________________
batch_normalization_4 (Batch (None, 4, 4, 64) 256
_________________________________________________________________
conv6 (Conv2D) (None, 4, 4, 64) 36928
_________________________________________________________________
batch_normalization_5 (Batch (None, 4, 4, 64) 256
_________________________________________________________________
conv7 (Conv2D) (None, 4, 4, 64) 36928
_________________________________________________________________
batch_normalization_6 (Batch (None, 4, 4, 64) 256
_________________________________________________________________
dropout_2 (Dropout) (None, 4, 4, 64) 0
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 2, 2, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 256) 0
_________________________________________________________________
fc1 (Dense) (None, 128) 32896
_________________________________________________________________
dropout_3 (Dropout) (None, 128) 0
_________________________________________________________________
fc2 (Dense) (None, 128) 16512
_________________________________________________________________
dropout_4 (Dropout) (None, 128) 0
_________________________________________________________________
fc3 (Dense) (None, 2) 258
=================================================================
Total params: 227,554
Trainable params: 226,786
Non-trainable params: 768
_________________________________________________________________

The model was compiled with the Adam optimizer and a learning rate of 0.0001.

Drowsiness Detection Dataset

The project uses the Drowsiness_dataset present on the Kaggle platform. The dataset is present on this link. The original dataset contains four classes for classifying images into Open Eyes, Closed Eyes, Yawning, or No-Yawning. However, this project’s scope is to classify drowsiness based on whether the eyes are closed or open. So, I will be using only two classes of the dataset. Characteristics of the dataset are as follows —

  • The dataset contains a total of 1452 images in two categories.
  • Each category has 726 images.
  • The dataset is already balanced, so no need to balance the dataset.
  • Class Labels — ‘Open Eye’ and ‘Closed Eye’.
  • Class Labels were encoded such that 0 represents Open Eye and 1 illustrates Closed Eye.

Loading the dataset —

https://gist.github.com/Yuvnish017/208ca6ce3c430a1ef8f78fd4ac252b1f

Preprocess the images to make the size of each image equal to (32, 32, 3). Then, the dataset is split into Train and Test Set in 80%-20% proportion.

Training the Model

https://gist.github.com/Yuvnish017/94284b1a4d31da07f33102da08c53dcd

Model training runs for a total of 200 epochs with a batch size of 128. ImageDataGenerator is used for randomizing the training images for better performance of the model.

Performance of the Model

The following Performance Metrics are used —

  • Loss vs Number of Epochs Plot
  • Accuracy vs Number of Epochs Plot
  • Classification Report
  • Confusion Matrix

The results of the metrics mentioned above are as follows —

Accuracy and Loss Plots

Classification Report —

precision    recall  f1-score   support
           0       0.99      0.98      0.99       169
1 0.98 0.99 0.98 122
    accuracy                           0.99       291
macro avg 0.98 0.99 0.99 291
weighted avg 0.99 0.99 0.99 291

Confusion Matrix —

Testing the Model

The predictions of the model on images of eyes can be seen in the following pictures —

Building a Pipeline for Predictions on Full Face Images

This project’s last and final step is to build a pipeline for making predictions on full-face images. The channel includes Face detection, face alignment, eyes detection, preprocessing the ROI of the image, passing to the model for prediction and displaying results on the image.

The implementation is as follows —

https://gist.github.com/Yuvnish017/797163dbbc418433c2869694e8e346dc

The implementation uses the ‘dlib’ library for face detection in the image. Face Alignment is done using FaceAlignment class of imutils.face_utils library for better eye detection. Eye detection is performed using Haar Cascade Classifiers.

Finally, the results can be viewed as follows —

https://gist.github.com/Yuvnish017/c21a21c3c29f4bcf24bdc419033fd27c


Conclusion

Driver Drowsiness is a significant reason for thousands of road accidents all over the world. Driver drowsiness detection is a car safety technology that helps prevent accidents caused by the driver getting drowsy. The project aims at providing a solution of Driver Drowsiness Detection using CNN and image processing. The project aimed at optimizing the model to limit the number of parameters under 250k for easy deployment on edge devices. This deployment is possible through the Cainvas Platform by making use of their compiler called deepC. Thus effectively bringing AI out on edge — in actual and physical real-world use cases.


Notebook link is here.

Credit: YUVNISH MALHOTRA