Masked face recognition on mobile devices using deep learning

Deep Convolution Neural Network (DCNN) based facial recognition has made significant progress in recent years. Currently, facial recognition technology has emerged as an important authentication tool on mobile devices. Hence, a fast and lightweight DCNN model is required to work accurately in limited computing resources. Meanwhile, the outbreak of the COVID-19 pandemic has led to new challenges in face recognition due to the use of facemasks. Therefore, in this study, we develop a masked face recognition application using a lightweight and efficient DCNN, which is applicable to mobile devices. Two networks for face verification tasks, named MobileFaceNet and SeesawFaceNet are explored for this purpose. We train these models on the augmented version of CelebA dataset, which originally is a set of celebrity images. We put synthetic mask on the face images in CelebA to provide a training dataset contain mix of face images with and without mask. The trained models, which are able to recognize people either wearing or not wearing masks, are then retrained on the face dataset commonly used for verification purposes, i.e. LFW (face images without mask) and MFR2 (face images wearing masks). Transfer learning is utilized to improve the network learning ability, and cosine similarity is adopted to quantify the similarity for pairs of examples. In experiment, the SeesawFaceNet model obtains better performance, with 98.8% accuracy on LFW dataset, 96% accuracy on MFR2 masked dataset. In contrast, the experiment after deployment the models on a smartphone application, the MobileFaceNet model is more superior than the SeesawFaceNet with an accuracy of 85%, an average speed of 44 milliseconds, and model size of 4.9 MB.