**Deep Learning in Face Recognition for Attendance System:**

**An Exploratory Study**

\*\*Double blind review, please do not include authors information in this version \*\*

Received Date: 9 July 2022

Accepted Date: \*date

Published Date: \*date

**HIGHLIGHTS**

  - The use and the development of face recognition is advancing.

  - Deep-learning-based approach was proposed to be implemented in face recognition.

  - The data collection was done by interviewing an artificial intelligence expert.

  - Identified suitable algorithm for face recognition.

**ABSTRACT**

*Conventional-manual type of attendance systems can be very time-consuming to some extent, particularly for a significant number. The existence of face recognition technology can solve the inefficiency and ineffectiveness of conventional and manual attendance systems. Among many approaches to implement face recognition, this research focuses on using deep learning approaches as it has been proven to give promising results. There are various algorithms for face recognition, such as Local Binary Pattern Histogram (LBPH), Local Binary Pattern Network (LBPn), Haar Cascade, and Convolutional Neural Network. The use of deep learning can reach 98 percent accuracy. However, it is necessary to conduct further research on its implementation on the real system in order to evaluate the efficiency of the system. An interview was conducted with an expert in the field, to understand the concept, trend, and use of deep learning in face recognition, as well as to determine the suitable algorithm for attendance system. This paper presents the results from this interview, which provide an insight based on real practices.*

***Keywords:** face recognition, deep learning, attendance system*

# INTRODUCTION 

Conventionally, the attendance checking will be done manually either from the attendees, or the attendance checker who will check the attendance based on at least the existence of the person physically in a venue, e.g., classroom or lecture hall. Of course, this way sounds to be inefficient in the matter of time consumption and operational work in a circumstance where the people are in significant numbers. There is a proposed idea to solve the complexity of taking attendance in certain circumstances by using a smart biometric attendance system. One example of a biometric attendance system is Face Recognition based attendance system. This type of biometric will check the attendance automatically by scanning the face of the attendees using artificial intelligence.

Face recognition is one of a biometric-based artificial intelligence products that contributes in various fields of application in today’s digital world such as in security and law enforcement and commercial retail services. The trend of the use of face recognition is rapidly increasing over the years. Researchers around the world are racing in developing face recognition with various algorithms in order to get the optimum result. Over the years, one of the best approaches that have been used is using deep learning models. Deep Learning is a branch of artificial intelligence with a wide scope of knowledge and disciplines, therefore the implementation of deep learning as face recognition should apply adequate knowledge and discipline of related fields. 

This research aims to discover and develop the understanding of the implementation of deep learning in face recognition. Furthermore, this paper will discuss the contribution of deep learning to face recognition as well as the scope and its limitations as the attendance system.

The cost of the time and its efficiency of conventional attendance systems is one of the problems that can be solved by using face recognition-based attendance systems. Although the trend of the development of face recognition with deep learning is rapidly advancing, yet the trend of its implementation as an attendance system in real scenario settings is still uncommon.

There are many methods in implementing deep learning in face recognition, however different methods can give different output. Technically, not all the methods can be used considering its effectiveness and its efficiency on the system. Therefore, the selection of the method should also be concerned.

The objectives of this research are twofold: to gain qualitative insights on the concept, the trend, and the use of deep learning in face recognition; and to determine the most convenient algorithm of the deep learning for face recognition, suitable for attendance system.

# RELATED WORKS 

In artificial intelligence, deep learning is the subset of artificial intelligence where it applies the concept of artificial neural networks (ANN) that is inspired by the biological neuron networks (Teoh et al., 2021). It is also known as deep structured learning that extends as a new form of machine learning. Computationally, the representation data will be processed through multiple layers of nonlinear processing that is called ANN in order to learn. Deep learning can be classified into three major classes which are unsupervised learning, supervised learning, and hybrid deep network (Deng, 2014). One of the major advantages of deep learning is that it allows to train a huge number of datasets that can give more accurate results (Trigueros et al., 2018).

Face recognition is an artificial intelligence technology that applies the principles of biometric technology approaches by observing and matching the given facial profile according to the pre-given data set. Since the beginning of the invention of the face recognition concept, it has led experts and researchers to come up with the application of face recognition in numerous real-world settings such as in personal identification and authentication, where in this case it is also applicable for the attendance system (Kar et al., 2012). Hence, face recognition has become one of the major focuses of modern technology research and development.

Face recognition is built on multiple disciplines of mathematics and computer science studies with various approaches with different performance. In the early attempt of creating modern face recognition, a machine learning based approach was introduced called eigenface by using principal components to identify facial profile in 1988 (Tolba et al., 2006). However, despite the succession of eigenfaces in identifying facial profiles and their identities, the performance is not efficient and unsatisfactory due to its limitation in

recognition under different circumstances.

Another more modernised approach is called DeepFace, which uses a deep learning framework to recognize facial profiles. The proposed approach modifies the recognition by using 3D modelling techniques in order to identify the facial profile precisely as similar as human beings (Taigman et al., 2014). The study found that the DeepFace system reaches 97.35 percent accuracy which is only 0.28 percent less than human capability in recognizing facial profiles. Considering the performance of deep learning-based face recognition, therefore this research aims to determine the performance and the use of deep learning in facial recognition-based attendance systems.

Patel et al. (2018) Proposed system uses the OpenCV library. OpenCv is a computer vision library that provides multiple language interfaces such as Python, Java, and C++ that can be operated in various operating systems such as Windows, MacOS, Linux, iOS, and Android. The proposed computer vision library provides more than 2500 optimised algorithms for face recognition and detection.

# METHODOLOGY 

Based on expertise required to provide better insights on this research, as well as the need to get consent for taking time off the potential respondent, the convenience sampling technique is used to identify the right expert to be interviewed. An interview was conducted, in which descriptive data is collected during the interview. In terms of ethical consideration, the consent of the respondent is sought after before an appointment was made for the interview session.

The interview was conducted with the Head of Smart DigitalLab Unit, who also holds a position as the Coordinator of Postgraduate Study of Computer Science under the School of Computer Science and Engineering, in University Malaysia of Computer Science and Engineering. The respondent is known as the expert in artificial intelligence, in which she had a convincing number of related works in this area of research, which include experiences and publications on face recognition systems. The interview design was set as a semi-structured interview where the respondent was asked a set of open-ended questions. The interview session took approximately 39 minutes, and it was conducted online due to the locations and distance between the researcher and the respondent. The online interview was conducted via Zoom teleconference platform on Friday, 21st May 2022.

The data analysis was conducted using content analysis on the transcription of the recorded interview. The transcribed interview data was examined and summarised correspondingly to the topic of the interview. Every content was examined to achieve the objective of the research. By conducting an interview, the researcher was able to get better understanding from the expert of the related field by having deep conversation. Through the interview, the researcher was able to eliminate any confusion and misconceptions on certain aspects of the discussion.

**FINDINGS**

**Perception Towards the Trend of The Use of Face Recognition **

When the respondent was asked about the perception towards the trend of the use of face recognition, the respondent explained that since the first time the concept of face recognition was introduced in 1987, face recognition began to develop and be used for certain purposes such as in military purposes. At the current stage, the respondent believes that everyone wants to equip that technology as the inventions evolve.

However, the respondent tended to believe that the definition as a completely effective and efficient way or implementation cannot be embedded in face recognition since either face recognition or any other offered detection methods or any application will have their own pros and cons.

The respondent also added that indeed, face recognition has its vulnerabilities and issues. Every single embedded technology, regardless of what type of the application and how it is implemented, each of them has vulnerabilities as well as weaknesses. However, the possible issues in face recognition such as in low accuracy can be handled by correcting the certain method such as in choosing the integrated algorithm that is used.

Finally, the respondent stated that the presence and the use of face recognition has changed and began to be widely revolutionised to be implemented in wider applications over the time. It has not only been used for the comprehensive system, it is even being used now for the mobile application, which is not only limited to security purposes, but also to provide media-entertainment features such as in the enhancement of the face recognition in the camera for the users.

**Deep Learning Algorithm in Face Recognition**

According to the definition that is stated by the respondent, deep learning is actually an algorithm, where the root of deep learning itself is an algorithm. The respondent added that the researchers consider that deep learning comes from machine learning. However, the respondent tended to consider that Machine learning is a different thing from deep learning whereas it only has the similarity of the concept between both where there is an overlapped definition that arise among the people between machine learning and deep learning. Finally, the respondent stated that deep learning does not come from machine learning.

The goal of deep learning is to achieve highest accuracy. It is designed to find the best among the best. The respondent added that the terms Optimization Algorithm can be used to define deep learning. In its process, the respondent stated that multiple algorithms can be used instead using specific algorithms in Deep Learning. For instance, the respondent explains the implementation of Neuro-Fuzzy Logic where the algorithm is trained within multiple layers of Neuro-Fuzzy Logic. When they are applied, the algorithms are not standalone anymore, it is combined to generate new findings. The combination of the algorithm is meant to try to find the best output.

In face recognition, the combination of algorithms is used in order to get clearer detection and recognition. Examples of the algorithm that is commonly used in face recognition are CSR algorithm, Eigenface, LBPH, and Vector Algorithm. However, the respondent stated that the combination of algorithms in Deep Learning is not widely used for physical systems settings such as in face recognition.

The respondent added, in the physical system, in this case we don’t combine the algorithm of deep learning but in fact combine algorithms of the parameters to try to optimise the parameters. Hence technically the respondent stated that it is not the deep learning algorithm that is combined but the algorithm of the parameters optimization that is combined. The respondent said that precisely Deep Learning is placed on the optimization of the parameters in each algorithm according to her paper.

**Face Recognition Based Attendance System Tools and Methods**

According to the respondent, the respondent suggested using LBPH as the proposed algorithm and OpenCV-Python as the vision library. Local Binary Pattern Histogram (LBP) is known to be one of the popular algorithms for physical systems, particularly for face recognition devices due to its simplicity. Face Recognition can be designated for detecting motion and emotion, however according to the respondent LBPH is only capable in detecting motion. The system architecture of the system can be various depending on the intended application, for instance the respondent proposed a web-application based system where the face recognition is integrated with the PHPmyAdmin as the web server and MySQL as the database.

**DISCUSSION**

Initially, the face recognition idea was introduced in 1987 by using principal component analysis which was later used as the fundamentals of the first face recognition technology which is known as Eigenfaces (Turk & Pentland, 1991). Since then, the use began to widen, and development of face recognition began to advance especially with the existence of deep learning methods (Guo & Zhang, 2019). One of the recent achievements of face recognition development is using a deep learning-based algorithm that can precisely recognize humans’ faces that reaches 97 percent of the accuracy (Taigman et al., 2014). Due to its advantages and capabilities to be used in many applications, the use of face recognition keeps increasing while the development of face recognition is even more rapid. According to the findings from the respondent's perspective, it is to be believed that nowadays face recognition technology has begun to be highly demanded. Accordingly, previous research found that approximately market on face recognition to be growth at 14 percent of the compound annual growth rate in 2027 (Grand View Research, 2020). It is obvious that the trend of face recognition is increasing over the years.

According to the findings, there are various algorithms that can be used in face recognition, one of the algorithms that can be used is Local Binary Pattern Histograms (LBPH). In face recognition, LBPH is commonly used to define the face by extracting facial features (ST et al., 2022). LBPH is known to be one of the old face recognition algorithms that requires less computational time and is easy to implement. However, there is no affirmative documentation that states that LBPH is part of deep learning, yet the use of LBPH standalone is a bit simpler, while deep learning has more complex architecture (Farfade et al., 2015). Besides that, the comparison of the result of LBPH stand alone with the deep learning approaches also can be a factor that makes LBPH and deep learning distinguished. In an experiment, LBPH reached 63 percent of accuracy, while CNN (Convolutional Neural Network) which is a deep learning algorithm reached 99.88 percent of accuracy. However, it requires a huge scale of datasets of the samples in order to get excellent performance (Pei et al., 2019).

Although the LBPH still can be categorised as deep learning algorithms if it is combined or implemented with the deep learning environment. For instance, (Xi et al., 2016) proposed an algorithm named Local Binary Pattern Network (LBPNet) using the Local Binary Pattern operator and CNN as the principles. It focuses on increasing the efficiency of extracting and comparing the features. Similarly, Dusa and Phulpagar (2020) proposed a method of a two combination of Deep Neural Network as the detection and LBPH algorithms as the training data set, showing better accuracy compared to Haar Cascade algorithm.

Furthermore, the deep learning algorithm can be obtained by using the DNN module in OpenCV and Histogram Oriented Gradient (HOG) – CNN in dlib. OpenCv is a cross-platform vision library that can be used to implement the overall system of face recognition-based attendance system, while dlib can be used for the face recognition by using *face\_recognition* library, a deep learning based face recognition library. It is to be claimed that it reached 98 percent accuracy (Patel et al., 2018).

**CONCLUSION**

The use of deep learning in face recognition-based attendance systems is more likely applicable considering the trend of face recognition is increasing, yet it is still uncommon in wide use. Although deep learning brings more excellent results in face recognition, it requires a huge amount of samples of the faces and datasets storage. Hence, it is important to create proper planning on creating an efficient face recognition-based attendance system. In order to get a clearer understanding and more efficient system it is essential to conduct deeper research on face recognition algorithms. Furthermore, it is also important to have a test on the implementation of the face recognition on the system in order to analyse its efficiency and performance.

**REFERENCES**

Alhanaee, K., Alhammadi, M., Almenhali, N., & Shatnawi, M. (2021). Face recognition smart attendance system using Deep Transfer Learning. Procedia Computer Science, 192, 4093–4102. https://doi.org/10.1016/j.procs.2021.09.184

Arsenovic, M., Sladojevic, S., Anderla, A., & Stefanovic, D. (2017). FaceTime — Deep learning based face recognition attendance system. 2017 IEEE 15th International Symposium on Intelligent Systems and Informatics (SISY). https://doi.org/10.1109/sisy.2017.8080587

Arya, S., Pratap, N., & Bhatia, K. (2015). Future of face recognition: A Review. Procedia Computer Science, 58, 578–585. https://doi.org/10.1016/j.procs.2015.08.076

Balaban, S. (2015). Deep learning and face recognition: The state of the art. SPIE Proceedings. https://doi.org/10.1117/12.2181526

Beham, M. P., \& Roomi, S. M. (2013). A review of face recognition methods. International Journal of Pattern Recognition and Artificial Intelligence, 27(04), 1356005. https://doi.org/10.1142/s0218001413560053

CBS Interactive. (2014, March 19). Facebook's Deepface shows serious facial recognition skills. CBS News. Retrieved March 25, 2022, from https://www.cbsnews.com/news/facebooks-deepface-shows-serious-facial-recognition-skills/

Duong, C. N., Quach, K. G., Jalata, I., Le, N., \& Luu, K. (2019). MobiFace: A lightweight deep learning face recognition on mobile devices. 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS). https://doi.org/10.1109/btas46853.2019.9185981

Dusa, P., \& Phulpagar, B. D. (2020). Criminal Detection Using OpenCV DNN And OpenCV LBPH Methods. Aegaeum Journal, 8(8).

Ghazi, M. M., &; Ekenel, H. K. (2016). A comprehensive analysis of deep learning based representation for face recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw.2016.20

Grand View Research. (2020). Facial Recognition Market Size, Share & Trends Analysis Report By Technology (2D, 3D, Facial Analytics), By Application (Access Control, Security & Surveillance), By End-use, By Region, And Segment Forecasts, 2021 - 2028. https://www.grandviewresearch.com/industry-analysis/facial-recognition-market

Gross, C. G., & Sergent, J. (1992). Face recognition. Current Opinion in Neurobiology, 2(2), 156–161. https://doi.org/10.1016/0959-4388(92)90004-5

Guo, G., & Zhang, N. (2019). A survey on deep learning based face recognition. Computer vision and image understanding, 189, 102805.

Hu, G., Yang, Y., Yi, D., Kittler, J., Christmas, W., Li, S. Z., \& Hospedales, T. (2015). When face recognition meets with deep learning: An evaluation of Convolutional Neural Networks for face recognition. 2015 IEEE International Conference on Computer Vision Workshop (ICCVW). https://doi.org/10.1109/iccvw.2015.58

Ismail, S., \& Ismail, S. (2022). A preliminary study of cashless payment face recognition system development in Malaysia. 2022 16th International Conference on Ubiquitous Information Management and Communication (IMCOM). https://doi.org/10.1109/imcom53663.2022.9721723

Kar, N., Debbarma, M. K., Saha, A., & Pal, D. R. (2012). Study of implementing automated attendance system using face recognition technique. International Journal of Computer and Communication Engineering, 100–103. https://doi.org/10.7763/ijcce.2012.v1.28

Kawaguchi, K., Kaelbling, L. P., & Bengio, Y. (2017). Generalization in deep learning. arXiv preprint arXiv:1710.05468.

Kawaguchi, Yohei & Shoji, Tetsuo. (2005). Face Recognition-based Lecture Attendance System.

Khan, M., Hussian, Z. M., Khan, S., Khan, S., \& Pathan, N. (2016). (rep.). Automated attendance system using face recognition. Navi Mumbai, India: AIKTC. Retrieved from http://www.aiktcdspace.org:8080/jspui/handle/123456789/1574.

LeCun, Y., Bengio, Y., \& Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539

Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition. Proceedings of the British Machine Vision Conference 2015. https://doi.org/10.5244/c.29.41

Patel, S., Kumar, P., Garg, S., & Kumar, R. (2018). Face recognition based smart attendance system using IOT. International Journal of Computer Sciences and Engineering, 6(5), 871–877. https://doi.org/10.26438/ijcse/v6i5.871877

Pei, Z., Xu, H., Zhang, Y., Guo, M., \& Yang, Y.-H. (2019). Face recognition via deep learning using data augmentation based on orthogonal experiments. Electronics, 8(10), 1088. https://doi.org/10.3390/electronics8101088

Sai, E. C. (2021). Student Attendance Monitoring System using face recognition. International Journal for Research in Applied Science and Engineering Technology, 9(5), 2023–2029. https://doi.org/10.22214/ijraset.2021.34749

Shepley, A. J. (2019). Deep learning for face recognition: a critical analysis. arXiv preprint arXiv:1907.12739.

Smitha, Hegde, P. S., \& Afshin. (2020). Face recognition based attendance management system. International Journal of Engineering Research and Technical Research, V9(05). https://doi.org/10.17577/ijertv9is050861

ST, S., Ayoobkhan, M. U., V, K. K., Bacanin, N., K, V., Štěpán, H., \& Pavel, T. (2022). Deep learning model for deep fake face recognition and detection. PeerJ Computer Science, 8. https://doi.org/10.7717/peerj-cs.881

Sun, Y., Wang, X., \& Tang, X. (2014). Deep learning face representation from predicting 10,000 classes. 2014 IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2014.244

Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2014.220

Tam, A. (2021, October 30). Face recognition using principal component analysis. Machine Learning Mastery. Retrieved March 25, 2022, from https://machinelearningmastery.com/face-recognition-using-principal-component-analysis/

Tolba, A. S., El-Baz, A. H., & El-Harby, A. A. (2006). Face recognition: A literature review. International Journal of Signal Processing, 2(2), 88-103.

Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86. https://doi.org/10.1162/jocn.1991.3.1.71

Wang, L., \& Siddique, A. A. (2020). Facial recognition system using LBPH face recognizer for anti-theft and surveillance application based on drone technology. Measurement and Control, 53(7-8), 1070–1077. https://doi.org/10.1177/0020294020932344

Xi, M., Chen, L., Polajnar, D., \& Tong, W. (2016). Local Binary Pattern Network: A deep learning approach for face recognition. 2016 IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/icip.2016.7532955

Zhao, W., Chellappa, R., Phillips, P. J., \& Rosenfeld, A. (2003). Face recognition. ACM Computing Surveys, 35(4), 399–458. https://doi.org/10.1145/954339.954342