**Deep Learning in Face Recognition for Attendance System:  
An Exploratory Study**

Mochamad Azkal Azkiya Aziz <sup>1</sup>, Shahrinaz Ismail <sup>2</sup>\*, Noormadinah Allias <sup>3</sup>

<sup>1,2,3</sup> School of Computing and Informatics, Albukhary International University, Alor Setar, Kedah, Malaysia

Corresponding author: \* shahrinaz.ismail@aiu.edu.my

Received Date: \*date

Accepted Date: \*date

Published Date: \*date

**HIGHLIGHTS**

  - The use and the development of face recognition is advancing.

  - Deep-learning-based approach was proposed to be implemented in face recognition.

  - The findings were analysed from an interview with an artificial intelligence expert.

  - Local Binary Pattern Histogram based algorithm for deep learning approach.

ABSTRACT

*Conventional-manual type of attendance systems can be very time-consuming to some extent, particularly for a significant number. The existence of face recognition technology can solve the inefficiency and ineffectiveness of conventional and manual attendance systems. Among many approaches to implement face recognition, this research focuses on using deep learning approaches as it has been proven to give promising results. There are various algorithms for face recognition, such as Local Binary Pattern Histogram (LBPH), Local Binary Pattern Network (LBPn), Haar Cascade, and Convolutional Neural Network. The use of deep learning can reach 98 percent accuracy. However, it is necessary to conduct further research on its implementation on the real system in order to evaluate the efficiency of the system. An interview was conducted with an expert in the field, to understand the concept, trend, and use of deep learning in face recognition, as well as to determine the suitable algorithm for the attendance system. This paper presents the results from this interview, which provide an insight based on real practices.*

*Keywords: face recognition, deep learning, attendance system, Local Binary Pattern Histogram, Local Binary Network*

# INTRODUCTION

Conventionally, the attendance checking will be done manually either from the attendees, or the attendance checker who will check the attendance based on at least the existence of the person physically in a venue, e.g., classroom or lecture hall. Of course, this way sounds to be inefficient in the matter of time consumption and operational work in a circumstance where the people are in significant numbers. There is a proposed idea to solve the complexity of taking attendance in certain circumstances by using a smart biometric attendance system. One example of a biometric attendance system is face recognition-based attendance system. This type of biometric system will check the attendance automatically by scanning the face of the attendees using artificial intelligence.

Face recognition is one of a biometric-based artificial intelligence products that contributes in various fields of application in today’s digital world, such as in security and law enforcement, as well as commercial retail services. The trend of the use of face recognition is rapidly increasing over the years. Researchers around the world are racing in developing face recognition with various algorithms in order to get the optimum result. Over the years, one of the best approaches that have been used is the deep learning models. According to Setiowati et al. (2017) in a comparative study of deep learning and non-deep learning approaches for face recognition, it shows that deep learning is capable of solving the high complexity input. Necessarily, deep learning has overall better performance in terms of recognition rate compared to the non-deep learning approaches. Thus, deep learning is advantageous for face recognition. Deep learning is a branch of artificial intelligence with a wide scope of knowledge and disciplines, therefore the implementation of deep learning as face recognition should apply adequate knowledge and discipline of related fields.

The cost of the time and its efficiency of conventional attendance systems is one of the problems that can be solved by using face recognition-based attendance systems. Although the trend of the development of face recognition with deep learning is rapidly advancing, yet the trend of its implementation as an attendance system in real scenario settings is still uncommon.

There are many methods in implementing deep learning in face recognition, however different methods can give different output. Technically, not all the methods can be used considering its effectiveness and its efficiency on the system. Therefore, the selection of the method should also be concerned.

This research aims to discover and develop the understanding of the implementation of deep learning in face recognition. Furthermore, this paper will discuss the contribution of deep learning to face recognition as well as the scope and its limitations for an attendance system. The objectives of this research are twofold: to gain qualitative insights on the concept, the trend, and the use of deep learning in face recognition; and to discover the applicable algorithms with deep learning approaches.

# RELATED WORKS

In artificial intelligence, deep learning is the subset of artificial intelligence where it applies the concept of artificial neural networks (ANN) that is inspired by the biological neuron networks (Teoh et al., 2021). It is also known as deep structured learning that extends as a new form of machine learning. Computationally, the representation data will be processed through multiple layers of nonlinear processing that is called ANN in order to learn. Deep learning can be classified into three major classes which are unsupervised learning, supervised learning, and hybrid deep network (Deng, 2014). One of the major advantages of deep learning is that it allows to train a huge number of datasets that can give more accurate results (Trigueros et al., 2018).

Face recognition is an artificial intelligence technology that applies the principles of biometric technology approaches by observing and matching the given facial profile according to the pre-given dataset. Since the beginning of the invention of the face recognition concept, it has led experts and researchers to come up with the application of face recognition in numerous real-world settings, such as in personal identification and authentication, where in this case it is also applicable for the attendance system (Kar et al., 2012). Hence, face recognition has become one of the major focuses of modern technology research and development.

Face recognition is built on multiple disciplines of mathematics and computer science studies with various approaches with different performance. In the early attempt of creating modern face recognition, a machine learning based approach was introduced called eigenface by using principal components to identify facial profile in 1988 (Tolba et al., 2006). However, despite the succession of eigenfaces in identifying facial profiles and their identities, the performance is not efficient and is unsatisfactory due to its limitation in recognition under different circumstances.

Another more modernised approach is called DeepFace, which uses a deep learning framework to recognize facial profiles. The proposed approach modifies the recognition by using 3D modelling techniques in order to identify the facial profile precisely as similar as human beings (Taigman et al., 2014). The study found that the DeepFace system reaches 97.35 percent accuracy which is only 0.28 percent less than human capability in recognizing facial profiles.

Patel et al. (2018) proposed a system that uses the OpenCV library. OpenCV is a computer vision library that provides multiple language interfaces, such as Python, Java, and C++, that can be operated in various operating systems like Windows, MacOS, Linux, iOS, and Android. The proposed computer vision library provides more than 2,500 optimised algorithms for face recognition and detection. Zangeneh et al. (2020) proposed a system using FERET dataset with coupled mapping method and deep convolutional neural network (DCNNs) with 99.2 percent accuracy. AlBdairi et al. (2022), through comparative study of deep learning-based face recognition, found that GoogleNet DCNN based with 94 percent accuracy, AlexNet CNN based with 87 percent accuracy, and ResNet ANN based with 94 percent accuracy.

# METHODOLOGY

In achieving the purpose of this exploratory study, a qualitative method is chosen to be conducted. Based on expertise required to provide better insights on this research, as well as the need to get consent for taking time off the potential respondent, the convenience sampling technique is used to identify the right expert to be interviewed. An interview was conducted, i.e., qualitative data collection method, in which descriptive data is collected during the interview. In terms of ethical consideration, the consent of the respondent is sought after before an appointment was made for the interview session.

The interview was conducted with the Head of Smart DigitalLab Unit, who also holds a position as the Coordinator of Postgraduate Study of Computer Science under the School of Computer Science and Engineering, in a private university in Malaysia. The respondent is known as the expert in artificial intelligence, in which she has a convincing number of related works in the area of research related to this study, which include experiences and publications on face recognition systems. The interview design was set as a semi-structured interview, where the respondent was asked a set of open-ended questions. The interview session took approximately 39 minutes, and it was conducted online due to the locations and distance between the researcher and the respondent. The online interview was conducted via Zoom teleconference platform on Friday, 21st May 2022.

The data analysis was conducted using content analysis on the transcription of the recorded interview. The transcribed interview data was examined and summarised correspondingly to the topic of the interview. Every content was examined to achieve the objective of this research. By conducting the interview, the researcher was able to get better understanding from the expert of the related field by having deep conversation. Through the interview, the researcher was able to eliminate any confusion and misconceptions on certain aspects of the discussion.

# FINDINGS

This section presents the findings from the interview survey, which answer the questions to discover and develop the understanding of the implementation of deep learning in face recognition. This section is separated into three parts as how the interview session was guided: perception towards the trend of the use of face recognition; deep learning algorithm in face recognition; and face recognition-based attendance system tools and methods.

**Perception towards the Trend of the Use of Face Recognition**

When the respondent was asked about the perception towards the trend of the use of face recognition, the respondent explained that since the first time the concept of face recognition was introduced in 1987, face recognition began to develop and be used for certain purposes, such as in military purposes. At the current stage, the respondent believes that everyone wants to equip that technology as the inventions evolve. However, the respondent tended to believe that the definition as a completely effective and efficient way or implementation cannot be embedded in face recognition, since either face recognition or any other offered detection methods or any application will have their own pros and cons.

The respondent also added that indeed, face recognition has its vulnerabilities and issues. Every single embedded technology, regardless of what type of the application and how it is implemented, each of them has vulnerabilities as well as weaknesses. However, the possible issues in face recognition, such as in low accuracy, can be handled by correcting the certain method like in choosing the integrated algorithm that is used.

Finally, the respondent stated that the presence and the use of face recognition has changed and began to be widely revolutionised to be implemented in wider applications over the time. It has not only been used for the comprehensive system, it is even being used now for the mobile application, which is not only limited to security purposes, but also to provide media-entertainment features, such as in the enhancement of the face recognition in the camera for the users.

**Deep Learning Algorithm in Face Recognition**

According to the definition stated by the respondent, deep learning is actually an algorithm, where the root of deep learning itself is an algorithm. The respondent added that researchers consider that deep learning comes from machine learning. However, the respondent tended to consider that machine learning is a different thing from deep learning, whereas it only has the similarity of the concept between both where there is an overlapped definition that arise among the people between machine learning and deep learning. Finally, the respondent concluded that deep learning does not come from machine learning.

The goal of deep learning is to achieve the highest accuracy. It is designed to find the best among the best. The respondent added that the terms “optimization algorithm” can be used to define deep learning. In its process, the respondent stated that multiple algorithms can be used instead using specific algorithms in deep learning. For instance, the respondent explained the implementation of Neuro-Fuzzy Logic, where the algorithm is trained within multiple layers of Neuro-Fuzzy Logic. When they are applied, the algorithms are not standalone anymore, but it is combined to generate new findings. The combination of the algorithm is meant to try to find the best output.

In face recognition, the combination of algorithms is used in order to get clearer detection and recognition. Examples of the algorithms that are commonly used in face recognition are Coupled Spectral Regression (CSR) algorithm, Eigenface, Local Binary Pattern Histogram (LBPH), and Vector Algorithm. However, the respondent stated that the combination of deep learning algorithms is not widely used for physical systems settings like face recognition. The respondent redefined that the term of combining the deep learning algorithms means assembling the existing deep learning algorithm with one to another.

The interview continued with the respondent adding that instead of combining the deep learning algorithm, in order to achieve better performance in face recognition, the combination of the algorithms to optimise the parameters utilisation of the algorithm may be required. There are many proposed algorithms with various outputs. Some of the algorithms may have better performance in image features extraction while some other algorithms may have advantages in low computational complexity. Necessarily, the parameters and the way it is utilised may be different from one to another algorithm. Thus, the combination of algorithms can be used to obtain certain results. The respondent added that according to her article, in this case, deep learning algorithms are precisely to be used to enhance parameters optimization (Ismail & Ismail, 2022).

**Face Recognition-based Attendance System Tools and Methods**

The interview respondent suggested using LBPH as the main proposed algorithm and OpenCV-Python as the vision library. Local Binary Pattern Histogram (LBP) is known to be one of the popular algorithms for physical systems, particularly for face recognition devices, due to its simplicity. Face Recognition can be designated for detecting motion and emotion, however, according to the respondent, LBPH is only capable in detecting motion. The system architecture of the system can be various depending on the intended application, for instance a web application-based system where the face recognition is integrated with the PHPmyAdmin as the web server and MySQL as the database.

# DISCUSSION

Initially, the face recognition idea was introduced in 1987 by using principal component analysis, which was later used as the fundamentals of the first face recognition technology known as Eigenfaces (Turk & Pentland, 1991). Since then, the use began to widen, and development of face recognition began to advance especially with the existence of deep learning methods (Guo & Zhang, 2019). One of the recent achievements of deep learning-based face recognition is using convolutional neural networks (CNN). A study found that a CNN-based face recognition combined with Principal Component Analysis (PCA) can achieve 98 percent accuracy (Singhal et al., 2021). Due to its advantages and capabilities to be used in many applications, the use of face recognition keeps increasing while the development of face recognition is even more rapid. According to the findings from the respondent’s perspective, it is to be believed that nowadays face recognition technology has begun to be highly demanded. Accordingly, previous research found that approximately market on face recognition to be growing at 14 percent of the compound annual growth rate in 2027 (Grand View Research, 2020). It is obvious that the trend of face recognition is increasing over the years.

According to the findings, there are various algorithms that can be used in face recognition. One of the algorithms that can be used is Local Binary Pattern Histograms (LBPH). In face recognition, LBPH is commonly used to define the face by extracting facial features (ST et al., 2022). LBPH is known to be one of the old face recognition algorithms that requires less computational time and is easy to implement. However, there is no affirmative documentation that states that LBPH is part of deep learning, yet the use of LBPH standalone is a bit simpler, while deep learning has more complex architecture (Farfade et al., 2015). Besides that, the comparison of the result of LBPH stand alone with the deep learning approaches also can be a factor that makes LBPH and deep learning distinguished. In an experiment, LBPH reached 63 percent of accuracy, while Convolutional Neural Network (CNN), which is a deep learning algorithm, reached 99.88 percent of accuracy. However, CNN requires a huge scale of datasets of the samples in order to get excellent performance (Pei et al., 2019).

Nevertheless, the LBPH can still be categorised as deep learning algorithm if it is combined or implemented with the deep learning environment. For instance, Xi et al. (2016) proposed an algorithm named Local Binary Pattern Network (LBPNet) using the Local Binary Pattern operator and CNN as the principles. It focuses on increasing the efficiency of extracting and comparing the features. Similarly, Dusa and Phulpagar (2020) proposed a method of a two combination of Deep Neural Network (DNN) as the detection and LBPH algorithms as the training data set, showing better accuracy compared to Haar Cascade algorithm.

Furthermore, the deep learning algorithm can be obtained by using the DNN module in OpenCV and Histogram Oriented Gradient (HOG) – CNN in dlib. OpenCV is a cross-platform vision library that can be used to implement the overall system of face recognition-based attendance system, while dlib can be used for the face recognition by using face\_recognition library, a deep learning-based face recognition library. It is to be claimed that it reached 98 percent accuracy (Patel et al., 2018).

# CONCLUSION

The use of deep learning in face recognition-based attendance systems is more likely applicable considering the trend of face recognition is increasing, yet it is still uncommon in wide use. Although deep learning brings more excellent results in face recognition, it requires a huge number of samples of the faces and datasets storage. Hence, it is important to create proper planning on creating an efficient face recognition-based attendance system. In order to get a clearer understanding and more efficient system, it is essential to conduct deeper research on face recognition algorithms. Furthermore, it is also important to have a test on the implementation of the face recognition on the system in order to analyse its efficiency and performance.

**REFERENCES**

AlBdairi, A. J., Xiao, Z., Alkhayyat, A., Humaidi, A. J., Fadhel, M. A., Taher, B. H., Alzubaidi, L., Santamaría, J., & Al-Shamma, O. (2022). Face recognition based on Deep Learning and FPGA for ethnicity identification. *Applied Sciences*, 12(5). https://doi.org/10.3390/app12052605

Deng, L. & Yu, D. (2014). Deep Learning: Methods and Applications, *Foundations and Trends in Signal Processing*, 7(3), 197-387. https://doi.org/10.1561/2000000039

Dusa, P., & Phulpagar, B. D. (2020). Criminal Detection Using OpenCV DNN and OpenCV LBPH Methods. *Aegaeum Journal*, 8(8), 675-684. https://doi.org/16.10089.AJ.2020.V8I8.285311.3967

Farfade, S. S., Saberian, M. J. & Li, L.J. (2015). Multiview Face Detection using Deep Convolutional Neural Networks, *5th ACM on International Conference on Multimedia Retrieval*, Shanghai, 2015, 643-650. https://doi.org/10.1145/2671188.2749408

Grand View Research. (2020). Facial Recognition Market Size, Share & Trends Analysis Report by Technology (2D, 3D, Facial Analytics), by Application (Access Control, Security & Surveillance), by End-use, by Region and Segment Forecasts, 2021-2028. Retrieved August 01, 2022, from https://www.grandviewresearch.com/industry-analysis/facial-recognition-market

Guo, G., & Zhang, N. (2019). A survey on deep learning-based face recognition. *Computer vision and image understanding*, 189. https://doi.org/10.1016/j.cviu.2019.102805.

Ismail, S., & Ismail, S. (2022). A preliminary study of cashless payment face recognition system development in Malaysia. *16th International Conference on Ubiquitous Information Management and Communication (IMCOM)*, Seoul, 2022, 1-5. https://doi.org/10.1109/IMCOM53663.2022.9721723

Kar, N., Debbarma, M. K., Saha, A., & Pal, D. R. (2012). Study of implementing automated attendance system using face recognition technique. *International Journal of Computer and Communication Engineering*, 1(2), 100–103. https://doi.org/10.7763/ijcce.2012.v1.28

Patel, S., Kumar, P., Garg, S., & Kumar, R. (2018). Face recognition based smart attendance system using IOT. *International Journal of Computer Sciences and Engineering*, 6(5), 871–877. https://doi.org/10.26438/ijcse/v6i5.871877

Pei, Z., Xu, H., Zhang, Y., Guo, M., & Yang, Y.-H. (2019). Face recognition via deep learning using data augmentation based on orthogonal experiments. *Electronics*, 8(10). https://doi.org/10.3390/electronics8101088

Setiowati, S., Zulfanahri, Franita, E. L., & Ardiyanto, I. (2017). A review of optimization method in face recognition: Comparison deep learning and non-deep learning methods. *9th International Conference on Information Technology and Electrical Engineering (ICITEE)*, Puket, 2017, 1-6. https://doi.org/10.1109/ICITEED.2017.8250484

Singhal, N., Ganganwar, V., Yadav, M., Chauhan, A., Jakhar, M., & Sharma, K. (2021). Comparative study of machine learning and Deep Learning Algorithm for face recognition. *Jordanian Journal of Computers and Information Technology*, 7(3), 313-325. https://doi.org/10.5455/jjcit.71-1624859356

ST, S., Ayoobkhan, M. U., V, K. K., Bacanin, N., K, V., Štěpán, H., & Pavel, T. (2022). Deep learning model for deep fake face recognition and detection. *PeerJ Computer Science*, 8. https://doi.org/10.7717/peerj-cs.881

Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification. *IEEE Conference on Computer Vision and Pattern Recognition*, Columbus, 2014, 1701-1708. https://doi.org/10.1109/CVPR.2014.220

Teoh, K. H., Ismail, R. C., Naziri, S. Z. M., Hussin, R., Isa, M. N. M. & Basir, M. S. S. M. (2021). Face Recognition and Identification using Deep Learning Approach, *Journal of Physics: Conference Series*, 1755. https://doi.org/10.1088/1742-6596/1755/1/012006

Tolba, A. S., El-Baz, A. H., & El-Harby, A. A. (2006). Face recognition: A literature review. *International Journal of Signal Processing*, 2(2), 88-103.

Trigueros, D. S., Li, M. & Hartnett, M. (2018). Face Recognition: From Traditional to Deep Learning Methods, *Computer Vision and Pattern Recognition*. https://doi.org/10.48550/arXiv.1811.0011

Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. *Journal of Cognitive Neuroscience*, 3(1), 71–86. https://doi.org/10.1162/jocn.1991.3.1.71

Xi, M., Chen, L., Polajnar, D., & Tong, W. (2016). Local Binary Pattern Network: A deep learning approach for face recognition. *IEEE International Conference on Image Processing (ICIP)*, Phoenix, 2016, 3224-3228. https://doi.org/10.1109/ICIP.2016.7532955

Zangeneh, E., Rahmati, M., & Mohsenzadeh, Y. (2020). Low resolution face recognition using a two-branch deep convolutional neural network architecture. *Expert Systems with Applications*, 139. https://doi.org/10.1016/j.eswa.2019.112854