Performance evaluation of a three modalities biometric recognition system with global decision fusion First Author\[1\]\*, Second Author2 (Double Blind Review: Please do not type or edit anything here until final-camera ready submissions) *1First affiliation, City and Country (Please do not type or edit anything here, until final camera-ready paper submission)* *2Second affiliation, City and Country (Please do not type or edit anything here, until final camera-ready paper submission)*
ARTICLE INFO ABSTRACT

Article history:

Received

Revised

Accepted

Online first

Published 1 March 2024

 Many of the problems associated with single-mode or dual-mode biometric systems make them unsuitable for modern biometric applications that require high levels of security and reliability. These include the use of a single biometric feature or two biometric features that are prone to noise, inadequate capture, an insufficient number of biometric points and a decline in biometric data quality. This paper evaluates the performance of a three-mode global decision fusion biometric recognition system based on fingerprint, voice and facial recognition. On a biometric three-modal pattern recognition system, the performance of access control by decision fusion is evaluated by the evaluation parameters (sensitivity, specificity, positive predictive value, negative predictive value and false negative) using the confusion matrix. Our access control system collected data from a sample of 500 people, 250 of whom were registered in our system and the remaining 250 were not registered in our system. The results showed 248 true positives, 2 false negatives, 1 false positive and 249 true negatives, which make up our confusion matrix. But on the basis of all the tests carried out, we can say that by using the combination of these three modalities, we improve the verification performance of the biometric recognition system. In the future, it will be interesting to optimise the computation times of this system.

Keywords:

biometrics

biometric recognition system

access control

facial recognition

fingerprints pattern

voice recognition

DOI:

10.24191/jcrinn.v9i1
## Introduction According to Benaliouche and Touahria (2014), single-mode biometric systems have a number of issues that render them unsuitable for modern biometric applications that need high degrees of security and dependability. According to Bopatriciat Boluma Mangata et al. (2022) these issues include the use of a single biometric feature that is susceptible to noise, poor capture, a lack of biometric points, and a decline in the quality of biometric input. **Problem-solvers** Several more factors must be taken into account in a workable biometric system that employs biometrics for personal recognition (Mathivet, 2017): Acceptability: the degree to which people are willing to accept the use of a specific form of biometric identification in daily life; Performance: the accuracy and speed of recognition achievable in terms of the resources required, as well as the operational and environmental factors that affect accuracy and speed; Bypass: the ease with which the system can be tricked using fraudulent methods. Every one of these qualities has unique qualities that align with the demands of various security systems (Norman, 2011). One kind of component, such as fingerprints, iris, face, and others, is used by a single-modal biometric system based on a single approach (Raji & Fried, 2021). According to Wirotius (2005), a realistic biometric system should meet predetermined standards for speed, accuracy, and recognition capabilities; it should also be safe for all users, be well-liked by the intended audience, and be resilient enough to fend off system intrusions and fraudulent techniques. A biometric system's ability to detect variations in the data it captures, its learning method, and the manner in which pertinent information is gathered all have a role in its overall performance (Kaur, Krishan, Sharma & Kanchan, 2020). **Theory** These issues can be resolved with the advent of multimodal biometrics, as multimodal systems can enhance recognition performance by merging many information sources (Bowers, D. M. (2013).). We will realize this access control system from the fusion of decisions on facial recognition, voice and fingerprint systems (Bopatriciat Boluma Mangata et al., 2021). **The work's motivation and purpose** Our work involves researching, developing, and putting into practice global decision fusion based biometric tree-modal individual recognition systems that utilize fingerprint, voice and facial recognition for automated access control. Multimodal systems combine several information sources to provide better recognition performance. Additionally, they solve the issue of some biometrics' non-universality and provide a high level of flexibility by allowing other biometric modalities to make up for biometric features that are undesirable or unusable in some people. Since it is more difficult to gather and replicate multiple features at once, they offer additional protection and reduce the likelihood of fraud (Cao & Jain, 2018). Our strategy will accomplish the following goals: - Create biometric access control systems employing fingerprint, voice and facial recognition technology (Hamann & Smith, 2019). - Use confusion matrices to assess these systems' performance (Zeng, 2020). **Interest in the topic** By giving scientists a dependable tool for biometric three-modal pattern identification, such an approach hopes to greatly benefit the scientific community. Methodology for research Our work is articulated through analytical and experimental methodologies as well as critical analysis. This type of work is interspersed with theoretical pursuits (documentary research), but primarily involves practical activities (participation in workshops, seminars, and colloquia, as well as the organization of exchange/discussion sessions with other residents in their respective disciplinary specializations and diversities) pertaining to particular project aspects. Installation of a three-modal biometric access control system using facial recognition, voice and fingerprints In this phase of the project, we will combine decisions made on fingerprint, voice and facial recognition systems to create an access control system (Conger, Fausset & Kovaleski, 2019; Benaliouche & Touahria, 2014). The following is the project's implementation architecture (Bopatriciat Boluma Mangata et al., 2021): ![](65b36a478432d_media/media/image1.png) Figure 1: The system's implementation architecture # RESULTS This is an illustration of a few of our application's graphical user interfaces: > ![](65b36a478432d_media/media/image2.png) > > Figure 2: The enrolment window ![](65b36a478432d_media/media/image3.png) > Figure 3: The verification window **The performance of the system** This article assesses how well our access control system, which is based on facial recognition and fingerprint recognition, secures our premises (Conger, Fausset & Kovaleski, 2019). Our three-modal access control system produced the following findings on a sample of 500 people, of which 250 were registered and 250 were not: 248 true positives, 2 false negatives, 1 false positive, and 249 true negatives. **Evaluation parameter estimation** A 2x2 confusion matrix with numbers specified can be used to summarize experiment findings when a representative sample of the population is available (Guizani, Zavala & Funamizu, 2016). The random events are B1: "being enrolled" and B2: "not being enrolled," given two fundamental subspaces of the space E (Caelen, 2017). **Assign the events from the fundamental set E to VP, FP, FN, and VN**. The following are the constituents of E and potential test findings (Bopatriciat Boluma Mangata et al., 2022). True Positives (VP): these are the people who are enrolled (E+) and are accepted by the system ({S}); False Positives (FP): these are the people who are not enrolled (E-) and who the system accepts ({S}); False Negatives (FN): these are the people who are enrolled (E+) and who the system rejects ({S-}); True Negatives (VN): these are the people who were not enrolled (E-) and where the system rejected them ({S-}); **Confusing matrix** The confusion matrix is a matrix used in supervised machine learning that assesses the caliber of a classification scheme. Real classes are represented by each row, while approximated classes are represented by each column. The number of items of the actual class L that have been predicted to belong to the class is contained in cell row L, column C. (Doyle, Heydarian, & Samavi, 2022). The confusion matrix's ability to rapidly display if a classification system is successful in classifying objects correctly is one of its advantages. > Table 1: Confusion Matrix | | E+ | E- | | -- | -- | -- | | S+ | VP | FP | | S- | FN | VN | **Calculating the specificity and sensitivity** The likelihood that a sign will exist if the person has the condition in question is known as the sign's sensitivity for that sickness. The following can consequently be noticed regarding the conditional probability (Markoulidakis, Rallis, Georgoulas, Kopsiaftis, Doulamis & Doulamis, 2021). As a result, the \(\text{Sensitivity} = Se = IP(S/E)\) (1) By calculating the ratio of the relevant numbers to the observed confusion matrix, this conditional probability is estimated: > \(Se \approx \frac{\text{VP}}{VP + FN}\) (2) Note: According to accepted practice, true parameters (conditional probabilities) and their estimations (ratios of observed numbers) are recorded in the same way (Xu, Zhang & Miao, 2020). The likelihood that a person won't be enrolled and that the system would refuse access is known as the specificity of an access control test. > \(\text{Specificity} = Sp = IP(\overline{S}/\overline{E}) \approx \frac{\text{VN}}{VN + FP}\) (3) Because people free of the disease exhibit the S sign less frequently, a diagnostic test becomes even more specific. The values of specificity and sensitivity for a "perfect" test, or one that produces no errors, are equal to 1. **Predictive value estimation** Let us imagine that in real life, a doctor receiving a positive or negative result from a complementary examination does not know if the patient has the condition he is trying to diagnose. The probabilities he is interested in are as follows: given that the examination has produced a positive (or negative) result, what is the likelihood that the patient has disease E? Predictive values are the names given to these probabilities (Ruuska, Hämäläinen, Kajava, Mughal, Matilainen & Mononen, 2018). In particular, we have: The likelihood that a person will have a disease if a symptom is present is known as the positive predictive value, or VPP, of the sign; The chance that a subject is free of an illness if a symptom is absent is known as the negative predictive value (VPN) of a given sign (Zeng, 2020). The estimates are from the identical data table: > \(\text{VPP} = IP\left( E/S \right) \approx \frac{\text{VP}}{VP + FP}\) (4) > > \(\text{VPN} = IP(\overline{E}/\overline{S}) \approx \frac{\text{VN}}{VN + FN}\) (5) **The evaluation parameters' computation** In the event that we wish to assess the effectiveness of the access control system on the foundational set E, which consists of 500 people dispersed throughout the confusion matrix below, let's compute the estimators of these parameters: Table 2: People arranged in the Confusing Matrix | | E+ | E- | | -- | --- | --- | | S+ | 248 | 1 | | S- | 2 | 249 | ** ** **Evaluation parameters computation on E's partitions** Let's think about occurrence A1. Such that, according to (Bopatriciat Boluma Mangata & al., 2022), they constitute a partition of the essential set E. Ai are mutually exclusive by definition, and E is the union of them. \(\forall\ \left( i \neq j \right),\left( A_{\text{i\ }} \cap A_{\text{j\ }} = \varnothing \right);\) (6) \(\bigcup_{j = 1}^{n}A_{\text{i\ }} = E\ \ \ \ \ \ \) (7) The breakdown of our 10 events that form the partition of the fundamental set E is shown in the following table (Haghighi, Jasemi, Hessabi & Zolanvari, 2018): Table 3: The division of the fundamental set E is created by ten occurrences. | N° | VP | FN | FP | VN | Total | | ----- | --- | -- | -- | --- | ----- | | 1 | 25 | 0 | 1 | 24 | 50 | | 2 | 25 | 0 | 0 | 25 | 50 | | 3 | 25 | 0 | 0 | 25 | 50 | | 4 | 25 | 0 | 0 | 25 | 50 | | 5 | 25 | 0 | 0 | 25 | 50 | | 6 | 25 | 0 | 0 | 25 | 50 | | 7 | 25 | 0 | 0 | 25 | 50 | | 8 | 25 | 0 | 0 | 25 | 50 | | 9 | 24 | 1 | 0 | 25 | 50 | | 10 | 24 | 1 | 0 | 25 | 50 | | Total | 248 | 2 | 1 | 249 | 500 | In the event that we want to assess the effectiveness of the access control system, let's calculate the estimators of these parameters. We obtain the table below (Se=VP/(VP+FN), Sp=VN/(VN+FP), VPP=VP/(VP+FP), VPN=VN/(VN+FN), and FN = 1-Sp) for the provided thresholds S1, S2,..., S10, and Average (AVG): Table 5: Estimating parameters computation

Group

 Se Sp VPP VPN FN
S1 1 0.96 0.961538 1 0.04
S2 1 1 1 1 0
S3 1 1 1 1 0
S4 1 1 1 1 0
S5 1 1 1 1 0
S6 1 1 1 1 0
S7 1 1 1 1 0
S8 1 1 1 1 0
S9 0.96 1 1 0.961538 0
S10 0.96 1 1 0.961538 0

AVG

 0.992 0.996 0.996154 0.992308 0.004
**Displaying data with bar graphs** This is our distribution bar chart for a quantitative statistical variable. The results of the ten groups, along with the average of the observations, are shown in this diagram: ![](65b36a478432d_media/media/image4.png) Figure 4: Observation By bar diagram As we can see, the average values are shown as follows in the above figure: 0.992 is the sensitivity, 0.996 is the specificity, 0.996154 is the positive predictive value, 0.992308 is the negative predictive value, and 0.004 is the false negative = 1-Sp. **Visual illustration utilizing the ROC curve** Plotting the sensitivity values Se against 1-Sp results in a ROC curve. As seen by the ROC curve above, which serves as a performance indicator, our model is a superior classifier in determining an individual's acceptance status. ![](65b36a478432d_media/media/image5.png) Figure 4 : Visual illustration utilizing the ROC curve **Conclusion** In order to provide secure access control, this work implements and studies three-modality individual recognition biometric systems based on global decision fusion that take advantage of fingerprint recognition technology, voice recognition and facial recognition. Many of the problems associated with single-mode biometric systems make them unsuitable for modern biometric applications that require high levels of security and reliability. These include the use of a single biometric feature that is prone to noise, inadequate capture, an insufficient number of biometric points and a decline in the quality of biometric data. This study therefore focuses on decision fusion access control in a three-modality pattern recognition biometric system that uses facial identification, fingerprints and voice. Using a three-modality biometric recognition system, the confusion matrix and evaluation parameters (false negative, positive predictive value, negative predictive value, specificity and sensitivity) are calculated to evaluate the performance of access control by global decision fusion. Our access control system collected data from a sample of 500 people, 250 of whom were registered and the remaining 250 were not. The results showed 248 true positives, 2 false negatives, 1 false positive and 249 true negatives, which constitute our confusion matrix. However, on the basis of all the tests carried out, we can conclude that by using the fusion of these three modalities, the system becomes increasingly efficient. # **REFERENCES** Benaliouche, H., & Touahria, M. (2014). Comparative study of multimodal biometric recognition by fusion of iris and fingerprint. The Scientific World Journal. Bopatriciat Boluma Mangata & Al. (2021). Contribution of an Embedded and Biometric System in a Replicated Database for Access Control in a Multi-Entry Institution. International Journal of Science and Research (IJSR), Volume (10 Issue 3), p2-5. Bopatriciat Boluma Mangata & al. (2022). Performance evaluation of a single access control system. Journal of research in engeneering and applied sciences. Volume (7 Issue 01), p4-6 Bowers, D. M. (2013). Access control and personal identification systems. Butterworth-Heinemann. Caelen, O. (2017). A Bayesian interpretation of the confusion matrix. *Annals of Mathematics and Artificial Intelligence*, *81*(3), 429-450. Cao, K., & Jain, A. K. (2018). Automated latent fingerprint recognition. *IEEE transactions on pattern analysis and machine intelligence*, *41*(4), 788-800. Conger, K., Fausset, R., & Kovaleski, S. F. (2019). San Francisco bans facial recognition technology. *The New York Times*, *14*, 1. Douaa, M. E. C. H. T. A., & Radhwane, G. H. E. R. B. I. (2019). AUTOMATISATION DES TACHES DOMOTIQUES D’UNE MAISON A L’AIDE D’UNE CARTE ARDUINO ET > LABVIEW (Doctoral dissertation, UNIVERSITE MOHAMED BOUDIAF-M’SILA). Guizani, M., Zavala, M. L., & Funamizu, N. (2016). Assessment of endotoxin removal from reclaimed wastewater using coagulation-flocculation. Journal of Water Resource and Protection, 8(9), 855-864. Haghighi, S., Jasemi, M., Hessabi, S., & Zolanvari, A. (2018). PyCM: Multiclass confusion matrix library in Python. *Journal of Open Source Software*, *3*(25), 729. Hamann, K., & Smith, R. (2019). Facial recognition technology. *Criminal Justice*, *34*(1), 9-13. Heydarian, M., Doyle, T. E., & Samavi, R. (2022). MLCM: multi-label confusion matrix. *IEEE Access*, *10*, 19083-19095. Jacob, I. J. (2019). Capsule network based biometric recognition system. *Journal of Artificial Intelligence*, *1*(02), 83-94. Kaur, P., Krishan, K., Sharma, S. K., & Kanchan, T. (2020). Facial-recognition algorithms: A literature review. *Medicine, Science and the Law*, *60*(2), 131-139. Markoulidakis, I., Rallis, I., Georgoulas, I., Kopsiaftis, G., Doulamis, A., & Doulamis, N. (2021). Multiclass Confusion Matrix Reduction Method and Its Application on Net Promoter Score Classification Problem. Technologies, 9(4), 81. Mathivet, V. (2017). L'intelligence artificielle pour les développeurs: concepts et implémentations en C\#. Éditions ENI. Norman, T. L. (2011). Electronic access control. Elsevier. Raji, I. D., & Fried, G. (2021). About face: A survey of facial recognition evaluation. *arXiv preprint arXiv:2102.00813*. Ruuska, S., Hämäläinen, W., Kajava, S., Mughal, M., Matilainen, P., & Mononen, J. (2018). Evaluation of the confusion matrix method in the validation of an automated system for measuring feeding behaviour of cattle. *Behavioural processes*, *148*, 56-62. Wirotius, M. (2005). Authentification par signature manuscrite sur support nomade (Doctoral dissertation, Tours). Xu, J., Zhang, Y., & Miao, D. (2020). Three-way confusion matrix for classification: A measure driven view. *Information sciences*, *507*, 772-794. Zeng, G. (2020). On the confusion matrix in credit scoring and its analytical properties. Communications in Statistics-Theory and Methods, 49(9), 2080-2093.

A picture containing text, clipart Description automatically generated

 © 2024 by the authors. Submitted for open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
1. \* Corresponding author. *E-mail address*: (Please do not type or edit anything here, our editors will do the work for you)