**A Web-based Image Recognition System for Detecting Harumanis Mangoes** **Mohamad Shahmil Saari1, Romiza Md Nor2\*, Huzaifah A Hamid3** *1,2 Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, 02600 Arau, Perlis, Malaysia* *3 Academy of Language Studies, Universiti Teknologi MARA, 02600 Arau, Perlis, Malaysia* [***2romiza@uitm.edu.my***](mailto:2romiza@uitm.edu.my) [***3huzaifahhamid@uitm.edu.my***](mailto:3huzaifahhamid@uitm.edu.my) **ABSTRACT** *Harumanis mango cultivar is special to Perlis (north state of Malaysia) and has been declared in the national agenda as a special fruit. For those who are not acquainted with aromatic mango, it is difficult to tell the distinction between Harumanis and the others. By using image recognition, people can identify Harumanis feature details by image recognition technique where algorithm is applied to recognize the mango. Convolutional neural networks method is a suitable technique for the creation of a multi-fruit in real-time classification sorter with the camera and for the detection of moving fruit. Furthermore, the accuracy of the image classification can be improved by increasing the number of datasets, the distance of images from the camera, and the labelling process. This project used Mobile Net architecture model because it consumes less computational power and it can also provide efficiency of the accuracy. A web-based image recognition system for detecting Harumanis mangoes was developed and known as CamPauh to recognize four classes of mango which are Harumanis, apple mango, other types of mangoes and not mango. CamPauh can identify different type of mangoes and the result was stored into the database and appeared on the website. Evaluation on the accuracy was conducted discussed to support users’ satisfaction in identifying the correct mango type.* ***Keywords**: image recognition, fruit texture**,** convolutional neural networks* **INTRODUCTION** Today, mango is an important commercial crop not only in India but also in Indonesia, Thailand, and Malaysia. In Peninsular Malaysia, there are over three hundred cultivars with fruits considerably differ in size, shape, colour, flavor and fiber content. For example, Harumanis mango cultivar is special to Perlis (north state of Malaysia) and has been declared in the national agenda as a special fruit from Perlis for the world (Farook et al., 2013). Perlis has exported 3.1 tons Harumanis to Japan and the export market was targeted at increasing to 100 metric tons, by 2020. According to Ramli (2017), some dishonest traders claim other mango types to be the iconic Harumanis in which they sold other mango varieties at a premium price. Some traders also put false labels where many visitors or tourists thought that they are purchasing Harumanis mangoes. Therefore, it is crucial to identify the physical characteristics of Harumanis mangoes, and this can be done by observing their shape, colours and smell such as an elevated ventricle shoulder, an oval shape, around top, dark green colors and an aromatic smell. These are important for the public to learn and be aware of. Currently, the recognition process of Harumanis mangoes is done manually. As such, in this project, the image recognition for mango type detection is built up to reduce the time consumed by farmers and plantation agencies. Besides that, more profit can be gained by reducing detection time and the result is more accurate. Harumanis mango trees can produce tons of mangoes and this creates a problem for plantation agencies to grade them as it is difficult to tell the distinction between Harumanis and other types of mangoes especially for those who are not acquainted with aromatic mango. Next, by using image recognition, customers no longer need to familiarize themselves with Harumanis feature details because image recognition technique would create the algorithm to identify the mango. The purpose of this study is to solve customers’ problems of buying fake mango by developing image recognition application to identify the accuracy of mango type based on their texture. In this project, image recognition is applied because it could deal with large number of database and automatically identify objects. Hence, with this technique, it can help people identifying objects more precisely. **RELATED WORKS** Image recognition is an important task of computer vision where it provides the algorithm to a computer to understand images. Image recognition refers to the potential of systems or code to identify images objects, individuals, places, and behaviour, using artificial intelligence vision technologies and qualified algorithms for camera-based image recognition (Gupta, 2019). In many fields and areas, including the agricultural sector, image recognition technology has become possible and more important. Intermittent weights on the neural networks are modified to enhance the accuracy of the system to identify images to increase the accuracy of the systems. The essential analysis technique or model of this project is convolutional neural network. According to Basri et al (2019), this method is particularly suitable for the creation of a multi-fruit in real-time classification sorter with the camera and for the detection of moving fruit. Furthermore, the accuracy of the image classification can be improved by increasing the number of datasets, the distance of images from the camera, and the labelling process. The software or framework chosen is TensorFlow as it can be easily learned and worked with and provides easy ways to express how high-level abstractions can be connected. TensorFlow applications can be run at most convenient goals, iOS and Android phones, local computers, clusters in the cloud, CPUs or GPUs**.** Arivazhagan (2010) in his project has recognized the fruit's intensity, color, shape, and texture of four fundamental properties. This work involved two sections which are training and classification. The number of datasets per class taken was between 75 to 264. Arivazhagan (2010) also stated that the recognition system shape and size features can be combined with colors and texture features to enhance usability and versatility. The increase number of images in the database also can increase the recognition rate. Furthermore, Basri (2019) also conducted a research in detecting multi-fruit classification. The input used was a mango and pitaya fruit. The real dataset was taken from a farmer at harvest time and then a two-class data was created, whereby the classification of train objects was mango and pitaya. The analysis technique used was Faster R-CNN. It is a completion of R-CNN and Fast R-CNN techniques for solving problems of image classification. The resulting accuracy of this analysis technique can reach 99%. The appropriate labelling process in the image and the randomly selected image will ensure the accuracy of the data. The analysis technique is useful for developing in real-time with the camera a sorting system for the identification and detection of moving fruits. Therefore, in this paper the essential technique or model used is convolutional neural network. According to Basri et al.(2019), this method is particularly suitable for the creation of a multi-fruit in real-time classification sorter with the camera and for the detection of moving fruit. Furthermore, the accuracy of the image classification can be improved by increasing the number of datasets, the distance of images from the camera, and the labelling process. As it can be easily learned and worked with, TensorFlow was chosen as the software or framework and it can also provide easy ways to express how high-level abstractions can be connected. **METHODOLOGY** Figure 1 shows the experimental design that illustrated the flow and operation of CamPauh. Firstly, preparation of dataset which consists of the images of different types of mangoes such as Harumanis, apple and others and other types of fruits that are not in the mango family were collected. From this dataset, four types of mangoes were classified. Each class needs 400 images. The increase of the number of images collected enhances the accuracy of this project. The next step is training images and saved model using TensorFlow and Keras by applying Mobile Net model or architectures. This step is essential as in order to enhance the degree of accuracy, the number of training steps also needs to be increased. The next step is creating GUI using Tkinter using Python. Tkinter loaded the model and labels of saved images. Classification was also created to allow users to select the image that they would like to classify or recognize. Besides that, stored result to database was also written to enable the website to provide information to the user. Lastly is creating website (Figure 2) for CamPauh to let users view the history of recognized images and to provide a graph to help users identify the mango type. > ![](153-1-484-1-6-20201020_media/media/image1.png) > > Figure 1: Experimental design of CamPauh > > ![](153-1-484-1-6-20201020_media/media/image2.png) > > Figure 2: CamPauh website **FINDINGS AND DISCUSSION** In this project, functionality testing, usability testing and accuracy evaluation were conducted. In this study, only accuracy testing result is discussed. Most of the respondents agreed on the CamPauh accuracy. This is proven when every time users recognize the mango types; it provides an accurate result. This is because CamPauh had collected numerous images to the dataset to ensure that the result is accurate and to increase the number of training steps which can enhance the accuracy of the result. This is because during the training data, CamPauh had met the range of training data process of no overfitting and underfitting. Overfitting means that the dataset is trained for too long whereas underfitting means the model does not have enough training. Overfitting training can be solved by increasing the number of data and underfitting can be solved by training the dataset in a longer time range. Calculation on the accuracy of CamPauh was done using confusion matric technique. The confusion matrix is a table that is used to define the output of a classification model on a collection of test data for which the true values are known. It enables the visualization of the output of an algorithm. Figure 3 shows the number of samples from every class that was used to calculate the accuracy of CamPauh. Each class had six samples. Every sample was renamed from 1 to 6 in the JPEG format images. The actual images were sort in the correct folder before they were being recognized using CamPauh. There were 24 images selected as a sample to calculate the accuracy of CamPauh. CamPauh would then recognize the images and provide the result such as prediction name or label and the accuracy score. The acceptable accuracy score from the system is 0.8 to 1.0. ![](153-1-484-1-6-20201020_media/media/image3.png) Figure 3: Testing sample from CamPauh Every single result was recorded as shown in Table 1. The green highlight in Table 1 refers to the accurate prediction of the images. The correct prediction means that the images that the system predicted are correct and the correct labels provided with the accuracy percentage was 0.8 and above. There were 21 images that obtained accurate prediction whereas three of the images obtained inaccurate prediction. Inaccurate prediction images were highlighted in red. The incorrect prediction would have the lowest accuracy score. Table 1: Prediction table ![](153-1-484-1-6-20201020_media/media/image4.emf)Table 2: Confusion matrix table | | **Predicted** | **Predicted** | **Predicted** | **Predicted** | **Predicted** | | ------ | ------------- | ------------- | ------------- | ------------- | ------------- | | Actual | | Harumanis | Apple Mango | Other Mango | Not Mango | | Actual | Harumanis | 5.8836 | | | | | Actual | Apple Mango | | 5.7423 | | | | Actual | Other Mango | 0.4893 | | 4.7376 | | | Actual | Not Mango | | 1.1841 | | 3.9122 | Table 2 shows the confusion matrix table for CamPauh. It shows the calculation on the accuracy of CamPauh where the number of total correct prediction was summed up and divided by the total number of classifications. The green highlight indicates the correct number prediction where red indicates incorrect prediction. Table 3: Accuracy result from prediction result and confusion matrix
Total correct prediction:

  • Harumanis = 5.8836

  • Apple Mango = 5.7423‬

  • Other Mango = 4.7376

  • Not Mango = 3.9122

Total=20.2757‬
Total incorrect prediction:

  • Harumanis = 0

  • Apple Mango = 0

  • Other Mango = 0.4893

  • Apple Mango = 1.1841

Total=1.6734

Total number of predictions = total correct prediction + total incorrect prediction

20.2757+1.6732

21.9491

Accuracy = Sum of correct prediction / Total number of predictions

20.2757‬ / 21.9491

Accuracy = 0.9238 => 92%
Table 3 shows the accuracy result that was determined by prediction results where it clearly shows that the accuracy is 92%. The result shows that the users were aware and confirmed the type of mango and this willincrease the possibility of users buying the type of mango that they prefer, Harumanis for example. **CONCLUSION** Image recognition is a technique to identify images objects, individuals, places, and behavior, using artificial intelligence vision technologies and qualified algorithms for camera-based image recognition. In this project, image recognition was used to predict the actual mango types and reduce the time spent by users to choose the correct mango type. This project also aims to provide a high accuracy result where it can satisfy the users. In CamPauh project, there were four mango types classified as Harumanis, apple mango, other mango types and not mango. The simple graphical user interface was created to help the users understand and use this system with ease. This project managed to complete and achieve the objectives which are to develop and to evaluate the accuracy of mango type based on their texture properties. The accuracy of the system was tested using a confusion matric algorithm with an accuracy that reached 92 percent. Nevertheless, there are also recommendations for this project for future study. Firstly, this system can improve its accuracy by increasing the number of datasets. Secondly, to increase the efficiency level of the system, researcher can deploy recognition system into the mobile application by using TensorFlow Lite. This will allow users to use it directly from their smartphones. Lastly, this system should allow users to recognize numerous images at a time. This will reduce users' time to detect their mango types. **REFERENCES** Arivazhagan. (2010). Fruit Recognition using Color and Texture Features. Journal of Emerging Trends in Computing and Information Sciences, (October), 1–5. Basri, R., Jacobs, D., Kasten, Y. and Kritchman, S. (2019). The convergence rate of neural networks for learned functions of different frequencies. In Advances in Neural Information Processing Systems. Farook, R. S. M., Ali, H., Harun, A., Ndzi, D. L., Shakaff, A. Y. M., Nor Jaafar, M., Aziz, A. H. A. (2013). Harumanis Mango Flowering Stem Prediction using Machine Learning Techniques. Research Notes in Information Science (RNIS), 13(May), 46–51. https://doi.org/10.4156/rnis.vol13.10 Gupta S. (2018). Understanding Image Recognition and Its Uses. Retrieved from Jalled, F., & Voronkov, I. (2016). Object Detection using Image Processing, 1–6. Retrieved from http://arxiv.org/abs/1611.07791 Mustakim Ramli. (2017, April 11). Fake 'Harumanis': Perlis to work with Domestic Trade Ministry to Monitor Traders. Retrieved from https://www.nst.com.my/news/nation/2017/04/229549/fake-harumanis-perlis-work-domestic-trade-ministry-monitor-traders Singh, R. (2019, June 10). Computer Vision? An Introduction. Retrieved from https://towardsdatascience.com/computer-vision-an-introduction-bbc81743a2f7