Enhancement of Depth Value Approximation for 3D Image-Based Modelling using Noise Filtering and Inverse Perspective Mapping Techniques for Complex Object

Intan Syaherra Ramli; Rahmita Wirza OK Rahmat; Seng Beng Ng

doi:10.24191/jcrinn.v8i2.356

Enhancement of Depth Value Approximation for 3D Image-Based Modelling using Noise Filtering and Inverse Perspective Mapping Techniques for Complex Object

Authors

Intan Syaherra Ramli Universiti Teknologi MARA
Rahmita Wirza OK RAhmat Universiti Putra Malaysia
Seng Beng Ng Universiti Putra Malaysia

DOI:

https://doi.org/10.24191/jcrinn.v8i2.356

Keywords:

Depth Value Approximation, Optical Flow, Trigonometry

Abstract

This article proposes the methods to enhance the depth value approximation in 3D Image Based Modelling for complex object. Fundamentally, the fast and accurate depth value approximation is crucial as the 3D modelling used in virtual and augmented reality applications, reverse engineering, and the architecture. Therefore, the enhanced method must be robust against the challenges with noise, complexity, distortion and longer processing time. In this experiment, five small and complex objects were captured using a turntable, laptop, and a webcam. The feature points between images were tracked and matched using good features to tracks and Pyramidal Lucas Kanade's optical flow. Next, the depth value was approximated using trigonometry equation. To enhance the accuracy, the noise filtering, and Inverse Perspective Mapping (IPM) were introduced. The results show that the average error based on the approximated width and depth dimensions was 3.27% and 6.88% compared with the actual object. Furthermore, the processing speed was 1519 points per second. Therefore, this method enhanced the depth value approximation, which can be used to build the full texture 3D model in future.

Downloads

Download data is not yet available.

References

Adikari, S. B., Ganegoda, N. C., Meegama, R. G., & Wanniarachchi, I. L. (2020). Applicability of a single depth sensor in real-time 3D clothes simulation: augmented reality virtual dressing room using Kinect sensor. Advances in Human-Computer Interaction. https://doi.org/10.1155/2020/1314598

Agudo, A. (2022). Safari from Visual Signals: Recovering Volumetric 3d Shapes, ICASSP 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2495-2499.

Awange, J., & Kiema, J. (2019) Light Detection and Ranging (LiDAR). In: Environmental Geoinformatics. Environmental Science and Engineering. Springer, Cham. https://doi-org.ezaccess.library.uitm.edu.my/10.1007/978-3-030-03017-9_21

Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2011). A database and evaluation methodology for optical flow. A Journal of Computer Vision, 92(1), 1-31. https://doi:10.1007/s11263-010-0390-2

Cao, M., Jia, W., Lv, Z., Li, Y., Xie, W., Zheng, L., & Liu, X. (2019). Fast and robust feature tracking for 3D reconstruction. Optics & Laser Technology, 110, 120-128. https://doi.org/10.1016/j.optlastec.2018.05.036

Chen, S., Liang, L., & Ouyang, J. (2022). Accurate structure from motion using consistent cluster merging. Multimed Tools Appl 81, 24913–24935. https://doi.org/10.1007/s11042-022-12202.

Cho, J., Jung, Y., Kim, D.-S., Lee, S., & Jung, Y. (2019). Moving Object Detection Based on Optical Flow Estimation and a Gaussian Mixture Model for Advanced Driver Assistance Systems. Sensors, 19(14), 3217. https://dx.doi.org/10.3390/s19143217

Choi, H., Kang, B., & Kim, D.(2022). Moving Object Tracking Based on Sparse Optical Flow with Moving Window and Target Estimator. Sensors.22(8):2878. https://doi.org/10.3390/s22082878

Chungyup, L., Soohyeon, C., Jung, Y., & Kwanghee, W.(2019). Instance segmentation in urban scenes using inverse perspective mapping of 3D point clouds and 2D images.RACS '19: Proceedings of the Conference on Research in Adaptive and Convergent Systems. pp.147–152 https://doi.org/10.1145/3338840.3355677

Clément, R., Vincent, N., & Pascal M. (2022). Automatic RANSAC by Likelihood Maximization, Image Processing On Line, (12),pp. 27-49.

https://doi.org/10.5201/ipol.2022.357

Crisnapati, P. N., Setiawan, M., Wikranta Arsa, I. G. N., Devi Novayanti, P., Wibawa, M. S., & Oka Ciptahadi, K. G.(2019) Real-Time Hand Palm Detection and Tracking Augmented Reality Game Using Lucas Kanade Optical Flow Combined with Color Blob Detection. 2019 1st International Conference on Cybernetics and Intelligent System (ICORIS), 263-268, https://doi:%2010.1109/ICORIS.2019.8874892

Davide, M., Simone, B., & Gianluigi, C. (2022). SfM Flow: A comprehensive toolset for the evaluation of 3D reconstruction pipelines, SoftwareX, (17),100931, ISSN 2352-7110. https://doi.org/10.1016/j.softx.2021.100931

Dhal, K., Karmokar, P., & Chakravarthy, A. et al. (2022). Vision-Based Guidance for Tracking Multiple Dynamic Objects. J Intell Robot Syst 105, 66 https://doi.org/10.1007/s10846-022-01657-6

Ding, T., Yuan, J., Lin, X., Zhang, N., Zhang, Y., & Gao, X. (2021). Three Dimensions Reconstruction of Single-spectrum Multi-X-ray Views of Contraband Based on Space Carving Method. In Journal of Physics: Conference Series 1986(1), p.p 012129. https://doi.org/10.1088/1742-6596/1986/1/012129

Domen, G., Janko S., Aleš B.,& Miha B.(2021). Still-camera multiview Spectral Optical Flow Imaging for 3D operating-deflection-shape identification. Mechanical Systems and Signal Processing,152,107456, ISSN 0888-3270. https://doi.org/10.1016/j.ymssp.2020.107456

Dong,Y. (2022). Faint Moving Small Target Detection based on Optical Flow Method, 7th International Conference on Intelligent Computing and Signal Processing (ICSP), Xi'an, pp. 391-395.

Elkhrachy, I. (2022). 3D Structure from 2D Dimensional Images Using Structure from Motion Algorithms. Sustainability,14(9):5399. https://doi.org/10.3390/su14095399

Escott, K., Shaw, R., & Lensen, A. (2023). Feature-based Image Matching for Identifying Individual K=ak=a. ArXiv. https://doi.org/10.48550/arXiv.2301.06678

Fan, R., Ai, X., & Dahnoun, N. (2018). Road surface 3D reconstruction based on dense subpixel disparity map estimation. IEEE Transactions on Image Processing, 27(6), 3025-3035. EC Accession Number: 17665641. https://doi.org.10.1109/TIP.2018.2808770

Justs, D. J., Novickis, R., Ozols, K., & Greitāns, M. (2020). Bird’s-eye view image acquisition from simulated scenes using geometric inverse perspective mapping. In 2020 17th Biennial Baltic Electronics Conference (BEC) (pp. 1-6).

https://doi.org/10.1109/BEC49624.2020.9277042

Kang, Z., Yang, J., & Yang, Z., & Cheng, S.(2020). A Review of Techniques for 3D Reconstruction of Indoor Environments. ISPRS International Journal of Geo-Information, 9(5),330. https://doi.org/10.3390/ijgi9050330

Li, M., Zheng, D., Zhang, R., Yin, J., & Tian, X. (2015). Overview of 3d reconstruction methods based on multi-view. In 2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics (2), pp. 145-148. https://doi.org/10.1109/IHMSC.2015.117

Menna, F., Nocerino, E., Morabito, D., Farella, E. M., Perini, M., & Remondino, F. (2017). An open-source low-cost automatic system for image-based 3D digitization. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 155. https://doi.org/10.5194/isprs-archives-XLII-2-W8-155-2017

Miller, A., Miller, B., Popov, A., & Stepanyan, K. (2019). UAV landing based on the optical flow video navigation. Sensors, 19(6), 1351.http:/dx.doi.org/10.3390/s19061351

Nadour, M., Boumehraz, M., Cherroun, L., & Puig, V. (2019). Mobile robot visual navigation based on fuzzy logic and optical flow approach. International Journal of System Assurance Engineering and Management, 10(6), 1654-1667. https://doi.org/10.1007/s13198-019-00918-2

Ondrašovič, M., & Tarábek, P. (2021). Homography Ranking Based on Multiple Groups of Point Correspondences. Sensors, 21(17), 5752. https://dx.doi.org/10.3390/s21175752

Paul, M., Karsh, R. K., & Talukdar, F. A. (2019). Image hashing based on shape context and speeded up robust features (SURF).International Conference on Automation, Computational and Technology Management (ICACTM) (pp. 464-468). https://doi.org.10.1109/ICACTM.2019.8776713

Rodriguez-Gonzalvez, P., Gonzalez-Aguilera, D., Lopez-Jimenez, G., & Picon-Cabrera, I. (2014). Image-based modelling of the built environment from an unmanned aerial system. Automation in Construction, 48, 44-52. https://doi.org/10.1016/j.autcon.2014.08.010

Shalma, H., & Selvaraj, P. (2021). A review on 3D image reconstruction on specific and generic objects. Materials Today: Proceedings. https://doi.org/10.1016/j.matpr.2021.06.371

Shi, J., & Tomasi, C. (1994). Good features to track. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 593-600). https://doi.org/10.1109/CVPR.1994.323794.

Stathopoulou, E. K., Welponer, M., & Remondino, F. (2019). Open-source image-based 3D reconstruction pipelines: Review, comparison and evaluation. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W17, 331-338.

Tan, P. (2020) Image-Based Modeling. In: Ikeuchi K. (eds) Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-030-03243-2_11-1

Tareen, S. A. K., & Saleem Z., (2018). A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK, 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pp. 1-10, doi: 10.1109/ICOMET.2018.8346440.

Tushar J., Kulbir S., Aditya A. (2017) A review and comparison of multi-view 3D reconstruction methods. Journal of Engineering Research, Vol(5)(3). https://kuwaitjournals.org/jer/index.php/JER/article/view/2307

Wang,X., Rottensteiner,F., & Heipke, C. (2019). Structure from motion for ordered and unordered image sets based on random k-d forests and global pose estimation, ISPRS Journal of Photogrammetry and Remote Sensing, Volume 147, Pages 19-41, ISSN 0924-2716, https://doi.org/10.1016/j.isprsjprs.2018.11.009.

Wirza, R., & Azmi, S. (2012). Depth Value Deduction for 3D Reconstruction using Optical Flow to Reverse Engineered a Geometrical Shape. In Workshop On Advanced Information Technology (WIT-A2012) (p.6).

Wongsaree, P., Sinchai, S., Wardkein P., & Koseeyaporn, J., (2018). Distance Detection Technique Using Enhancing Inverse Perspective Mapping, 3rd International Conference on Computer and Communication Systems (ICCCS), pp. 217-221, https://doi.org/10.1109/CCOMS.2018.8463318.

Liu, Y-F., Nie, X., Fan, J-S., Liu, X-G.(2020). Image-based crack assessment of bridge piers using unmanned aerial vehicles and three-dimensional scene reconstruction. Comput Aided Civ Inf.; 35: 511– 529. https://doi.org/10.1111/mice.12501

Ze, L.,Yong, Q., Hui,W., Xiaoli, Z., Gehao, S., & Xiuchen, J.(2022). A novel image-orientation feature extraction method for partial discharges.IET Generation, Transmission & Distribution,16(6),1139-1150.https://doi.org/10.1049/gtd2.12356

Zhiliang, M., & Shilong L., (2018). A review of 3D reconstruction techniques in civil engineering and their applications, Advanced Engineering Informatics,37,163-174, ISSN 1474-0346. https://doi.org/10.1016/j.aei.2018.05.005.

Zhou, Z.X. Gong, J. Guo, M.Y. (2016) Image-based 3D reconstruction for post-hurricane residential building damage assessment, J. Comput. Civil Eng. 30 (2). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000480