A Systematic Review on Facial Detection and Recognition: Limitations and Opportunities

Mohammad Amir Khan; Hasan Firdaus Bin Mohd Zaki; Azhar Bin Mohd Ibrahim; S.M Afzal Hoq; Amir Akramin bin Shafie

doi:10.69955/ajoeee.24.v4i2.70

Authors

Mohammad Amir Khan International Islamic University Malaysia
Hasan Firdaus Bin Mohd Zaki International Islamic University Malaysia https://orcid.org/0000-0002-7209-4355
Azhar Bin Mohd Ibrahim International Islamic University Malaysia
S.M Afzal Hoq International Islamic University Malaysia https://orcid.org/0000-0002-5917-2507
Amir Akramin bin Shafie International Islamic University Malaysia https://orcid.org/0000-0003-2440-7537

DOI:

https://doi.org/10.69955/ajoeee.24.v4i2.70

Keywords:

Detection, Recognition, Masked, Facial, Artificial Intelligence, Algorithms.

Abstract

Face recognition technology is a biometric tool that identifies people by facial characteristics. Individuals collect the facial photography, which is then automatically processed by picture recognition software. Face detection and recognition have several potential applications in various departments like security, education, healthcare, etc. Therefore, the fundamentals and methods of broad facial detection and recognition have been discussed in this article. Owing to the outbreak of the pandemic, people are now required to wear masks so that the spreading of the coronavirus is prevented, which makes it challenging to monitor sizable crowds of mask-wearing individuals. Face masks have higher interclass similarities and interclass variability because they cover a significant portion of the face, fooling face recognition systems' facial verification process. Thus, this paper has also discussed various aspects of masked face recognition.

Metrics

Metrics Loading ...

Downloads

Download data is not yet available.

Author Biography

Mohammad Amir Khan, International Islamic University Malaysia

Mechatronics Dept, Kulliyyah of Engg, IIUM, Kuala Lumpur, Malaysia

References

[1] T. Serre, G. Kreiman, M. Kouh, C. Cadieu, U. Knoblich, and T. Poggio, "A quantitative theory of immediate visual recognition," Prog. Brain Res., vol. 165, pp. 33–56, Jan. 2007. https://doi.org/10.1016/S0079-6123(06)65004-8 DOI: https://doi.org/10.1016/S0079-6123(06)65004-8

[2] N. Khan and S. Fahad, "Critical Review of the Present Situation of Corona Virus in China," SSRN Electron. J., Feb. 2020. https://doi.org/10.2139/SSRN.3543177 DOI: https://doi.org/10.2139/ssrn.3543177

[3] P. Kaur, K. Krishan, S. K. Sharma, and T. Kanchan, "Facial-recognition algorithms: A literature review," https://doi.org/10.1177/0025802419893168, vol. 60, no. 2, pp. 131–139, Jan. 2020. https://doi.org/10.1177/0025802419893168 DOI: https://doi.org/10.1177/0025802419893168

[4] P. Viola and M. J. Jones, "Robust Real-Time Face Detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137–154, May 2004. https://doi.org/10.1023/B: VISI.0000013087.49260.fb DOI: https://doi.org/10.1023/B:VISI.0000013087.49260.fb

[5] M. K. Hasan, M. S. Ahsan, Abdullah-Al-Mamun, S. H. S. Newaz, and G. M. Lee, "Human Face Detection Techniques: A Comprehensive Review and Future Research Directions," Electron. 2021, Vol. 10, Page 2354, vol. 10 no. 19, p. 2354, Sep. 2021. https://doi.org/10.3390/electronics10192354 DOI: https://doi.org/10.3390/electronics10192354

[6] R. Ranjan et al., "A Fast and Accurate System for Face Detection, Identification, and Verification," IEEE Trans. Biometrics, Behav. Identity Sci., vol. 1, no. 2, pp. 82–96, Apr. 2019. https://doi.org/10.1109/TBIOM.2019.2908436. DOI: https://doi.org/10.1109/TBIOM.2019.2908436

[7] S. Hemathilaka and A. Aponso, "A Comprehensive Study on Occlusion Invariant Face Recognition under Face Mask Occlusion," Jan. 2022. https://doi.org/10.5121/csit.2021.111804 DOI: https://doi.org/10.5121/csit.2021.111804

[8] S. Stein and G. A. Fink, "A new method for combined face detection and identification using interest point descriptors," 2011 IEEE Int. Conf. Autom. Face Gesture Recognition. Work. FG 2011, pp. 519–524, 2011. https://doi.org/ 10.1109/FG.2011.5771452 DOI: https://doi.org/10.1109/FG.2011.5771452

[9] H. Zeng, F. Qin, and K. Lin, "An Optimized Face Detection Based on Adaboost Algorithm," Proc. 2018 Int. Conf. Inf. Syst. Comput. Aided Educ. ICISCAE 2018, pp. 375–378, Mar. 2019. https://doi.org/ 10.1109/ICISCAE.2018.8666925 DOI: https://doi.org/10.1109/ICISCAE.2018.8666925

[10] W. Chen, H. Huang, S. Peng, C. Zhou, and C. Zhang, "YOLO-face: a real-time face detector," Vis. Comput., vol. 37, no. 4, pp. 805–813, Apr. 2021. https://doi.org/10.1007/S00371-020-01831-7 DOI: https://doi.org/10.1007/s00371-020-01831-7

[11] B. Wang, J. Zheng, and C. L. P. Chen, "A Survey on Masked Facial Detection Methods and Datasets for Fighting Against COVID-19," IEEE Trans. Artif. Intell., vol. 3, no. 3, pp. 323–343, 2022. https://doi.org/10.1109/TAI.2021.3139058 DOI: https://doi.org/10.1109/TAI.2021.3139058

[12] S. Ge, J. Li, Q. Ye, and Z. Luo, "Detecting masked faces in the wild with LLE-CNNs," Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 426–434, 2017. https://doi.org/10.1109/CVPR.2017.53 DOI: https://doi.org/10.1109/CVPR.2017.53

[13] A. Cabani, K. Hammoudi, H. Benhabiles, and M. Melkemi, "MaskedFace-Net – A dataset of correctly/incorrectly masked face images in the context of COVID-19," Smart Heal., vol. 19, p. 100144, Mar. 2021. https://doi.org/10.1016/J.SMHL.2020.100144 DOI: https://doi.org/10.1016/j.smhl.2020.100144

[14] B. Roy, S. Nandy, · Debojit Ghosh, · Debarghya Dutta, · Pritam Biswas, and T. Das, "MOXA: A Deep Learning Based Unmanned Approach for Real-Time Monitoring of People Wearing Medical Masks," Trans. Indian Natl. Acad. Eng. 2020 53, vol. 5, no. 3, pp. 509–518, Jul. 2020. https://doi.org/10.1007/S41403-020-00157-z DOI: https://doi.org/10.1007/s41403-020-00157-z

[15] T. Karras, S. Laine, and T. Aila, "A Style-Based Generator Architecture for Generative Adversarial Networks." pp. 4401–4410, 2019. https://doi.org/10.1109/CVPR.2019.00453 DOI: https://doi.org/10.1109/CVPR.2019.00453

[16] G. B. Huang and E. Learned-Miller, "Labeled Faces in the Wild: Updates and New Reporting Procedures," Accessed: Dec. 21, 2022. [Online]. Available: http://vis-www.cs.umass.edu/.

[17] Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep Learning Face Attributes in the Wild." pp. 3730–3738, 2015, Accessed: Dec. 21, 2022. [Online]. Available: http://personal.ie.cuhk.edu.hk/.

[18] F. I. Eyiokur, H. K. Ekenel, and A. Waibel, "Unconstrained face mask and face-hand interaction datasets: building a computer vision system to help prevent the transmission of COVID-19," Signal, Image Video Process., pp. 1–8, Jul. 2022. https://doi.org/10.1007/s11760-022-02308-x DOI: https://doi.org/10.1007/s11760-022-02308-x

[19] S. Singh, U. Ahuja, M. Kumar, K. Kumar, and M. Sachdeva, "Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment," Multimed. Tools Appl., vol. 80, no. 13, pp. 19753–19768, May 2021. https://doi.org/10.1007/s11042-021-10711-8 DOI: https://doi.org/10.1007/s11042-021-10711-8

[20] S. Yang, P. Luo, C.-C. Loy, and X. Tang, "WIDER FACE: A Face Detection Benchmark." pp. 5525–5533, 2016. https://doi.org/10.1109/CVPR.2016.596 DOI: https://doi.org/10.1109/CVPR.2016.596

[21] A. Kumar, A. Kaur, and M. Kumar, "Face detection techniques: a review," Artif. Intell. Rev., vol. 52, no. 2, pp. 927–948, 2019. https://doi.org/10.1007/s10462-018-9650-2 DOI: https://doi.org/10.1007/s10462-018-9650-2

[22] D. S. Brar, A. Kumar, Pallavi, U. Mittal, and P. Rana, "Face Detection for Real World Application," Proc. 2021 2nd Int. Conf. Intell. Eng. Manag. ICIEM 2021, pp. 239–242, Apr. 2021, https://doi.org/10.1109/ICIEM51511.2021.9445287 DOI: https://doi.org/10.1109/ICIEM51511.2021.9445287

[23] A. Review, "Past, Present, and Future of Face Recognition: A Review," 2020.

[24] K. Nagano et al., "Deep face normalization," ACM Trans. Graph., vol. 38, no. 6, Nov. 2019. https://doi.org/10.1145/3355089.3356568 DOI: https://doi.org/10.1145/3355089.3356568

[25] S. Ayesha, M. K. Hanif, and R. Talib, "Overview and comparative study of dimensionality reduction techniques for high dimensional data," Inf. Fusion, vol. 59, pp. 44–58, Jul. 2020. https://doi.org/10.1016/J.INFFUS.2020.01.005 DOI: https://doi.org/10.1016/j.inffus.2020.01.005

[26] B. Venkatesh and J. Anuradha, "A review of Feature Selection and its methods," Cybern. Inf. Technol., vol. 19, no. 1, pp. 3–26, 2019, https://doi.org/10.2478/CAIT-2019-0001 DOI: https://doi.org/10.2478/cait-2019-0001

[27] A. Alahmadi, M. Hussain, H. A. Aboalsamh, and M. Zuair, "PCAPooL: unsupervised feature learning for face recognition using PCA, LBP, and pyramid pooling," Pattern Anal. Appl., vol. 23, no. 2, pp. 673–682, May 2020. https://doi.org/10.1007/s10044-019-00818-y DOI: https://doi.org/10.1007/s10044-019-00818-y

[28] A. Ouyang, Y. Liu, S. Pei, X. Peng, M. He, and Q. Wang, "A hybrid improved kernel LDA and PNN algorithm for efficient face recognition," Neurocomputing, vol. 393, pp. 214–222, Jun. 2020. https://doi.org/10.1016/j.neucom.2019.01.117 DOI: https://doi.org/10.1016/j.neucom.2019.01.117

[29] L. Li, X. Mu, S. Li, and H. Peng, "A Review of Face Recognition Technology," IEEE Access, vol. 8, pp. 139110–139120, 2020. https://doi.org/10.1109/ACCESS.2020.3011028 DOI: https://doi.org/10.1109/ACCESS.2020.3011028

[30] P. VenkateswarLal, G. R. Nitta, and A. Prasad, "Ensemble of texture and shape descriptors using support vector machine classification for face recognition," J. Ambient Intell. Humaniz. Comput., pp. 1–8, Apr. 2019. https://doi.org/10.1007/S12652-019-01192-7 DOI: https://doi.org/10.1007/s12652-019-01192-7

[31] K. Sowmya, "Facial Recognition for Automated Attendance System Using Ada Boost Algorithm."

[32] F. Xu, J. Gao, and X. Pan, "Cow Face Recognition for a Small Sample Based on Siamese DB Capsule Network," IEEE Access, vol. 10, pp. 63189–63198, 2022. https://doi.org/10.1109/ACCESS.2022.3182806 DOI: https://doi.org/10.1109/ACCESS.2022.3182806

[33] S. Almabdy and L. Elrefaei, "Deep Convolutional Neural Network-Based Approaches for Face Recognition," Appl. Sci. 2019, Vol. 9, Page 4397, vol. 9, no. 20, p. 4397, Oct. 2019. https://doi.org/ 10.3390/APP9204397 DOI: https://doi.org/10.3390/app9204397

[34] J. Zhang, S. Shan, M. Kan, and X. Chen, "Coarse-to-Fine Auto-encoder Networks (CFAN) for real-time face alignment," Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8690 LNCS, no. PART 2, pp. 1–16, 2014. https://doi.org/10.1007/978-3-319-10605-2_1 DOI: https://doi.org/10.1007/978-3-319-10605-2_1

[35] D. Schofield et al., "Chimpanzee face recognition from videos in the wild using deep learning," Sci. Adv., vol. 5, no. 9, Sep. 2019. https://doi.org/10.1126/sciadv.aaw0736 DOI: https://doi.org/10.1126/sciadv.aaw0736

[36] P. Li, L. Prieto, D. Mery, and P. J. Flynn, "On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques," IEEE Trans. Inf. Forensics Secur., vol. 14, no. 8, pp. 2000–2012, Aug. 2019. https://doi.org/10.1109/TIFS.2018.2890812 DOI: https://doi.org/10.1109/TIFS.2018.2890812

[37] S. H. Oh, G. W. Kim, and K. S. Lim, "Compact deep learned feature-based face recognition for Visual Internet of Things," J. Supercomput., vol. 74, no. 12, pp. 6729–6741, Dec. 2018. https://doi.org/10.1007/S11227-017-2198-0 DOI: https://doi.org/10.1007/s11227-017-2198-0

[38] L. C. Jain, U. Halici, I. Hayashi, S. B. Lee, and S. Tsutsui, "Intelligent Biometric Techniques in Fingerprint and Face Recognition," Intell. Biometric Tech. Fingerpr. Face Recognition., no. July 1997, pp. 1–463, 2022, https://doi.org/10.1201/9780203750520 DOI: https://doi.org/10.1201/9780203750520

[39] P. Payal and M. M. Goyani, "A comprehensive study on face recognition: methods and challenges," https://doi.org/10.1080/13682199.2020.1738741, vol. 68, no. 2, pp. 114–127, Feb. 2020. https://doi.org/10.1080/13682199.2020.1738741 DOI: https://doi.org/10.1080/13682199.2020.1738741

[40] K. Amit, "Artificial Intelligence and Soft Computing : Behavioral and Cognitive Modeling of the Human Brain," Artif. Intell. Soft Comput., Oct. 2018. https://doi.org/10.1201/9781315219738 DOI: https://doi.org/10.1201/9781315219738

[41] Y. I. Daradkeh, I. Tvoroshenko, V. Gorokhovatskyi, L. A. Latiff, and N. Ahmad, "Development of effective methods for structural image recognition using the principles of data granulation and apparatus of fuzzy logic," IEEE Access, vol. 9, pp. 13417–13428, 2021. https://doi.org/10.1109/ACCESS.2021.3051625 DOI: https://doi.org/10.1109/ACCESS.2021.3051625

[42] S. Katoch, S. S. Chauhan, and V. Kumar, "A review on the genetic algorithm: past, present, and future," Multimed. Tools Appl., vol. 80, no. 5, pp. 8091–8126, Feb. 2021. https://doi.org/10.1007/s11042-020-10139-6 DOI: https://doi.org/10.1007/s11042-020-10139-6

[43] J. Li, T. Qiu, C. Wen, K. Xie, and F. Q. Wen, "Robust Face Recognition Using the Deep C2D-CNN Model Based on Decision-Level Fusion," Sensors 2018, Vol. 18, Page 2080, vol. 18, no. 7, p. 2080, Jun. 2018. https://doi.org/10.3390/S18072080 DOI: https://doi.org/10.3390/s18072080

[44] M. P. Beham and S. M. M. Roomi, "A review of face recognition methods," Int. J. Pattern Recognit. Artif. Intell., vol. 27, no. 4, pp. 1–35, 2013. https://doi.org/10.1142/S0218001413560053 DOI: https://doi.org/10.1142/S0218001413560053

[45] J. J. Howard, L. R. Rabbitt, and Y. B. Sirotin, "Human-algorithm teaming in face recognition: How algorithm outcomes cognitively bias human decision-making," PLoS One, vol. 15, no. 8, p. e0237855, Aug. 2020. https://doi.org/10.1371/journal.pone.0237855 DOI: https://doi.org/10.1371/journal.pone.0237855

[46] P. Grother, M. Ngan, K. Hanaoka, W. Ross, and W. Copan, "Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects," 2019. https://doi.org/10.6028/NIST.IR.8280 DOI: https://doi.org/10.6028/NIST.IR.8280

[47] M. Alexiou, I. P. Ktistakis, and G. Goodman, "Towards a Masked Face Recognition Algorithm: A Novel Rule Based Hybrid Algorithm," 6th South-East Eur. Des. Autom. Comput. Eng. Comput. Networks Soc. Media Conf. SEEDA-CECNSM 2021, 2021. https://doi.org/10.1109/SEEDA-CECNSM53056.2021.9566244 DOI: https://doi.org/10.1109/SEEDA-CECNSM53056.2021.9566244

[48] Y. Martínez-Díaz, H. Méndez-Vázquez, L. S. Luevano, M. Nicolás-Díaz, L. Chang, and M. González-Mendoza, “Towards Accurate and Lightweight Masked Face Recognition: An Experimental Evaluation,” IEEE Access, vol. 10, pp. 7341–7353, 2022. https://doi.org/10.1109/ACCESS.2021.3135255 DOI: https://doi.org/10.1109/ACCESS.2021.3135255

[49] H. Du, H. Shi, Y. Liu, D. Zeng, and T. Mei, "Towards NIR-VIS Masked Face Recognition," IEEE Signal Process. Lett., vol. 28, pp. 768–772, 2021. https://doi.org/10.1109/LSP.2021.3071663 DOI: https://doi.org/10.1109/LSP.2021.3071663

[50] K. Wang et al., "Mask Aware Network for Masked Face Recognition in the Wild," Proc. IEEE Int. Conf. Comput. Vis., vol. 2021-Octob, pp. 1456–1461, 2021. https://doi.org/10.1109/ICCVW54120.2021.00168 DOI: https://doi.org/10.1109/ICCVW54120.2021.00168

[51] J. Yu, X. Hao, Z. Cui, P. He, and T. Liu, "Boosting Fairness for Masked Face Recognition," Proc. IEEE Int. Conf. Comput. Vis., vol. 2021-Octob, pp. 1531–1540, 2021. https://doi.org/10.1109/ICCVW54120.2021.00178 DOI: https://doi.org/10.1109/ICCVW54120.2021.00178

[52] Y. Zhang, X. Wang, M. S. Shakeel, H. Wan, and W. Kang, "Learning upper patch attention using dual-branch training strategy for masked face recognition," Pattern Recognit., vol. 126, 2022. https://doi.org/10.1016/j.patcog.2022.108522 DOI: https://doi.org/10.1016/j.patcog.2022.108522

[53] N. Ullah, A. Javed, M. Ali Ghazanfar, A. Alsufyani, and S. Bourouis, "A novel DeepMaskNet model for face mask detection and masked facial recognition," J. King Saud Univ. - Comput. Inf. Sci., Volume 34, Issue 10, Part B, 2022. https://doi.org/10.1016/j.jksuci.2021.12.017 DOI: https://doi.org/10.1016/j.jksuci.2021.12.017

[54] R. Mar-Cupido, V. García, G. Rivera, and J. S. Sánchez, "Deep transfer learning for the recognition of types of face masks as a core measure to prevent the transmission of COVID-19," Appl. Soft Comput., vol. 125, p. 109207, 2022. https://doi.org/10.1016/j.asoc.2022.109207 DOI: https://doi.org/10.1016/j.asoc.2022.109207

[55] M. Gupta, G. Chaudhary, D. Bansal, and S. Pandey, "DTLMV2—A real-time deep transfer learning mask classifier for overcrowded spaces," Appl. Soft Comput., vol. 127, no. June 2020, p. 109313, 2022. https://doi.org/10.1016/j.asoc.2022.109313 DOI: https://doi.org/10.1016/j.asoc.2022.109313

[56] Y. Wang, Z. Yang, Z. Zhang, H. Zang, Q. Zhu, and S. Zhan, "Masked Face Recognition with 3D Facial Geometric Attributes," ACM Int. Conf. Proceeding Ser., pp. 13–21, 2022. https://doi.org/10.1145/3529446.3529449 DOI: https://doi.org/10.1145/3529446.3529449

[57] F. Ennaama, K. Benhida, and S. Essalki, "Proposed approach to improve facial recognition techniques for occluded faces by Covid-19 mask protection," IFAC-PapersOnLine, vol. 55, no. 12, pp. 456–461, 2022. https://doi.org/10.1016/j.ifacol.2022.07.354 DOI: https://doi.org/10.1016/j.ifacol.2022.07.354

[58] G. Kaur et al., "Face mask recognition system using CNN model," Neurosci. Informatics, vol. 2, no. 3, p. 100035, 2022. https://doi.org/10.1016/j.neuri.2021.100035 DOI: https://doi.org/10.1016/j.neuri.2021.100035

[59] W. Moungsouy, T. Tawanbunjerd, N. Liamsomboon, and W. Kusakunniran, "Face recognition under mask-wearing based on residual inception networks," Appl. Comput. Informatics, 2022. https://doi.org/10.1108/ACI-09-2021-0256 DOI: https://doi.org/10.1108/ACI-09-2021-0256

[60] Y. Bong and G. Lee, "International Journal on Robotics, Automation and Sciences," vol. 3, pp. 33–41, 2021. https://doi.org/10.33093/ijoras.2021.3.6 DOI: https://doi.org/10.33093/ijoras.2021.3.6

[61] N. Damer, F. Boutros, M. Süßmilch, M. Fang, F. Kirchbuchner, and A. Kuijper, "Masked face recognition: Human versus machine," IET Biometrics, vol. 11, no. 5, pp. 512–528, 2022. https://doi.org/10.1049/bme2.12077 DOI: https://doi.org/10.1049/bme2.12077