In recent years, researchers have proposed many deep learning (DL) methodsfor various tasks, and particularly face recognition (FR) made an enormous leapusing these techniques. Deep FR systems benefit from the hierarchicalarchitecture of the DL methods to learn discriminative face representation.Therefore, DL techniques significantly improve state-of-the-art performance onFR systems and encourage diverse and efficient real-world applications. In thispaper, we present a comprehensive analysis of various FR systems that leveragethe different types of DL techniques, and for the study, we summarize 168recent contributions from this area. We discuss the papers related to differentalgorithms, architectures, loss functions, activation functions, datasets,challenges, improvement ideas, current and future trends of DL-based FRsystems. We provide a detailed discussion of various DL methods to understandthe current state-of-the-art, and then we discuss various activation and lossfunctions for the methods. Additionally, we summarize different datasets usedwidely for FR tasks and discuss challenges related to illumination, expression,pose variations, and occlusion. Finally, we discuss improvement ideas, currentand future trends of FR tasks.