Abstract
This research explores the positive application of deepfake technology forupper body generation, specifically sign language for the Deaf and Hard ofHearing (DHoH) community. Given the complexity of sign language and thescarcity of experts, the generated videos are vetted by a sign language expertfor accuracy. We construct a reliable deepfake dataset, evaluating itstechnical and visual credibility using computer vision and natural languageprocessing models. The dataset, consisting of over 1200 videos featuring bothseen and unseen individuals, is also used to detect deepfake videos targetingvulnerable individuals. Expert annotations confirm that the generated videosare comparable to real sign language content. Linguistic analysis, usingtextual similarity scores and interpreter evaluations, shows that theinterpretation of generated videos is at least 90% similar to authentic signlanguage. Visual analysis demonstrates that convincingly realistic deepfakescan be produced, even for new subjects. Using a pose/style transfer model, wepay close attention to detail, ensuring hand movements are accurate and alignwith the driving video. We also apply machine learning algorithms to establisha baseline for deepfake detection on this dataset, contributing to thedetection of fraudulent sign language videos.