Abstract
Talking head generation intends to produce vivid and realistic talking headvideos from a single portrait and speech audio clip. Although significantprogress has been made in diffusion-based talking head generation, almost allmethods rely on autoregressive strategies, which suffer from limited contextutilization beyond the current generation step, error accumulation, and slowergeneration speed. To address these challenges, we present DAWN (Dynamic frameAvatar With Non-autoregressive diffusion), a framework that enables all-at-oncegeneration of dynamic-length video sequences. Specifically, it consists of twomain components: (1) audio-driven holistic facial dynamics generation in thelatent motion space, and (2) audio-driven head pose and blink generation.Extensive experiments demonstrate that our method generates authentic and vividvideos with precise lip motions, and natural pose/blink movements.Additionally, with a high generation speed, DAWN possesses strong extrapolationcapabilities, ensuring the stable production of high-quality long videos. Theseresults highlight the considerable promise and potential impact of DAWN in thefield of talking head video generation. Furthermore, we hope that DAWN sparksfurther exploration of non-autoregressive approaches in diffusion models. Ourcode will be publicly available at https://github.com/Hanbo-Cheng/DAWN-pytorch.