Abstract
Dancing to music is an instinctive move by humans. Learning to model themusic-to-dance generation process is, however, a challenging problem. Itrequires significant efforts to measure the correlation between music and danceas one needs to simultaneously consider multiple aspects, such as style andbeat of both music and dance. Additionally, dance is inherently multimodal andvarious following movements of a pose at any moment are equally likely. In thispaper, we propose a synthesis-by-analysis learning framework to generate dancefrom music. In the analysis phase, we decompose a dance into a series of basicdance units, through which the model learns how to move. In the synthesisphase, the model learns how to compose a dance by organizing multiple basicdancing movements seamlessly according to the input music. Experimentalqualitative and quantitative results demonstrate that the proposed method cansynthesize realistic, diverse,style-consistent, and beat-matching dances frommusic.