Recent years have witnessed an explosive growth of mobile devices. Mobiledevices are permeating every aspect of our daily lives. With the increasingusage of mobile devices and intelligent applications, there is a soaring demandfor mobile applications with machine learning services. Inspired by thetremendous success achieved by deep learning in many machine learning tasks, itbecomes a natural trend to push deep learning towards mobile applications.However, there exist many challenges to realize deep learning in mobileapplications, including the contradiction between the miniature nature ofmobile devices and the resource requirement of deep neural networks, theprivacy and security concerns about individuals' data, and so on. To resolvethese challenges, during the past few years, great leaps have been made in thisarea. In this paper, we provide an overview of the current challenges andrepresentative achievements about pushing deep learning on mobile devices fromthree aspects: training with mobile data, efficient inference on mobiledevices, and applications of mobile deep learning. The former two aspects coverthe primary tasks of deep learning. Then, we go through our two recentapplications that apply the data collected by mobile devices to inferring mooddisturbance and user identification. Finally, we conclude this paper with thediscussion of the future of this area.