Abstract
Deep Neural Networks have achieved remarkable progress during the past fewyears and are currently the fundamental tools of many intelligent systems. Atthe same time, the computational complexity and resource consumption of thesenetworks are also continuously increasing. This will pose a significantchallenge to the deployment of such networks, especially for real-timeapplications or on resource-limited devices. Thus, network acceleration havebecome a hot topic within the deep learning community. As for hardwareimplementation of deep neural networks, a batch of accelerators based onFPGA/ASIC have been proposed these years. In this paper, we provide acomprehensive survey about the recent advances on network acceleration,compression and accelerator design from both algorithm and hardware side.Specifically, we provide thorough analysis for each of the following topics:network pruning, low-rank approximation, network quantization, teacher-studentnetworks, compact network design and hardware accelerator. Finally, we make adiscussion and introduce a few possible future directions.