Abstract
As the backbone technology of machine learning, deep neural networks (DNNs)have have quickly ascended to the spotlight. Running DNNs onresource-constrained mobile devices is, however, by no means trivial, since itincurs high performance and energy overhead. While offloading DNNs to the cloudfor execution suffers unpredictable performance, due to the uncontrolled longwide-area network latency. To address these challenges, in this paper, wepropose Edgent, a collaborative and on-demand DNN co-inference framework withdevice-edge synergy. Edgent pursues two design knobs: (1) DNN partitioning thatadaptively partitions DNN computation between device and edge, in order toleverage hybrid computation resources in proximity for real-time DNN inference.(2) DNN right-sizing that accelerates DNN inference through early-exit at aproper intermediate DNN layer to further reduce the computation latency. Theprototype implementation and extensive evaluations based on Raspberry Pidemonstrate Edgent's effectiveness in enabling on-demand low-latency edgeintelligence.