Abstract
Resistive crossbars have emerged as promising building blocks for realizingDNNs due to their ability to compactly and efficiently realize the dominant DNNcomputational kernel, viz., vector-matrix multiplication. However, a keychallenge with resistive crossbars is that they suffer from a range of deviceand circuit level non-idealities such as interconnect parasitics, peripheralcircuits, sneak paths, and process variations. These non-idealities can lead toerrors in vector-matrix multiplication that eventually degrade the DNN'saccuracy. There has been no study of the impact of non-idealities on theaccuracy of large-scale DNNs, in part because existing device and circuitmodels are infeasible to use in application-level evaluation. In this work, wepresent a fast and accurate simulation framework to enable evaluation andre-training of large-scale DNNs on resistive crossbar based hardware fabrics. We first characterize the impact of crossbar non-idealities on errorsincurred in the realized vector-matrix multiplications and observe that theerrors have significant data and hardware-instance dependence that should beconsidered. We propose a Fast Crossbar Model (FCM) to accurately capture theerrors arising due to crossbar non-idealities while being four-to-five ordersof magnitude faster than circuit simulation. Finally, we develop RxNN, asoftware framework to evaluate and re-train DNNs on resistive crossbar systems.RxNN is based on the popular Caffe machine learning framework, and we use it toevaluate a suite of large-scale DNNs developed for the ImageNet Challenge(ILSVRC). Our experiments reveal that resistive crossbar non-idealities canlead to significant accuracy degradations (9.6%-32%) for these large-scaleDNNs. To the best of our knowledge, this work is the first quantitativeevaluation of the accuracy of large-scale DNNs on resistive crossbar basedhardware.