Abstract
Binary Neural Networks (BNNs) are neural networks which use binary weightsand activations instead of the typical 32-bit floating point values. They havereduced model sizes and allow for efficient inference on mobile or embeddeddevices with limited power and computational resources. However, thebinarization of weights and activations leads to feature maps of lower qualityand lower capacity and thus a drop in accuracy compared to traditionalnetworks. Previous work has increased the number of channels or used multiplebinary bases to alleviate these problems. In this paper, we instead presentMeliusNet consisting of alternating two block designs, which consecutivelyincrease the number of features and then improve the quality of these features.In addition, we propose a redesign of those layers that use 32-bit values inprevious approaches to reduce the required number of operations. Experiments onthe ImageNet dataset demonstrate the superior performance of our MeliusNet overa variety of popular binary architectures with regards to both computationsavings and accuracy. Furthermore, with our method we trained BNN models, whichfor the first time can match the accuracy of the popular compact networkMobileNet in terms of model size and accuracy. Our code is published online:https://github.com/hpi-xnor/BMXNet-v2