Abstract
Neural networks for image recognition have evolved through extensive manualdesign from simple chain-like models to structures with multiple wiring paths.The success of ResNets and DenseNets is due in large part to their innovativewiring plans. Now, neural architecture search (NAS) studies are exploring thejoint optimization of wiring and operation types, however, the space ofpossible wirings is constrained and still driven by manual design despite beingsearched. In this paper, we explore a more diverse set of connectivity patternsthrough the lens of randomly wired neural networks. To do this, we first definethe concept of a stochastic network generator that encapsulates the entirenetwork generation process. Encapsulation provides a unified view of NAS andrandomly wired networks. Then, we use three classical random graph models togenerate randomly wired graphs for networks. The results are surprising:several variants of these random generators yield network instances that havecompetitive accuracy on the ImageNet benchmark. These results suggest that newefforts focusing on designing better network generators may lead to newbreakthroughs by exploring less constrained search spaces with more room fornovel design.