Abstract
Neural architecture search methods are able to find high performance deeplearning architectures with minimal effort from an expert. However, currentsystems focus on specific use-cases (e.g. convolutional image classifiers andrecurrent language models), making them unsuitable for general use-cases thatan expert might wish to write. Hyperparameter optimization systems aregeneral-purpose but lack the constructs needed for easy application toarchitecture search. In this work, we propose a formal language for encodingsearch spaces over general computational graphs. The language constructs allowus to write modular, composable, and reusable search space encodings and toreason about search space design. We use our language to encode search spacesfrom the architecture search literature. The language allows us to decouple theimplementations of the search space and the search algorithm, allowing us toexpose search spaces to search algorithms through a consistent interface. Ourexperiments show the ease with which we can experiment with differentcombinations of search spaces and search algorithms without having to implementeach combination from scratch. We release an implementation of our languagewith this paper.