Abstract
We propose a new building block, IdleBlock, which naturally prunesconnections within the block. To fully utilize the IdleBlock we break thetradition of monotonic design in state-of-the-art networks, and introducinghybrid composition with IdleBlock. We study hybrid composition on MobileNet v3and EfficientNet-B0, two of the most efficient networks. Without any neuralarchitecture search, the deeper "MobileNet v3" with hybrid composition designsurpasses possibly all state-of-the-art image recognition network designed byhuman experts or neural architecture search algorithms. Similarly, thehybridized EfficientNet-B0 networks are more efficient than previousstate-of-the-art networks with similar computation budgets. These resultssuggest a new simpler and more efficient direction for network design andneural architecture search.