Abstract
We present a mechanism to compute a sketch (succinct summary) of how acomplex modular deep network processes its inputs. The sketch summarizesessential information about the inputs and outputs of the network and can beused to quickly identify key components and summary statistics of the inputs.Furthermore, the sketch is recursive and can be unrolled to identifysub-components of these components and so forth, capturing a potentiallycomplicated DAG structure. These sketches erase gracefully; even if we erase afraction of the sketch at random, the remainder still retains the `high-weight'information present in the original sketch. The sketches can also be organizedin a repository to implicitly form a `knowledge graph'; it is possible toquickly retrieve sketches in the repository that are related to a sketch ofinterest; arranged in this fashion, the sketches can also be used to learnemerging concepts by looking for new clusters in sketch space. Finally, in thescenario where we want to learn a ground truth deep network, we show thataugmenting input/output pairs with these sketches can theoretically make iteasier to do so.