Abstract
Comparing different neural network representations and determining howrepresentations evolve over time remain challenging open questions in ourunderstanding of the function of neural networks. Comparing representations inneural networks is fundamentally difficult as the structure of representationsvaries greatly, even across groups of networks trained on identical tasks, andover the course of training. Here, we develop projection weighted CCA(Canonical Correlation Analysis) as a tool for understanding neural networks,building off of SVCCA, a recently proposed method. We first improve the coremethod, showing how to differentiate between signal and noise, and then applythis technique to compare across a group of CNNs, demonstrating that networkswhich generalize converge to more similar representations than networks whichmemorize, that wider networks converge to more similar solutions than narrownetworks, and that trained networks with identical topology but differentlearning rates converge to distinct clusters with diverse representations. Wealso investigate the representational dynamics of RNNs, across both trainingand sequential timesteps, finding that RNNs converge in a bottom-up patternover the course of training and that the hidden state is highly variable overthe course of a sequence, even when accounting for linear transforms. Together,these results provide new insights into the function of CNNs and RNNs, anddemonstrate the utility of using CCA to understand representations.