Abstract

We introduce a technique based on the singular vector canonical correlationanalysis (SVCCA) for measuring the generality of neural network layers across acontinuously-parametrized set of tasks. We illustrate this method by studyinggenerality in neural networks trained to solve parametrized boundary valueproblems based on the Poisson partial differential equation. We find that thefirst hidden layer is general, and that deeper layers are successively morespecific. Next, we validate our method against an existing technique thatmeasures layer generality using transfer learning experiments. We findexcellent agreement between the two methods, and note that our method is muchfaster, particularly for continuously-parametrized problems. Finally, wevisualize the general representations of the first layers, and interpret themas generalized coordinates over the input domain.

 

Introduction (beta)

None

 

Conclusion (beta)

None