We develop a general approach to distill symbolic representations of alearned deep model by introducing strong inductive biases. We focus on GraphNeural Networks (GNNs). The technique works as follows: we first encouragesparse latent representations when we train a GNN in a supervised setting, thenwe apply symbolic regression to components of the learned model to extractexplicit physical relations. We find the correct known equations, includingforce laws and Hamiltonians, can be extracted from the neural network. We thenapply our method to a non-trivial cosmology example-a detailed dark mattersimulation-and discover a new analytic formula which can predict theconcentration of dark matter from the mass distribution of nearby cosmicstructures. The symbolic expressions extracted from the GNN using our techniquealso generalized to out-of-distribution data better than the GNN itself. Ourapproach offers alternative directions for interpreting neural networks anddiscovering novel physical principles from the representations they learn.