Abstract
Continuous symmetries and their breaking play a prominent role incontemporary physics. Effective low-energy field theories around symmetrybreaking states explain diverse phenomena such as superconductivity, magnetism,and the mass of nucleons. We show that such field theories can also be a usefultool in machine learning, in particular for loss functions with continuoussymmetries that are spontaneously broken by random initializations. In thispaper, we illuminate our earlier published work (Bamler & Mandt, 2018) on thistopic more from the perspective of theoretical physics. We show that theanalogies between superconductivity and symmetry breaking in temporalrepresentation learning are rather deep, allowing us to formulate a gaugetheory of `charged' embedding vectors in time series models. We show thatmaking the loss function gauge invariant speeds up convergence in such models.