1: \begin{abstract}

2: Continuous symmetries and their breaking play a prominent role in contemporary physics.

3: Effective low-energy field theories around symmetry breaking states explain diverse phenomena such as superconductivity, magnetism, and the mass of nucleons.

4: We show that such field theories can also be a useful tool in machine learning, in particular for loss functions with continuous symmetries that are spontaneously broken by random initializations.

5: In this paper, we illuminate our earlier published work~\citep{bamler2018improving} on this topic more from the perspective of theoretical physics.

6: We show that the analogies between superconductivity and symmetry breaking in temporal representation learning are rather deep, allowing us to formulate a gauge theory of `charged' embedding vectors in time series models.

7: We show that making the loss function gauge invariant speeds up convergence in such models.

8: \end{abstract}

9: