1: \begin{abstract}

2:    Non-contrastive \ac{SSL} methods like \acs{BYOL} and SimSiam rely on asymmetric predictor networks to avoid representational collapse without negative samples. 

3:    Yet, how predictor networks facilitate stable learning is not fully understood.

4:    While previous theoretical analyses assumed Euclidean losses, most practical implementations rely on cosine similarity.

5:    To gain further theoretical insight into non-contrastive \ac{SSL}, we analytically study learning dynamics in conjunction with Euclidean and cosine similarity in the eigenspace of closed-form linear predictor networks.

6:    We show that both avoid collapse through implicit variance regularization albeit through different dynamical mechanisms.

7:    Moreover, we find that the eigenvalues act as effective learning rate multipliers and propose a family of isotropic loss functions (IsoLoss) that equalize convergence rates across eigenmodes.

8:    Empirically, IsoLoss speeds up the initial learning dynamics and increases robustness, thereby allowing us to dispense with the \ac{EMA} target network typically used with non-contrastive methods.

9:    Our analysis sheds light on the variance regularization mechanisms of non-contrastive \ac{SSL} and lays the theoretical grounds for crafting novel loss functions that shape the learning dynamics of the predictor's spectrum.

10: \end{abstract}

11: