1: \begin{abstract}

2: 

3: This study presents a novel approach for understanding the properties of normalization layers in multilayer perceptrons (MLPs) by introducing isometric gap, a symmetry-preserving metric that measures the deviation from isometry of a covariance or Gram matrix. We explore the behavior of the isometric gap in the context of a normalized MLP, revealing its contractive properties and algebraic features. Our findings indicate that the isometric gap is non-decreasing under normalization and increases when the norms of input rows vary. This insight can be applied to characterize the eigenvalue spectra of MLP representations in depth, helping to advance our understanding of deep learning networks and their behavior.

4: 

5:   % Summary of contributions: 

6:   % \begin{itemize}

7:   %     \item Deterministic contraction with BN (we need a plot for this result and show that the volume between 3 vectors increase with BN). 

8:   %     \item Global convergence of mean-field predictions 

9:   %     \item Going beyond mean-field theory

10:   % \end{itemize}

11: \end{abstract}

12: