1: \begin{abstract}

2: 

3: Many recent works have studied the eigenvalue spectrum of the Conjugate Kernel

4: (CK) defined by the nonlinear feature map of a feedforward neural network.

5: However, existing results only establish weak convergence of the empirical

6: eigenvalue distribution, and fall short of providing precise quantitative

7: characterizations of the ``spike'' eigenvalues and eigenvectors that

8: often capture the low-dimensional signal structure of the 

9: learning problem. In this work, we characterize these signal eigenvalues and

10: eigenvectors for a nonlinear version of the spiked covariance model,

11: including the CK as a special case. Using this general result, we give

12: a quantitative description of how spiked eigenstructure in the input data

13: propagates through the hidden layers of a neural network with random weights.

14: As a second application, we study a simple regime of representation learning

15: where the weight matrix develops a rank-one signal component over training

16: and characterize the alignment of the target function with the spike

17: eigenvector of the CK on test data.

18: 

19: \end{abstract}

20: