1: \begin{abstract}%

2: 

3: Spectral methods which represent data points by eigenvectors of kernel

4: matrices or graph Laplacian matrices have been a primary tool in

5: unsupervised data analysis. In many application scenarios, parametrizing

6: the spectral embedding by a neural network that can be trained over

7: batches of data samples gives a promising way to achieve automatic

8: out-of-sample extension as well as computational scalability. Such an

9: approach was taken in the original paper of SpectralNet (Shaham et al.

10: 2018), which we call SpecNet1. The current paper introduces a new

11: neural network approach, named SpecNet2, to compute spectral embedding which optimizes an equivalent objective of the eigen-problem and removes the orthogonalization layer in SpecNet1. SpecNet2 also allows separating

12: the sampling of rows and columns of the graph affinity matrix by tracking

13: the neighbors of each data point through the gradient formula.

14: Theoretically, we show that any local minimizer of the new

15: orthogonalization-free objective reveals the leading eigenvectors.

16: Furthermore, global convergence for this new orthogonalization-free objective using a batch-based gradient descent method is proved. Numerical experiments demonstrate the improved

17: performance and computational efficiency of SpecNet2 on simulated data and

18: image datasets. 

19: 

20: \end{abstract}

21: