1: \begin{abstract}

2: { We examine the squared error loss landscape of shallow linear neural networks. %By utilizing a regularizer on the training samples,

3: We show---with  significantly milder assumptions than previous works---that the corresponding optimization problems have benign geometric properties: there are no spurious local minima and the Hessian at every saddle point has at least one negative eigenvalue. This means that at every saddle point there is a directional negative curvature which algorithms can utilize to further decrease the objective value.  These geometric properties imply that many local search algorithms (such as the gradient descent which is widely utilized for training neural networks) can provably solve the training problem with global convergence.

4: % {Under the same mild assumption on the input data, we further show that these geometric properties are preserved without the additional regularizer.}

5: }

6: %The additional regularizer has no effect on the global minimum value; rather, it plays a useful role in shrinking the set of critical points.   %Experiments show that this additional regularizer also speeds the convergence of iterative algorithms for solving the training optimization problem in certain cases.

7: 

8: 

9: 

10: \keywords{deep learning \and linear neural network \and optimization geometry \and strict saddle \and spurious local minima}

11: % \PACS{PACS code1 \and PACS code2 \and more}

12: % \subclass{MSC code1 \and MSC code2 \and more}

13: \end{abstract}