1: \begin{abstract}

2: In this paper, we study constrained nonconvex optimization problems in machine learning. It is well-known that these problems can be rewritten as a min-max problem in a Lagrangian form. However, due to the lack of convexity, their landscape is not well understood and how to find the stable equilibria of the Lagrangian function is still unknown. To bridge the gap, we study the landscape of the Lagrangian function. Further, we define a special class of Lagrangian functions. They enjoy two good properties: 1.Equilibria are either stable or unstable; 2.Stable equilibria correspond to the global optima of the original problem. We show that a generalized eigenvalue (GEV) problem, including canonical correlation analysis and other machine learning problems, belongs to this class. Specifically, we characterize its stable and unstable equilibria (more details in Section~\ref{GEV}). Motivated by these neat geometric structures, we propose a simple, efficient, stochastic primal-dual algorithm to solve online GEV. Theoretically, under certain regularity conditions, we establish an asymptotic rate of convergence and obtain the first asymptotic sample complexity result for online GEV by diffusion approximations. Numerical results are provided to support our theory.

3: \end{abstract}

4: