42221c24c4bff224.tex
1: \begin{abstract}%   <- trailing '%' for backward compatibility of .sty file
2: 		%{\color{red} There are some sentences missing in the beginning of the abstract to emphasize: (1) which problems we are interested in (2) why these problems are interesting (3) what are the difficulties.}
3: 		{The leaky ReLU network with a group sparse regularization term has been widely used in the recent years. } 
4: 		In this paper, we first  prove that the nonsmooth nonconvex minimization problem for training leaky ReLU neural network with a group sparse  regularization term has a nonempty and bounded solution set and its feasible set satisfies the Mangasarian-Fromovitz constraint qualification.  Moreover, we provide important properties of its KKT points and limiting stationary points, and the relation to stationary points of a mathematical program with complementarity problems.   Next, we propose an inexact augmented Lagrangian algorithm for solving the minimization problem, and show the convergence of the algorithm to a KKT point.  Numerical experiments demonstrate that
5: 		our algorithm is more efficient for training sparse leaky ReLU neural networks than some well-known algorithms.
6: 	\end{abstract}
7: