1: \begin{abstract}

2: This paper proposes a novel algorithm for training recurrent neural network models of nonlinear dynamical systems from an input/output training dataset. Arbitrary convex and twice-differentiable loss functions and regularization terms are handled by sequential least squares and either a line-search (LS) or a trust-region method of Levenberg-Marquardt (LM) type for ensuring convergence.

3: In addition, to handle non-smooth regularization terms such as $\ell_1$, $\ell_0$, and group-Lasso regularizers, as well as to impose possibly non-convex constraints such as integer and mixed-integer constraints, we combine sequential least squares

4: with the alternating direction method of multipliers (ADMM). We call the resulting algorithm

5: NAILS (nonconvex ADMM iterations and least squares) in the case

6: line search (LS) is used, or NAILM if a trust-region method (LM) is employed instead.

7: The training method, which is also applicable to feedforward neural networks as a special case,

8: is tested in three nonlinear system identification problems. 

9: \end{abstract}

10: