1: \begin{abstract}%

2:   We consider nonlinear optimization problems that involve surrogate models represented by neural networks.

3:   We demonstrate first how to directly embed neural network evaluation into optimization models,

4:   highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models.

5:   We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as

6:   a mathematical program with complementarity constraints. 

7:   For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation.

8:   Each of these formulations may be solved with 

9:   state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods.

10:   We compare our formulations on three practical applications arising in the design and control of 

11:   combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.%

12:   \keywords{Mixed-integer programming \and nonlinear programming \and complementarity constraints \and machine learning \and neural networks}

13:   \subclass{MSC codes}

14: \end{abstract}

15: