1: \begin{abstract}

2:   Implicit neural networks, a.k.a., deep equilibrium networks, are a

3:   class of implicit-depth learning models where function evaluation is

4:   performed by solving a fixed point equation.

5: %%

6:   They generalize classic feedforward models and are equivalent to

7:   infinite-depth weight-tied feedforward networks.

8: %%

9:   While implicit models show improved accuracy and significant

10:   reduction in memory consumption, they can suffer from ill-posedness

11:   and convergence instability.

12: %%  

13:  

14:   This paper provides a new framework, {\color{black}which we call Non-Euclidean

15:   Monotone Operator Network (NEMON),} to design well-posed and robust

16:   implicit neural networks based upon contraction theory for the

17:   non-Euclidean norm $\ell_\infty$.

18: %%

19:   Our framework includes (i) a novel condition for well-posedness

20:   based on one-sided Lipschitz constants, (ii) an average iteration

21:   for computing fixed-points, and (iii) explicit estimates on

22:   input-output Lipschitz constants.

23:   %%

24:   Additionally, we design a training problem with the well-posedness

25:   condition and the average iteration as constraints and, to achieve

26:   robust models, with the input-output Lipschitz constant as a

27:   regularizer.

28: %%

29:   Our $\ell_\infty$ well-posedness condition leads to a larger

30:   polytopic training search space than existing conditions and our

31:   average iteration enjoys accelerated convergence.

32: %%

33:   {\color{black}Finally, we evaluate our framework in

34:   image classification through the MNIST and the CIFAR-10 datasets.} Our numerical

35:   results demonstrate improved accuracy and robustness of the implicit

36:   models with smaller input-output Lipschitz bounds. Code is available

37:   at \url{https://github.com/davydovalexander/Non-Euclidean_Mon_Op_Net}.

38: \end{abstract}

39: