1: \begin{abstract}

2: A growing trend in deep learning replaces fixed depth models by approximations of the limit as network depth approaches infinity. This approach uses a portion of network weights to prescribe behavior by defining a limit condition. 

3: This makes network depth \textit{implicit}, varying based on the provided data and an error tolerance.  

4: Moreover, existing implicit models can be implemented and trained with \textit{fixed} memory costs in exchange for additional computational costs.

5: % This typically comes with added computational cost.

6: In particular, backpropagation through implicit depth models requires solving a Jacobian-based equation arising from the implicit function theorem.

7: We propose fixed point networks (FPNs), a simple setup for implicit depth learning that guarantees convergence of forward propagation to a unique limit defined by network weights and input data. 

8: Our key contribution is to provide a new Jacobian-free backpropagation (JFB) scheme that circumvents the need to solve Jacobian-based equations while maintaining fixed memory costs.

9: This makes FPNs much cheaper to train and easy to implement.

10: Our numerical examples yield state of the art classification results for implicit depth models and outperform corresponding explicit models.\footnote{Codes are available on Github: \href{https://github.com/howardheaton/fixed_point_networks}{\texttt{github.com/howardheaton/fixed\_point\_networks}}}

11: \end{abstract}

12: