1: \begin{abstract}

2: With an eye

3: toward understanding complexity control in deep learning, we study how infinitesimal regularization or gradient descent optimization lead to margin maximizing solutions in both homogeneous and {\em non homogeneous} models, extending previous work that focused on infinitesimal regularization only in homogeneous models.  To this end we study the limit of loss minimization with a diverging norm constraint (the ``constrained path''), relate it to the limit of a ``margin path'' and characterize the resulting solution.  For non-homogeneous ensemble models, which output is a sum of homogeneous sub-models, we show that this solution discards the shallowest sub-models if they are unnecessary. For homogeneous models, we show convergence to a ``lexicographic max-margin solution'', and provide conditions under which max-margin solutions are also attained as the limit of unconstrained gradient descent.

4: \end{abstract}

5: