1: \begin{abstract}

2:   We investigate the statistical behavior of gradient descent iterates with

3:   dropout in the linear regression model. In particular, non-asymptotic bounds

4:   for the convergence of expectations and covariance matrices of the iterates

5:   are derived. The results shed more light on the widely cited connection

6:   between dropout and $\ell_2$-regularization in the linear model. We indicate a

7:   more subtle relationship, owing to interactions between the gradient descent

8:   dynamics and the additional randomness induced by dropout. Further, we study a

9:   simplified variant of dropout which does not have a regularizing effect and

10:   converges to the least squares estimator.

11: \end{abstract}

12: