1: \begin{abstract}

2:   Motivated by penalized likelihood maximization in complex models, we

3:   study optimization problems where neither the function to optimize

4:   nor its gradient have an explicit expression, but its gradient can

5:   be approximated by a Monte Carlo technique.  We propose a new

6:   algorithm based on a stochastic approximation of the

7:   Proximal-Gradient (PG) algorithm. This new algorithm, named

8:   Stochastic Approximation PG (SAPG) is the combination of a

9:   stochastic gradient descent step which - roughly speaking - computes

10:   a smoothed approximation of the gradient along the iterations, and a

11:   proximal step.  The choice of the step size and of the Monte Carlo

12:   batch size for the stochastic gradient descent step in SAPG are

13:   discussed. Our convergence results cover the cases of biased and

14:   unbiased Monte Carlo approximations. While the convergence analysis

15:   of some classical Monte Carlo approximation of the gradient is

16:   already addressed in the literature

17:   \citep[see][]{atchade2014stochastic}, the convergence analysis of SAPG

18:   is new.  Practical implementation is discussed and guidelines to

19:   tune the algorithm are given. The two algorithms are compared on a

20:   linear mixed effect model as a toy example. A more challenging

21:   application is proposed on non-linear mixed effect models in high

22:   dimension with a pharmacokinetic data set including genomic

23:   covariates. To our best knowledge, our work provides the first

24:   convergence result of a numerical method designed to solve penalized

25:   Maximum Likelihood in a non-linear mixed effect model.

26: 

27:  

28:     

29:     \textbf{Keywords:} Proximal-Gradient algorithm; Stochastic Gradient;

30:     Stochastic EM algorithm; Stochastic Approximation; Non-linear mixed effect

31:     models.

32: \end{abstract}

33: