1: \begin{abstract}

2: The main aim of this paper is to provide an analysis of gradient descent ($GD$) algorithms

3: with gradient errors that do not necessarily vanish, asymptotically.

4: In particular, sufficient conditions are presented for both stability (almost sure boundedness of

5: the iterates) and convergence of $GD$ with bounded, (possibly) non-diminishing gradient errors. In addition

6: to ensuring stability, such an algorithm is shown to 

7: converge to a small neighborhood of the minimum set,

8: which depends on the gradient errors.

9: It is worth noting that the main result

10: of this paper can be used to show that $GD$ with asymptotically vanishing errors indeed converges 

11: to the

12: minimum set. 

13: The results presented herein are not only more general when compared to previous

14: results, but our analysis of \textit{$GD$ with errors} is new to the literature to the best of our

15: knowledge.

16: Our work extends the contributions of Mangasarian \& Solodov,

17: Bertsekas \& Tsitsiklis and Tadi{\'c} \& Doucet. 

18: Using our framework, a simple yet effective implementation of $GD$ using simultaneous

19: perturbation stochastic approximations ($SPSA$), with constant

20: sensitivity parameters, is presented. 

21: Another important improvement over many previous results is that there are

22: no `additional' restrictions imposed on the step-sizes. In machine learning applications

23: where step-sizes are related

24: to learning rates, our assumptions, unlike those of other papers, do not affect these learning

25: rates. Finally,

26: we present experimental results to validate our theory.

27: \end{abstract}

28: