1: \begin{abstract}

2: This paper proposes a novel first-order algorithm that solves composite nonsmooth and stochastic convex optimization problem with function constraints. %Such problems are becoming increasingly important in various machine learning and data science areas. 

3: Most of the works in the literature provide convergence rate guarantees on the average-iterate solution. There is growing interest in the convergence guarantees of the last iterate solution due to its favorable structural properties, such as sparsity

4: or privacy guarantees and good performance in practice. We provide the first method that obtains the best-known convergence rate guarantees on the last iterate for stochastic composite nonsmooth convex function-constrained optimization problems. Our novel and easy-to-implement algorithm is based on the augmented Lagrangian technique and uses a new linearized approximation of constraint functions, leading to its name, the Augmented Constraint Extrapolation ($\augconex$) method. We show that \augconex~achieves $\mathcal{O}(1/\sqrt{K})$ convergence rate in the nonsmooth stochastic setting without any strong convexity assumption and $\mathcal{O}(1/K)$ for the same problem with strongly convex objective function. While optimal for nonsmooth and stochastic problems, the \augconex~method also accelerates convergence in terms of Lipschitz smoothness constants to $\mathcal{O}(1/K)$ and $\mathcal{O}(1/K^2)$ in the aforementioned cases, respectively. To our best knowledge, this is the first method to obtain such differentiated

5: convergence rate guarantees on the last iterate for a composite nonsmooth stochastic setting without

6: additional $\log{K}$ factors. We validate the efficiency of our algorithm by comparing it with a state-of-the-art algorithm through numerical experiments. 

7: \vspace{1em}

8: % \noindent {\bf Keywords:} Variational Inequality, Function Constraints, Stochastic first-order methods, Saddle-point problems with coupled constraints

9: \end{abstract}

10: