1: \begin{abstract}

2: We study the $L_1$-regularized maximum likelihood estimator/estimation (MLE) problem for discrete Markov random fields (MRFs), where efficient and scalable learning requires both sparse regularization and approximate inference. To address these challenges, we consider a stochastic learning framework called stochastic proximal gradient (SPG; \citealt{honorio2012convergence, atchade2014stochastic, miasojedow2016sparse}). SPG is an \emph{inexact} proximal gradient algorithm \citep{schmidt2011convergence}, whose inexactness stems from the stochastic oracle (Gibbs sampling) for gradient approximation -- exact gradient evaluation is infeasible in general due to the NP-hard inference problem for discrete MRFs \citep{koller2009probabilistic}. Theoretically, we provide novel \emph{verifiable} bounds to inspect and control the quality of gradient approximation. Empirically, we propose the \emph{tighten asymptotically} (TAY) learning strategy based on the verifiable bounds to boost the performance of SPG.

3: \end{abstract}

4: