353a920a9fb2da71.tex
1: \begin{abstract}%   <- trailing '%' for backward compatibility of .sty file
2: Deep neural networks (DNNs) have made great progress in various fields. In particular, the quantized neural network is a promising technique making DNNs compatible on resource-limited devices for  memory and computation saving. In this paper, we mainly consider a non-convex minimization model with three blocks to train quantized DNNs and propose a new stochastic three-block alternating minimization (STAM) algorithm to solve it.  We develop a convergence theory for the STAM algorithm and obtain an $\epsilon$-stationary point with optimal convergence rate $\mathcal{O}(\epsilon^{-4})$. Furthermore, we apply our STAM algorithm to train DNNs with relaxed binary weights. The  experiments are carried out on three different network structures, namely VGG-11, VGG-16 and ResNet-18. These DNNs are trained using two different data sets, CIFAR-10 and CIFAR-100, respectively. We compare our STAM algorithm with  some classical efficient algorithms for training quantized neural networks. The test accuracy indicates the effectiveness of STAM algorithm for training relaxed binary quantization DNNs.
3: \end{abstract}
4: