12763e04ba7fccaf.tex
1: \begin{abstract}
2: %The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is useful in learning meaningful features from the given real-valued data. The difficulties associated with learning GB-RBM are reported extensively in earlier studies. They indicate that the training of GB-RBM using the standard algorithms like contrastive divergence (CD) and persistent contrastive divergence (PCD), require carefully chosen learning rate. A large learning rate results in divergence of the log-likelihood and a small learning rate leads to very slow convergence and poor local minima. Hence, careful parameter initialization and gradient restriction are used to update parameters.
3: % More importantly, the knowledge about the underlying data distribution is used to improve learning. In this work, we extend the stochastic {\em difference of convex functions} (DC) programming (S-DCP) algorithm proposed to learn Bernoulli-Bernoulli RBM (BB-RBM) to the GB-RBM case. In order to use S-DCP, we show that the GB-RBM log-likelihood too can be written as the difference of two convex functions like the BB-RBM case. We validate the performance of the S-DCP algorithm on several benchmark datasets and show that it is very efficient compared to the CD and PCD algorithms.  Through our experiments on the datasets, we show that the S-DCP algorithm is stable and fast.
4: %\end{abstract}
5: