d6a4c6d9c9775c30.tex
1: \begin{abstract} 
2:   Restricted Boltzmann Machines (RBMs) are general unsupervised
3:   learning devices to ascertain generative models of data
4:   distributions. RBMs are often trained using the Contrastive
5:   Divergence learning algorithm (CD), an approximation to the gradient
6:   of the data log-likelihood. A simple reconstruction error is often
7:   used as a stopping criterion for CD, although several
8:   authors~\cite{schulz-et-al-Convergence-Contrastive-Divergence-2010-NIPSw,
9:     fischer-igel-Divergence-Contrastive-Divergence-2010-ICANN} have
10:   raised doubts concerning the feasibility of this procedure.  In many
11:   cases the evolution curve of the reconstruction error is monotonic
12:   while the log-likelihood is not, thus indicating that the former is
13:   not a good estimator of the optimal stopping point for learning.
14:   However, not many alternatives to the reconstruction error have been
15:   discussed in the literature. In this manuscript we investigate
16:   simple alternatives to the reconstruction error, based on the
17:   inclusion of information contained in neighboring states to the
18:   training set, as a stopping criterion for CD learning.
19: \end{abstract}