1: \begin{abstract}

2: Revealing hidden features in unlabeled data is called unsupervised feature learning, which plays an important role in

3: pretraining a deep neural network. Here we provide a statistical mechanics analysis of the unsupervised learning in a restricted Boltzmann machine with binary synapses.

4: A message passing equation to infer the hidden feature is derived, and furthermore, variants of this equation are analyzed. 

5: A statistical analysis by replica theory describes the thermodynamic

6: properties of the model. Our analysis confirms an entropy crisis preceding the non-convergence of the

7: message passing equation, suggesting a discontinuous phase transition as a key characteristic of the restricted Boltzmann machine.

8: Continuous phase transition is also confirmed depending on the embedded feature strength in the data. The mean-field

9: result under the replica symmetric assumption agrees with that obtained by running message passing algorithms on single instances of finite

10: sizes. Interestingly, in an approximate Hopfield model, the entropy crisis is absent, and a continuous phase transition is observed instead. We also develop an

11: iterative equation to infer the hyper-parameter (temperature) hidden in the data, which in physics corresponds to iteratively imposing Nishimori condition.

12: Our study provides insights towards understanding the thermodynamic properties of the restricted Boltzmann

13: machine learning, and moreover important theoretical basis to build simplified deep networks.

14: 

15: \end{abstract}

16: