Gaussian-iki kutuplu kısıtlı Boltzmann makinelerinin betimleyici verimliliğinde saklı yanlılıkların ve saklı entropinin önemi üzerine

With development of machine learning and deep learning fields, the importance of unsupervised learning algorithms also increases. One of these algorithms is Gaussian-Bernoulli Restricted Boltzmann Machines (GBLRBMs), which are capable of modelling real-valued data. Moreover, GBLRBMs are used to pretrain weights in artificial neural networks, which improves performance of these networks. In this work, we analyze the role of hidden bias in representational efficiency of the Gaussian-Bipolar Restricted Boltzmann Machines (GBPRBMs), which are similar to the widely used Gaussian-Bernoulli RBMs. Our experiments show that hidden bias plays an important role in shaping of the probability density function of the visible units. Correspondingly, we define hidden entropy and propose it as a measure of representational efficiency of the model. By using this measure, we investigate the effect of hidden bias on the hidden entropy and provide a full analysis of the hidden entropy as function of the hidden bias for small models with up to three hidden units. We also provide an insight into understanding of the representational efficiency of the larger scale models. Furthermore, we introduce Normalized Empirical Hidden Entropy (NEHE) as an alternative to hidden entropy that can be computed for large models. Experiments on the MNIST, CIFAR-10 and Faces data sets show that NEHE can serve as measure of representational efficiency and gives an insight on minimum number of hidden units required to represent the data.