The Community for Technology Leaders
Green Image
Issue No. 09 - Sept. (2014 vol. 36)
ISSN: 0162-8828
pp: 1874-1887
Aaron Courville , Department of Computer Science and Operations Research, University of Montreal, Montreal, Canada
Guillaume Desjardins , DIRO, University of Montreal, 2920 chemin de la Tour, Montreal, Canada
James Bergstra , Centre for Theoretical Neuroscience , University of Waterloo, Waterloo, Canada
Yoshua Bengio , DIRO, University of Montreal, 2920 chemin de la Tour, Montreal, Canada
ABSTRACT
The spike-and-slab restricted Boltzmann machine (ssRBM) is defined to have both a real-valued “slab” variable and a binary “spike” variable associated with each unit in the hidden layer. The model uses its slab variables to model the conditional covariance of the observation—thought to be important in capturing the statistical properties of natural images. In this paper, we present the canonical ssRBM framework together with some extensions. These extensions highlight the flexibility of the spike-and-slab RBM as a platform for exploring more sophisticated probabilistic models of high dimensional data in general and natural image data in particular. Here, we introduce the subspace-ssRBM focused on the task of learning invariant features. We highlight the behaviour of the ssRBM and its extensions through experiments with the MNIST digit recognition task and the CIFAR-10 object classification task.
INDEX TERMS
Slabs, Data models, Vectors, Feature extraction, Covariance matrices, Training, Standards
CITATION

A. Courville, G. Desjardins, J. Bergstra and Y. Bengio, "The Spike-and-Slab RBM and Extensions to Discrete and Sparse Data Distributions," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 36, no. 9, pp. 1874-1887, 2014.
doi:10.1109/TPAMI.2013.238
311 ms
(Ver 3.3 (11022016))