This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Scaling Up Spike-and-Slab Models for Unsupervised Feature Learning
Aug. 2013 (vol. 35 no. 8)
pp. 1902-1914
Ian J. Goodfellow, Université de Montréal, Montréal
Aaron Courville, Université de Montréal, Montréal
Yoshua Bengio, Université de Montréal, Montréal
We describe the use of two spike-and-slab models for modeling real-valued data, with an emphasis on their applications to object recognition. The first model, which we call spike-and-slab sparse coding (S3C), is a preexisting model for which we introduce a faster approximate inference algorithm. We introduce a deep variant of S3C, which we call the partially directed deep Boltzmann machine (PD-DBM) and extend our S3C inference algorithm for use on this model. We describe learning procedures for each. We demonstrate that our inference procedure for S3C enables scaling the model to unprecedented large problem sizes, and demonstrate that using S3C as a feature extractor results in very good object recognition performance, particularly when the number of labeled examples is low. We show that the PD-DBM generates better samples than its shallow counterpart, and that unlike DBMs or DBNs, the PD-DBM may be trained successfully without greedy layerwise training.
Index Terms:
Encoding,Feature extraction,Data models,Training,Approximation methods,Vectors,Slabs,computer vision,Neural nets,pattern recognition
Citation:
Ian J. Goodfellow, Aaron Courville, Yoshua Bengio, "Scaling Up Spike-and-Slab Models for Unsupervised Feature Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1902-1914, Aug. 2013, doi:10.1109/TPAMI.2012.273
Usage of this product signifies your acceptance of the Terms of Use.