The Community for Technology Leaders
2007 IEEE Conference on Computer Vision and Pattern Recognition (2007)
Minneapolis, MN, USA
June 17, 2007 to June 22, 2007
ISBN: 1-4244-1179-3
pp: 1-8
Marc'Aurelio Ranzato , Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. ranzato@cs.nyu.e
Fu Jie Huang , Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. jhuangfu@cs.nyu.
Y-Lan Boureau , Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. ylan@cs.nyu.edu
Yann LeCun , Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. yann@cs.nyu.edu
ABSTRACT
We present an unsupervised method for learning a hierarchy of sparse feature detectors that are invariant to small shifts and distortions. The resulting feature extractor consists of multiple convolution filters, followed by a feature-pooling layer that computes the max of each filter output within adjacent windows, and a point-wise sigmoid non-linearity. A second level of larger and more invariant features is obtained by training the same algorithm on patches of features from the first level. Training a supervised classifier on these features yields 0.64% error on MNIST, and 54% average recognition rate on Caltech 101 with 30 training samples per category. While the resulting architecture is similar to convolutional networks, the layer-wise unsupervised training procedure alleviates the over-parameterization problems that plague purely supervised learning procedures, and yields good performance with very few labeled training samples.
INDEX TERMS
null
CITATION

Y. Boureau, M. Ranzato, Y. LeCun and F. J. Huang, "Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition," 2007 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Minneapolis, MN, USA, 2007, pp. 1-8.
doi:10.1109/CVPR.2007.383157
92 ms
(Ver 3.3 (11022016))