Issue No. 06 - June (2012 vol. 34)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2012.57
Chao Xu , Key Lab. of Machine Perception (Minist. of Educ.), Peking Univ., Beijing, China
Dacheng Tao , Centre for Quantum Comput. & Intell. Syst., Univ. of Technol., Sydney, NSW, Australia
Bo Geng , Key Lab. of Machine Perception (Minist. of Educ.), Peking Univ., Beijing, China
Linjun Yang , Microsoft Res. Asia, Beijing, China
Xian-Sheng Hua , Microsoft, Redmond, WA, USA
We propose an automatic approximation of the intrinsic manifold for general semi-supervised learning (SSL) problems. Unfortunately, it is not trivial to define an optimization function to obtain optimal hyperparameters. Usually, cross validation is applied, but it does not necessarily scale up. Other problems derive from the suboptimality incurred by discrete grid search and the overfitting. Therefore, we develop an ensemble manifold regularization (EMR) framework to approximate the intrinsic manifold by combining several initial guesses. Algorithmically, we designed EMR carefully so it 1) learns both the composite manifold and the semi-supervised learner jointly, 2) is fully automatic for learning the intrinsic manifold hyperparameters implicitly, 3) is conditionally optimal for intrinsic manifold approximation under a mild and reasonable assumption, and 4) is scalable for a large number of candidate manifold hyperparameters, from both time and space perspectives. Furthermore, we prove the convergence property of EMR to the deterministic matrix at rate root-n. Extensive experiments over both synthetic and real data sets demonstrate the effectiveness of the proposed framework.
matrix algebra, approximation theory, learning (artificial intelligence), deterministic matrix, ensemble manifold regularization framework, intrinsic manifold automatic approximation, general semisupervised learning problems, optimization function, optimal hyperparameters, cross validation, discrete grid search, composite manifold learning, candidate manifold hyperparameters, EMR convergence property, Manifolds, Laplace equations, Approximation methods, Kernel, Algorithm design and analysis, Support vector machines, Loss measurement, ensemble manifold regularization., Manifold learning, semi-supervised learning
Chao Xu, Dacheng Tao, Bo Geng, Linjun Yang and Xian-Sheng Hua, "Ensemble Manifold Regularization," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 34, no. , pp. 1227-1233, 2012.