The Community for Technology Leaders
RSS Icon
Issue No.07 - July (2013 vol.35)
pp: 1660-1673
Qibin Zhao , Lab. for Adv. Brain Signal Process., RIKEN, Saitama, Japan
C. F. Caiafa , Inst. Argentino de Radioastronom. (IAR), CCT La Plata, Buenos Aires, Argentina
D. P. Mandic , Dept. of Electr. & Electron. Eng., Imperial Coll. London, London, UK
Z. C. Chao , Lab. for Adaptive Intell., RIKEN, Saitama, Japan
Y. Nagasaka , Lab. for Adaptive Intell., RIKEN, Saitama, Japan
N. Fujii , Lab. for Adaptive Intell., RIKEN, Saitama, Japan
Liqing Zhang , Dept. of Comput. Sci. & Eng., Shanghai Jiao Tong Univ., Shanghai, China
A. Cichocki , Lab. for Adv. Brain Signal Process., RIKEN, Saitama, Japan
A new generalized multilinear regression model, termed the higher order partial least squares (HOPLS), is introduced with the aim to predict a tensor (multiway array) Y from a tensor X through projecting the data onto the latent space and performing regression on the corresponding latent variables. HOPLS differs substantially from other regression models in that it explains the data by a sum of orthogonal Tucker tensors, while the number of orthogonal loadings serves as a parameter to control model complexity and prevent overfitting. The low-dimensional latent space is optimized sequentially via a deflation operation, yielding the best joint subspace approximation for both X and Y. Instead of decomposing X and Y individually, higher order singular value decomposition on a newly defined generalized cross-covariance tensor is employed to optimize the orthogonal loadings. A systematic comparison on both synthetic data and real-world decoding of 3D movement trajectories from electrocorticogram signals demonstrate the advantages of HOPLS over the existing methods in terms of better predictive ability, suitability to handle small sample sizes, and robustness to noise.
Tensile stress, Vectors, Approximation methods, Matrix decomposition, Loading, Optimization, Data models,fusion of behavioral and neural data, Multilinear regression, partial least squares, higher order singular value decomposition, constrained block Tucker decomposition, electrocorticogram
Qibin Zhao, C. F. Caiafa, D. P. Mandic, Z. C. Chao, Y. Nagasaka, N. Fujii, Liqing Zhang, A. Cichocki, "Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1660-1673, July 2013, doi:10.1109/TPAMI.2012.254
[1] D.C. Montgomery, E.A. Peck, and G.G. Vining, Introduction to Linear Regression Analysis, third ed. John Wiley & Sons, 2001.
[2] C. Dhanjal, S. Gunn, and J. Shawe-Taylor, "Efficient Sparse Kernel Feature Extraction Based on Partial Least Squares," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 8, pp. 1347-1361, Aug. 2009.
[3] H. Wold, "Soft Modelling by Latent Variables: The Non-Linear Iterative Partial Least Squares (NIPALS) Approach," Perspectives in Probability and Statistics, pp. 117-142, Academic Press, 1975.
[4] A. Krishnan, L. Williams, A. McIntosh, and H. Abdi, "Partial Least Squares (PLS) Methods for Neuroimaging: A Tutorial and Review," NeuroImage, vol. 56, no. 2, pp. 455-475, 2010.
[5] H. Abdi, "Partial Least Squares Regression and Projection on Latent Structure Regression (PLS Regression)," Wiley Interdisciplinary Rev.: Computational Statistics, vol. 2, no. 1, pp. 97-106, 2010.
[6] R. Rosipal and N. Krämer, "Overview and Recent Advances in Partial Least Squares," Proc. Int'l Conf. Subspace, Latent Structure and Feature Selection, pp. 34-51, 2006.
[7] J. Trygg and S. Wold, "Orthogonal Projections to Latent Structures (O-PLS)," J. Chemometrics, vol. 16, no. 3, pp. 119-128, 2002.
[8] R. Ergon, "PLS Score-Loading Correspondence and a Bi-Orthogonal Factorization," J. Chemometrics, vol. 16, no. 7, pp. 368-373, 2002.
[9] S. Vijayakumar and S. Schaal, "Locally Weighted Projection Regression: An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space," Proc. 17th Int'l Conf. Machine Learning, vol. 1, pp. 288-293, 2000.
[10] R. Rosipal and L. Trejo, "Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space," J. Machine Learning Research, vol. 2, pp. 97-123, 2002.
[11] J. Ham, D. Lee, S. Mika, and B. Schölkopf, "A Kernel View of the Dimensionality Reduction of Manifolds," Proc. 21st Int'l Conf. Machine Learning, pp. 47-54, 2004.
[12] R. Bro, A. Rinnan, and N. Faber, "Standard Error of Prediction for Multilinear PLS-2. Practical Implementation in Fluorescence Spectroscopy," Chemometrics and Intelligent Laboratory Systems, vol. 75, no. 1, pp. 69-76, 2005.
[13] B. Li, J. Morris, and E. Martin, "Model Selection for Partial Least Squares Regression," Chemometrics and Intelligent Laboratory Systems, vol. 64, no. 1, pp. 79-89, 2002.
[14] R. Tibshirani, "Regression Shrinkage and Selection via the Lasso," J. Royal Statistical Soc., Series B (Methodological), vol. 58, pp. 267-288, 1996.
[15] R. Bro, "Multi-Way Analysis in the Food Industry: Models, Algorithms, and Applications," Academish Proefschrift, Dinamarca, 1998.
[16] T. Kolda and B. Bader, "Tensor Decompositions and Applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, 2009.
[17] A. Cichocki, R. Zdunek, A.H. Phan, and S.I. Amari, Nonnegative Matrix and Tensor Factorizations. John Wiley & Sons, 2009.
[18] E. Acar, D. Dunlavy, T. Kolda, and M. Mørup, "Scalable Tensor Factorizations for Incomplete Data," Chemometrics and Intelligent Laboratory Systems, vol. 106, pp. 41-56, 2011.
[19] R. Bro, "Multiway Calibration. Multilinear PLS," J. Chemometrics, vol. 10, no. 1, pp. 47-61, 1996.
[20] R. Bro, "Review on Multiway Analysis in Chemistry—2000-2005," Critical Rev. Analytical Chemistry, vol. 36, no. 3, pp. 279-293, 2006.
[21] K. Hasegawa, M. Arakawa, and K. Funatsu, "Rational Choice of Bioactive Conformations through Use of Conformation Analysis and 3-Way Partial Least Squares Modeling," Chemometrics and Intelligent Laboratory Systems, vol. 50, no. 2, pp. 253-261, 2000.
[22] J. Nilsson, S. de Jong, and A. Smilde, "Multiway Calibration in 3D QSAR," J. Chemometrics, vol. 11, no. 6, pp. 511-524, 1997.
[23] K. Zissis, R. Brereton, S. Dunkerley, and R. Escott, "Two-Way, Unfolded Three-Way and Three-Mode Partial Least Squares Calibration of Diode Array HPLC Chromatograms for the Quantitation of Low-Level Pharmaceutical Impurities," Analytica Chimica Acta, vol. 384, no. 1, pp. 71-81, 1999.
[24] E. Martinez-Montes, P. Valdés-Sosa, F. Miwakeichi, R. Goldman, and M. Cohen, "Concurrent EEG/fMRI Analysis by Multiway Partial Least Squares," NeuroImage, vol. 22, no. 3, pp. 1023-1034, 2004.
[25] E. Acar, C. Bingol, H. Bingol, R. Bro, and B. Yener, "Seizure Recognition on Epilepsy Feature Tensor," Proc. 29th Ann. IEEE Int'l Conf. Eng. Medical Biomedical Soc., pp. 4273-4276, 2007.
[26] R. Bro, A. Smilde, and S. de Jong, "On the Difference between Low-Rank and Subspace Approximation: Improved Model for Multi-Linear PLS Regression," Chemometrics and Intelligent Laboratory Systems, vol. 58, no. 1, pp. 3-13, 2001.
[27] A. Smilde and H. Kiers, "Multiway Covariates Regression Models," J. Chemometrics, vol. 13, no. 1, pp. 31-48, 1999.
[28] A. Smilde, R. Bro, and P. Geladi, Multi-Way Analysis with Applications in the Chemical Sciences. Wiley, 2004.
[29] R.A. Harshman, "Foundations of the PARAFAC Procedure: Models and Conditions for an Explanatory Multimodal Factor Analysis," UCLA Working Papers in Phonetics, vol. 16, pp. 1-84, 1970.
[30] A. Smilde, "Comments on Multilinear PLS," J. Chemometrics, vol. 11, no. 5, pp. 367-377, 1997.
[31] M.D. Borraccetti, P.C. Damiani, and A.C. Olivieri, "When Unfolding Is Better: Unique Success of Unfolded Partial Least-Squares Regression with Residual Bilinearization for the Processing of Spectral-pH Data with Strong Spectral Overlapping. Analysis of Fluoroquinolones in Human Urine Based on Flow-Injection pH-Modulated Synchronous Fluorescence Data Matrices," Analyst, vol. 134, pp. 1682-1691, 2009.
[32] L. De Lathauwer, "Decompositions of a Higher-Order Tensor in Block Terms—Part II: Definitions and Uniqueness," SIAM J. Matrix Analysis Applications, vol. 30, no. 3, pp. 1033-1066, 2008.
[33] L. De Lathauwer, B. De Moor, and J. Vandewalle, "A Multilinear Singular Value Decomposition," SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1253-1278, 2000.
[34] L.R. Tucker, "Implications of Factor Analysis of Three-Way Matrices for Measurement of Change," Problems in Measuring Change, C.W. Harris, ed., pp. 122-137, Univ. of Wisconsin Press, 1963.
[35] T. Kolda, "Multilinear Operators for Higher-Order Decompositions," Technical Report SAND2006-2081, Sandia Nat'l Laboratories, Albuquerque, N.M., Livermore, Calif., 2006.
[36] J.D. Carroll and J.J. Chang, "Analysis of Individual Differences in Multidimensional Scaling via an N-Way Generalization of 'Eckart-Young' Decomposition," Psychometrika, vol. 35, pp. 283-319, 1970.
[37] L. De Lathauwer, B. De Moor, and J. Vandewalle, "On the Best Rank-1 and Rank-(R1, R2,..., RN) Approximation of Higher-Order Tensors," SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1324-1342, 2000.
[38] L.-H. Lim and V.D. Silva, "Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem," SIAM J. Matrix Analysis and Applications, vol. 30, no. 3, pp. 1084-1127, 2008.
[39] H. Wold, "Soft Modeling: The Basic Design and Some Extensions," Systems under Indirect Observation, vol. 2, pp. 1-53, 1982.
[40] S. Wold, M. Sjostroma, and L. Erikssonb, "PLS-Regression: A Basic Tool of Chemometrics," Chemometrics and Intelligent Laboratory Systems, vol. 58, pp. 109-130, 2001.
[41] S. Wold, A. Ruhe, H. Wold, and W. DunnIII, "The Collinearity Problem in Linear Regression: The Partial Least Squares (PLS) Approach to Generalized Inverses," SIAM J. Scientific and Statistical Computing, vol. 5, pp. 735-743, 1984.
[42] B. Kowalski, R. Gerlach, and H. Wold, "Systems under Indirect Observation," Chemical Systems under Indirect Observation, pp. 191-209, 1982.
[43] A. McIntosh and N. Lobaugh, "Partial Least Squares Analysis of Neuroimaging Data: Applications and Advances," Neuroimage, vol. 23, pp. S250-S263, 2004.
[44] A. McIntosh, W. Chau, and A. Protzner, "Spatiotemporal Analysis of Event-Related fMRI Data Using Partial Least Squares," Neuroimage, vol. 23, no. 2, pp. 764-775, 2004.
[45] N. Kovacevic and A. McIntosh, "Groupwise Independent Component Decomposition of EEG Data and Partial Least Square Analysis," NeuroImage, vol. 35, no. 3, pp. 1103-1112, 2007.
[46] Z. Chao, Y. Nagasaka, and N. Fujii, "Long-Term Asynchronous Decoding of Arm Motion Using Electrocorticographic Signals in Monkeys," Frontiers in Neuroengineering, vol. 3, no. 3, 2010.
[47] L. Trejo, R. Rosipal, and B. Matthews, "Brain-Computer Interfaces for 1-D and 2-D Cursor Control: Designs Using Volitional Control of the EEG Spectrum or Steady-State Visual Evoked Potentials," IEEE Trans. Neural Systems and Rehabilitation Eng., vol. 14, no. 2, pp. 225-229, June 2006.
[48] H. Kim, J. Zhou, H. MorseIII, and H. Park, "A Three-Stage Framework for Gene Expression Data Analysis by L1-Norm Support Vector Regression," Int'l J. Bioinformatics Research and Applications, vol. 1, no. 1, pp. 51-62, 2005.
[49] Y. Nagasaka, K. Shimoda, and N. Fujii, "Multidimensional Recording (MDR) and Data Sharing: An Ecological Open Research and Educational Platform for Neuroscience," PLoS ONE, vol. 6, no. 7, p. e22561, 2011.
[50] J. Rickert, S. de Oliveira, E. Vaadia, A. Aertsen, S. Rotter, and C. Mehring, "Encoding of Movement Direction in Different Frequency Ranges of Motor Cortical Local Field Potentials," J. Neuroscience, vol. 25, no. 39, pp. 8815-8824, 2005.
22 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool