The Community for Technology Leaders
2015 IEEE 31st International Conference on Data Engineering (ICDE) (2015)
Seoul, South Korea
April 13, 2015 to April 17, 2015
ISBN: 978-1-4799-7964-6
pp: 1047-1058
Inah Jeon , Department of Computer Science, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 305-701, Republic of Korea
Evangelos E. Papalexakis , Computer Science Department, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh PA 15213, United States
U Kang , Department of Computer Science, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 305-701, Republic of Korea
Christos Faloutsos , Computer Science Department, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh PA 15213, United States
ABSTRACT
How can we find useful patterns and anomalies in large scale real-world data with multiple attributes? For example, network intrusion logs, with (source-ip, target-ip, port-number, timestamp)? Tensors are suitable for modeling these multi-dimensional data, and widely used for the analysis of social networks, web data, network traffic, and in many other settings. However, current tensor decomposition methods do not scale for tensors with millions and billions of rows, columns and ‘fibers’, that often appear in real datasets. In this paper, we propose HaTen2, a scalable distributed suite of tensor decomposition algorithms running on the MapReduce platform. By carefully reordering the operations, and exploiting the sparsity of real world tensors, HaTen2 dramatically reduces the intermediate data, and the number of jobs. As a result, using HaTen2, we analyze big real-world tensors that can not be handled by the current state of the art, and discover hidden concepts.
INDEX TERMS
Tensile stress, Matrix decomposition, Algorithm design and analysis, Scalability, Matrix converters, Computer science, Data models
CITATION
Inah Jeon, Evangelos E. Papalexakis, U Kang, Christos Faloutsos, "HaTen2: Billion-scale tensor decompositions", 2015 IEEE 31st International Conference on Data Engineering (ICDE), vol. 00, no. , pp. 1047-1058, 2015, doi:10.1109/ICDE.2015.7113355
81 ms
(Ver 3.3 (11022016))