Computer and Information Technology, International Conference on (2010)
Bradford, West Yorkshire, UK
June 29, 2010 to July 1, 2010
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/CIT.2010.71
Data indexing is common in data mining when working with high-dimensional, large-scale data sets. Hadoop, a cloud computing project using the MapReduce framework in Java, has become of significant interest in distributed data mining. A feasible distributed data indexing algorithm is proposed for Hadoop data mining, based on ZSCORE binning and inverted indexing and on the Hadoop SequenceFile format. A data mining framework on Hadoop using the Java Persistence API (JPA) and MySQL Cluster is proposed. The framework is elaborated in the implementation of a decision tree algorithm on Hadoop. We compare the data index-ing algorithm with Hadoop MapFile indexing, which performs a binary search, in a modest cloud environment. The results show the algorithm is more efficient than naïve MapFile indexing. We compare the JDBC and JPA implementations of the data mining framework. The performance shows the framework is efficient for data mining on Hadoop.
Data Mining, Distributed applications, JPA, ORM, Distributed file systems, Cloud computing
Shi ZhongZhi, Yang Lai, "An Efficient Data Mining Framework on Hadoop using Java Persistence API", Computer and Information Technology, International Conference on, vol. 00, no. , pp. 203-209, 2010, doi:10.1109/CIT.2010.71