|
| This Article | ||
| ||
| Share | ||
| Bibliographic References | ||
| Add to: | ||
| | ||
| Search | ||
| ||
2011 IEEE Third International Conference on Cloud Computing Technology and Science
Optimizing Multiple Machine Learning Jobs on MapReduce
Athens, Greece
November 29-December 01
ISBN: 978-0-7695-4622-3
| ASCII Text | x | ||
| Hiroshi Tamano, Shinji Nakadai, Takuya Araki, "Optimizing Multiple Machine Learning Jobs on MapReduce," 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings, pp. 59-66, 2011 IEEE Third International Conference on Cloud Computing Technology and Science, 2011. | |||
| BibTex | x | ||
| @article{ 10.1109/CloudCom.2011.18, author = {Hiroshi Tamano and Shinji Nakadai and Takuya Araki}, title = {Optimizing Multiple Machine Learning Jobs on MapReduce}, journal ={4th IEEE International Conference on Cloud Computing Technology and Science Proceedings}, volume = {0}, year = {2011}, isbn = {978-0-7695-4622-3}, pages = {59-66}, doi = {http://doi.ieeecomputersociety.org/10.1109/CloudCom.2011.18}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, } | |||
| RefWorks Procite/RefMan/Endnote | x | ||
| TY - CONF JO - 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings TI - Optimizing Multiple Machine Learning Jobs on MapReduce SN - 978-0-7695-4622-3 SP59 EP66 A1 - Hiroshi Tamano, A1 - Shinji Nakadai, A1 - Takuya Araki, PY - 2011 KW - MapReduce KW - Machine Learning KW - Job Scheduling VL - 0 JA - 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings ER - | |||
Recently, MapReduce has been used to parallelize machine learning algorithms. To obtain the best performance for these algorithms, tuning the parameters of the algorithms is required. However, this is time consuming because it requires executing a MapReduce program multiple times using various parameters. Such multiple executions can be assigned to a cluster in various ways, and the execution time varies depending on the assignments. To achieve the shortest execution time, we propose a method for optimizing the assignment of MapReduce jobs to a cluster assuming machine learning targeted runtime. We developed an execution cost model to predict the total execution time of jobs and obtained the optimal assignment by minimizing the cost model. To evaluate the proposed method, we implemented an experimental MapReduce runtime based on Message Passing Interface and executed logistic regression in four cases. The results showed that the proposed method can correctly predict the optimal job assignment. We also confirmed that the optimal assignment reduced execution time by a maximum 77% compared to the worst assignment.
Index Terms:
MapReduce, Machine Learning, Job Scheduling
Citation:
Hiroshi Tamano, Shinji Nakadai, Takuya Araki, "Optimizing Multiple Machine Learning Jobs on MapReduce," cloudcom, pp.59-66, 2011 IEEE Third International Conference on Cloud Computing Technology and Science, 2011
Usage of this product signifies your acceptance of the Terms of Use.
