کد مقاله | کد نشریه | سال انتشار | مقاله انگلیسی | نسخه تمام متن |
---|---|---|---|---|
720700 | 1461232 | 2013 | 12 صفحه PDF | دانلود رایگان |
عنوان انگلیسی مقاله ISI
MapReduce optimization algorithm based on machine learning in heterogeneous cloud environment
دانلود مقاله + سفارش ترجمه
دانلود مقاله ISI انگلیسی
رایگان برای ایرانیان
کلمات کلیدی
موضوعات مرتبط
مهندسی و علوم پایه
سایر رشته های مهندسی
مهندسی برق و الکترونیک
پیش نمایش صفحه اول مقاله

چکیده انگلیسی
We present an approach to optimize the MapReduce architecture, which could make heterogeneous cloud environment more stable and efficient. Fundamentally different from previous methods, our approach introduces the machine learning technique into MapReduce framework, and dynamically improve MapReduce algorithm according to the statistics result of machine learning. There are three main aspects: learning machine performance, reduce task assignment algorithm based on learning result, and speculative execution optimization mechanism. Furthermore, there are two important features in our approach. First, the MapReduce framework can obtain nodes' performance values in the cluster through machine learning module. And machine learning module will daily calibrate nodes' performance values to make an accurate assessment of cluster performance. Second, with the optimization of tasks assignment algorithm, we can maximize the performance of heterogeneous clusters. According to our evaluation result, the cluster performance could have 19% improvement in current heterogeneous cloud environment, and the stability of cluster has greatly enhanced.
ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: The Journal of China Universities of Posts and Telecommunications - Volume 20, Issue 6, December 2013, Pages 77-87, 121
Journal: The Journal of China Universities of Posts and Telecommunications - Volume 20, Issue 6, December 2013, Pages 77-87, 121
نویسندگان
Wen-hui LIN, Zhen-ming LEI, Jun LIU, Jie YANG, Fang LIU, Gang HE, Qin WANG,