کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
720700 1461232 2013 12 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
MapReduce optimization algorithm based on machine learning in heterogeneous cloud environment
موضوعات مرتبط
مهندسی و علوم پایه سایر رشته های مهندسی مهندسی برق و الکترونیک
پیش نمایش صفحه اول مقاله
MapReduce optimization algorithm based on machine learning in heterogeneous cloud environment
چکیده انگلیسی
We present an approach to optimize the MapReduce architecture, which could make heterogeneous cloud environment more stable and efficient. Fundamentally different from previous methods, our approach introduces the machine learning technique into MapReduce framework, and dynamically improve MapReduce algorithm according to the statistics result of machine learning. There are three main aspects: learning machine performance, reduce task assignment algorithm based on learning result, and speculative execution optimization mechanism. Furthermore, there are two important features in our approach. First, the MapReduce framework can obtain nodes' performance values in the cluster through machine learning module. And machine learning module will daily calibrate nodes' performance values to make an accurate assessment of cluster performance. Second, with the optimization of tasks assignment algorithm, we can maximize the performance of heterogeneous clusters. According to our evaluation result, the cluster performance could have 19% improvement in current heterogeneous cloud environment, and the stability of cluster has greatly enhanced.
ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: The Journal of China Universities of Posts and Telecommunications - Volume 20, Issue 6, December 2013, Pages 77-87, 121
نویسندگان
, , , , , , ,