کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
6935098 1449558 2018 15 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
Comparing load-balancing algorithms for MapReduce under Zipfian data skews
موضوعات مرتبط
مهندسی و علوم پایه مهندسی کامپیوتر نرم افزارهای علوم کامپیوتر
پیش نمایش صفحه اول مقاله
Comparing load-balancing algorithms for MapReduce under Zipfian data skews
چکیده انگلیسی
In this paper, we analyze applicability of various load-balancing methods in countering data skew in MapReduce computations. A MapReduce job consists of several phases: mapping, shuffling data, sorting and reducing. The distribution of the work in the last three phases is data-driven, and unequal distribution of the data keys may cause imbalance in the computation completion times and prolonged execution of the whole job. We propose algorithms of four different types for balancing computational effort in reduce-heavy MapReduce jobs and evaluate their performance under various degrees of data skew and system parameters. By applying an innovative method of visualizing algorithm dominance conditions, we are able to determine conditions under which certain load-balancing algorithms are capable of scheduling MapReduce computations well. We conclude that no single algorithm is a panacea and hybrid approaches are necessary.
ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Parallel Computing - Volume 72, February 2018, Pages 14-28
نویسندگان
, ,