Citation

Abdul Razak, Siti Fatimah and Azman, Afizan and Ahmad, Nazrul Muhaimin and Mhlanga, Imran Artwel J. (2019) Performance Analysis of the Effect of a Combiner on a MapReduce Job. In: 16th IEEE Student Conference on Research and Development, SCOReD 2018, 26-28 November 2018, Selangor, Malaysia.

Text 148.pdf - Published Version Restricted to Repository staff only Download (457kB)

Abstract

MapReduce has been widely deployed as the most efficient framework for big data processing due to its ability to run on commodity hardware as well as the ability to automatically and effectively manage parallel execution of tasks. During the shuffle phase, a lot of data traffic is generated which consumes a lot of bandwidth and in turn, leads to performance degradation. Many efforts have been made to reduce the data traffic during the shuffle phase, with the common one being the use of a combiner function which is default in the Hadoop framework. This paper presents a performance analysis of the effect of a combiner function on the reduce times and reduce shuffle bytes while varying the number of reduce tasks. The results of the analysis show that the combiner significantly reduces the reduce times as well as the reduce shuffle bytes.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Performance analysis
Subjects:	H Social Sciences > HF Commerce > HF5001-6182 Business > HF5549-5549.5 Personnel management. Employment management
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	26 Jan 2022 03:06
Last Modified:	26 Jan 2022 03:15
URII:	http://shdl.mmu.edu.my/id/eprint/9021

Downloads

Edit (login required)