Citation

Mohd Zebaral Hoque, Jesmeen and Hossen, Md. Jakir and Sayeed, Md. Shohel and Ho, Chin Kuan and Tawsif Khan, Chy. Mohammed and Rahman, Armanur and Arif, E. M. H. (2018) A Survey on Cleaning Dirty Data Using Machine Learning Paradigm for Big Data Analytics. Indonesian Journal of Electrical Engineering and Computer Science, 10 (3). p. 1234. ISSN 2502-4752

Text 65.pdf Restricted to Repository staff only Download (690kB)

Abstract

Recently Big Data has become one of the important new factors in the business field. This needs to have strategies to manage large volumes of structured, unstructured and semi-structured data. It’s challenging to analyze such large scale of data to extract data meaning and handling uncertain outcomes. Almost all big data sets are dirty, i.e. the set may contain inaccuracies, missing data, miscoding and other issues that influence the strength of big data analytics. One of the biggest challenges in big data analytics is to discover and repair dirty data; failure to do this can lead to inaccurate analytics and unpredictable conclusions. Data cleaning is an essential part of managing and analyzing data. In this survey paper, data quality troubles which may occur in big data processing to understand clearly why an organization requires data cleaning are examined, followed by data quality criteria (dimensions used to indicate data quality). Then, cleaning tools available in market are summarized. Also challenges faced in cleaning big data due to nature of data are discussed. Machine learning algorithms can be used to analyze data and make predictions and finally clean data automatically.

Item Type:	Article
Uncontrolled Keywords:	Machine learning, Big data, Big data analytics, Data cleaning, Dirty data
Subjects:	Q Science > Q Science (General) > Q300-390 Cybernetics
Divisions:	Faculty of Computing and Informatics (FCI) Faculty of Engineering and Technology (FET) Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Rosnani Abd Wahab
Date Deposited:	11 Nov 2020 12:11
Last Modified:	06 Mar 2023 06:46
URII:	http://shdl.mmu.edu.my/id/eprint/7336

Downloads

Edit (login required)