A Survey on Cleaning Dirty Data Using Machine Learning Paradigm for Big Data Analytics


Mohd Zebaral Hoque, Jesmeen and Hossen, Md. Jakir and Sayeed, Md. Shohel and Ho, Chin Kuan and K., Tawsif and Rahman, Armanur and Arif, E. M. H. (2018) A Survey on Cleaning Dirty Data Using Machine Learning Paradigm for Big Data Analytics. Indonesian Journal of Electrical Engineering and Computer Science, 10 (3). p. 1234. ISSN 2502-4752

[img] Text
Restricted to Repository staff only

Download (690kB)


Recently Big Data has become one of the important new factors in the business field. This needs to have strategies to manage large volumes of structured, unstructured and semi-structured data. It’s challenging to analyze such large scale of data to extract data meaning and handling uncertain outcomes. Almost all big data sets are dirty, i.e. the set may contain inaccuracies, missing data, miscoding and other issues that influence the strength of big data analytics. One of the biggest challenges in big data analytics is to discover and repair dirty data; failure to do this can lead to inaccurate analytics and unpredictable conclusions. Data cleaning is an essential part of managing and analyzing data. In this survey paper, data quality troubles which may occur in big data processing to understand clearly why an organization requires data cleaning are examined, followed by data quality criteria (dimensions used to indicate data quality). Then, cleaning tools available in market are summarized. Also challenges faced in cleaning big data due to nature of data are discussed. Machine learning algorithms can be used to analyze data and make predictions and finally clean data automatically.

Item Type: Article
Uncontrolled Keywords: Machine learning, Big data, Big data analytics, Data cleaning, Dirty data
Subjects: Q Science > Q Science (General) > Q300-390 Cybernetics
Divisions: Faculty of Engineering and Technology (FET)
Faculty of Computing and Informatics (FCI)
Faculty of Information Science and Technology (FIST)
Depositing User: Ms Rosnani Abd Wahab
Date Deposited: 11 Nov 2020 12:11
Last Modified: 11 Nov 2020 12:11
URII: http://shdl.mmu.edu.my/id/eprint/7336


Downloads per month over past year

View ItemEdit (login required)