Citation

Lee, Ting Wei and Ong, Lee Yeng and Leow, Meng Chew (2023) Experimental Study using Unsupervised Anomaly Detection on Server Resources Monitoring. In: 2023 11th International Conference on Information and Communication Technology (ICoICT), 23-24 August 2023, Melaka, Malaysia.

Text 32.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

— Monitoring resources in a server environment is an essential and indispensable process that ensures a decent resource allocation to show better performance. Existing resource monitoring solutions or tools can be used but require additional staff to monitor resource usage. It is a high workload task when there are thousands of servers and each server with thousands of application instances. When the consumption surpasses an unusual limit, it is considered an anomaly record. Unsupervised machine learning algorithms can detect anomalies through clustering. This study developed an experimental analysis using three clustering techniques to assess the performance of clustering for discovering the anomalies pattern in a multivariate time series dataset with 25,918 data points. The procedure to get the optimal numbers of clusters is performed before the experiment, which can be considered between 3 and 4 numbers of clusters. Cluster pattern analysis has been done for selecting the best optimal number of clusters among both selections. The experiment has included two dimensionality reduction techniques for dataset compression to investigate their influences towards clustering performance. Finally, three performance metrics are used for evaluating the cluster quality. This study concluded that the combination of KMeans with Principal Component Analysis produced the best cluster quality among 9 combinations of clustering techniques. Finally, the anomalies behaviors are identified in the distinguishable cluster from all the monitoring items. Nonetheless, it is worth noting that the output of the anomaly detection is different when clustering is performed using different numbers of clusters.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	clustering, dimensionality reduction, performance metrics
Subjects:	H Social Sciences > HA Statistics > HA1-4737 Statistics (General) > HA36-37 Statistical services. Statistical bureaus
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	31 Oct 2023 07:39
Last Modified:	31 Oct 2023 07:39
URII:	http://shdl.mmu.edu.my/id/eprint/11790

Downloads

Edit (login required)