Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients

Data classification is one of the important aspects in the real-world decision- making support function which can be severely affected by an imbalance class distribution in the training data especially in the medical field. In medical datasets, the data are mainly had imbalanced datasets problem whi...

Full description

Saved in:

Bibliographic Details
Main Author:	Dzulkefli, Syasya Farina
Format:	Thesis
Language:	English
Published:	2022
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/102725/1/SyasyaFarinaDzulkefliMSKE2022.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.102725
record_format	uketd_dc
spelling	my-utm-ep.1027252023-09-20T03:24:37Z Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients 2022 Dzulkefli, Syasya Farina TK Electrical engineering. Electronics Nuclear engineering Data classification is one of the important aspects in the real-world decision- making support function which can be severely affected by an imbalance class distribution in the training data especially in the medical field. In medical datasets, the data are mainly had imbalanced datasets problem which composed of minority of normal samples and majority of abnormal samples. As for today's example, the outbreak of novel coronavirus disease or also called as COVID-19 in late 2019 is still on-going which we can see new variants have been discovered from time to time and this can lead to increasing of number of cases around the world. The medical staffs can detect the patients by checking on the symptoms but one of the common COVID-19 symptoms that will be investigating in this research is pneumonia. It is important to detect the pneumonia faster at early stage to avoid it become more severe. Thus, Chest Xray scan images can be considered as one of the confirmatory approaches as they are fast to obtain and easily accessible. Diagnosing diseases in general is a considerable application of data analysis for medical science. In this research, data sampling methods will be explored and implemented for pneumonia detection for imbalanced datasets. The imbalanced datasets of pneumonia X-Ray images from Kaggle dataset will be obtained and different existing data sampling methods also new proposed methods that are achieved by combining or modifying exiting methods will be implemented to balance the images between majority and minority classes of the datasets. After achieved a balanced dataset, CNN model will be implemented to set benchmark of detection accuracy in terms of confusion matrix, precision, accuracy, F1-score and recall for each method and those results will be compared to choose which method will give the highest accuracy in detecting pneumonia. The best undersampling method is near miss with 85.47% accuracy, the best oversampling method is data augmentation with 88.78% accuracy and the best combination method is SMOTE + Tomek with 83.20% accuracy compared to 79% of accuracy when there is no method being implemented on the imbalanced dataset. Implementing data sampling methods will boost the performance of data classification in all applications especially in detecting pneumonia in COVID-19 patients. 2022 Thesis http://eprints.utm.my/102725/ http://eprints.utm.my/102725/1/SyasyaFarinaDzulkefliMSKE2022.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:149741 masters Universiti Teknologi Malaysia Faculty of Engineering - School of Electrical Engineering
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Dzulkefli, Syasya Farina Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
description	Data classification is one of the important aspects in the real-world decision- making support function which can be severely affected by an imbalance class distribution in the training data especially in the medical field. In medical datasets, the data are mainly had imbalanced datasets problem which composed of minority of normal samples and majority of abnormal samples. As for today's example, the outbreak of novel coronavirus disease or also called as COVID-19 in late 2019 is still on-going which we can see new variants have been discovered from time to time and this can lead to increasing of number of cases around the world. The medical staffs can detect the patients by checking on the symptoms but one of the common COVID-19 symptoms that will be investigating in this research is pneumonia. It is important to detect the pneumonia faster at early stage to avoid it become more severe. Thus, Chest Xray scan images can be considered as one of the confirmatory approaches as they are fast to obtain and easily accessible. Diagnosing diseases in general is a considerable application of data analysis for medical science. In this research, data sampling methods will be explored and implemented for pneumonia detection for imbalanced datasets. The imbalanced datasets of pneumonia X-Ray images from Kaggle dataset will be obtained and different existing data sampling methods also new proposed methods that are achieved by combining or modifying exiting methods will be implemented to balance the images between majority and minority classes of the datasets. After achieved a balanced dataset, CNN model will be implemented to set benchmark of detection accuracy in terms of confusion matrix, precision, accuracy, F1-score and recall for each method and those results will be compared to choose which method will give the highest accuracy in detecting pneumonia. The best undersampling method is near miss with 85.47% accuracy, the best oversampling method is data augmentation with 88.78% accuracy and the best combination method is SMOTE + Tomek with 83.20% accuracy compared to 79% of accuracy when there is no method being implemented on the imbalanced dataset. Implementing data sampling methods will boost the performance of data classification in all applications especially in detecting pneumonia in COVID-19 patients.
format	Thesis
qualification_level	Master's degree
author	Dzulkefli, Syasya Farina
author_facet	Dzulkefli, Syasya Farina
author_sort	Dzulkefli, Syasya Farina
title	Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
title_short	Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
title_full	Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
title_fullStr	Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
title_full_unstemmed	Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
title_sort	data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients
granting_institution	Universiti Teknologi Malaysia
granting_department	Faculty of Engineering - School of Electrical Engineering
publishDate	2022
url	http://eprints.utm.my/102725/1/SyasyaFarinaDzulkefliMSKE2022.pdf
_version_	1783729212805349376

Data sampling methods on imbalanced datasets for pneumonia detection in covid-19 patients

Similar Items