A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features

As internet encryption has grown to safeguard users’ privacy, malware has evolved to leverage encryption protocols such as Transport Layer Security (TLS) to conceal its hazardous connections. The difficulty and impracticality of decrypting TLS network traffic before it reaches the Intrusion Detectio...

Full description

Saved in:
Bibliographic Details
Main Author: Keshkeh, Kinan
Format: Thesis
Language:English
Published: 2022
Subjects:
Online Access:http://eprints.usm.my/60044/1/24%20Pages%20from%20KINAN%20KESHKEH.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
id my-usm-ep.60044
record_format uketd_dc
spelling my-usm-ep.600442024-03-04T01:20:45Z A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features 2022-11 Keshkeh, Kinan QA75.5-76.95 Electronic computers. Computer science As internet encryption has grown to safeguard users’ privacy, malware has evolved to leverage encryption protocols such as Transport Layer Security (TLS) to conceal its hazardous connections. The difficulty and impracticality of decrypting TLS network traffic before it reaches the Intrusion Detection System (IDS) has driven numerous research studies to focus on anomaly-based malware detection without decryption employing various features and Machine Learning (ML) algorithms. Nonetheless, several of these studies used flow features with low feature importance value and poor capability to distinguish malicious flows, such as the number of packets sent and received in a flow or its duration. Furthermore, the outliers and frequency-based flow feature transformations (FTT) applied to mitigate the poor flow feature have several flaws. This thesis proposes a TLS-based malware detection (TLSMalDetect) approach based on ML classification to address flow feature utilization limitations in related work. TLSMalDetect includes periodicity-independent entropy-based flow set (EFS) features produced by an FFT technique. The efficiency of EFS features is assessed in two ways: (1) by comparing them to the relevant related work’s features of outliers and flow using four feature importance methods, and (2) by analyzing the classification performance in the scenarios with and without EFS features. This study also investigates TLSMalDetect detection performance using seven ML classification algorithms and identifies the one with the highest accuracy. 2022-11 Thesis http://eprints.usm.my/60044/ http://eprints.usm.my/60044/1/24%20Pages%20from%20KINAN%20KESHKEH.pdf application/pdf en public masters Perpustakaan Hamzah Sendut Pusat Pengajian Sains Komputer
institution Universiti Sains Malaysia
collection USM Institutional Repository
language English
topic QA75.5-76.95 Electronic computers
Computer science
spellingShingle QA75.5-76.95 Electronic computers
Computer science
Keshkeh, Kinan
A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
description As internet encryption has grown to safeguard users’ privacy, malware has evolved to leverage encryption protocols such as Transport Layer Security (TLS) to conceal its hazardous connections. The difficulty and impracticality of decrypting TLS network traffic before it reaches the Intrusion Detection System (IDS) has driven numerous research studies to focus on anomaly-based malware detection without decryption employing various features and Machine Learning (ML) algorithms. Nonetheless, several of these studies used flow features with low feature importance value and poor capability to distinguish malicious flows, such as the number of packets sent and received in a flow or its duration. Furthermore, the outliers and frequency-based flow feature transformations (FTT) applied to mitigate the poor flow feature have several flaws. This thesis proposes a TLS-based malware detection (TLSMalDetect) approach based on ML classification to address flow feature utilization limitations in related work. TLSMalDetect includes periodicity-independent entropy-based flow set (EFS) features produced by an FFT technique. The efficiency of EFS features is assessed in two ways: (1) by comparing them to the relevant related work’s features of outliers and flow using four feature importance methods, and (2) by analyzing the classification performance in the scenarios with and without EFS features. This study also investigates TLSMalDetect detection performance using seven ML classification algorithms and identifies the one with the highest accuracy.
format Thesis
qualification_level Master's degree
author Keshkeh, Kinan
author_facet Keshkeh, Kinan
author_sort Keshkeh, Kinan
title A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
title_short A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
title_full A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
title_fullStr A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
title_full_unstemmed A Machine Learning Classification Approach To Detect Tls-Based Malware Using Entropy-Based Flow Set Features
title_sort machine learning classification approach to detect tls-based malware using entropy-based flow set features
granting_institution Perpustakaan Hamzah Sendut
granting_department Pusat Pengajian Sains Komputer
publishDate 2022
url http://eprints.usm.my/60044/1/24%20Pages%20from%20KINAN%20KESHKEH.pdf
_version_ 1794024081694130176