Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases

The maximum likelihood (ML) test in the structural covariance analysis is an effective tool in statistical analysis of multivariate test. However, the performance of the classical location and scatter estimators is usually flawed by singularity and outliers’ problems in high dimensional data sets....

Full description

Saved in:

Bibliographic Details
Main Author:	Hafeez, Ahmad
Format:	Thesis
Language:	eng eng eng
Published:	2021
Subjects:	QA Mathematics
Online Access:	https://etd.uum.edu.my/9512/1/depositpermission-not%20allow_s901078.pdf https://etd.uum.edu.my/9512/2/s901078_01.pdf https://etd.uum.edu.my/9512/3/s901078_02.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-uum-etd.9512
record_format	uketd_dc
spelling	my-uum-etd.95122022-06-26T01:29:04Z Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases 2021 Hafeez, Ahmad Mat Kasim, Maznah Zulkifli, Malina Awang Had Salleh Graduate School of Arts & Sciences Awang Had Salleh Graduate School of Arts & Sciences QA Mathematics The maximum likelihood (ML) test in the structural covariance analysis is an effective tool in statistical analysis of multivariate test. However, the performance of the classical location and scatter estimators is usually flawed by singularity and outliers’ problems in high dimensional data sets. The study aimed to modify the existing ML test by incorporating the Cholesky banded regularization methods and thresholding sample covariance matrix to resolve singularity problems and incorporating the L₁-median with Weiszfeld’s algorithm (MLw) covariance matrix to solve outliers’ problem in high dimensional data sets. This study suggested to replace the classical estimators with MLw estimator due to its good properties. On the other hand, several shortcomings such as inconsistency under normal distribution, based on small sample size with large variables in high dimensions were discovered. To improve the ML estimators and to maintain the high breakdown point while having high dimensional data sets, a new robust estimator of banded Cholesky and L₁-median with Weiszfeld algorithm, MLѡвсн was suggested. The performance of MLѡвсн and four more modified ML-tests, namely MLtests with banded Cholesky estimator MLвсн with Thresholding MLтн with Weiszfeld Algorithm MLѡ and with Weiszfeld’s Algorithm Estimator and Thresholding MLѡтн had been examined using Type I error and power of test values in a simulation study. The MLвсн outperformed other modified ML-tests and presents a robust estimator with high breakdown points and affine equivariance with better computational efficiency in resolving the singularity and outlier problems. The validation of this robust modified ML-test was conducted in a real application on the Pakistan microeconomic system and turned out with good performance. The study contributes to improve the performance of the sample covariance matrices in relation to singularity and outlier problem in high dimensional data cases. 2021 Thesis https://etd.uum.edu.my/9512/ https://etd.uum.edu.my/9512/1/depositpermission-not%20allow_s901078.pdf text eng staffonly https://etd.uum.edu.my/9512/2/s901078_01.pdf text eng staffonly https://etd.uum.edu.my/9512/3/s901078_02.pdf text eng staffonly other doctoral Universiti Utara Malaysia
institution	Universiti Utara Malaysia
collection	UUM ETD
language	eng eng eng
advisor	Mat Kasim, Maznah Zulkifli, Malina
topic	QA Mathematics
spellingShingle	QA Mathematics Hafeez, Ahmad Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
description	The maximum likelihood (ML) test in the structural covariance analysis is an effective tool in statistical analysis of multivariate test. However, the performance of the classical location and scatter estimators is usually flawed by singularity and outliers’ problems in high dimensional data sets. The study aimed to modify the existing ML test by incorporating the Cholesky banded regularization methods and thresholding sample covariance matrix to resolve singularity problems and incorporating the L₁-median with Weiszfeld’s algorithm (MLw) covariance matrix to solve outliers’ problem in high dimensional data sets. This study suggested to replace the classical estimators with MLw estimator due to its good properties. On the other hand, several shortcomings such as inconsistency under normal distribution, based on small sample size with large variables in high dimensions were discovered. To improve the ML estimators and to maintain the high breakdown point while having high dimensional data sets, a new robust estimator of banded Cholesky and L₁-median with Weiszfeld algorithm, MLѡвсн was suggested. The performance of MLѡвсн and four more modified ML-tests, namely MLtests with banded Cholesky estimator MLвсн with Thresholding MLтн with Weiszfeld Algorithm MLѡ and with Weiszfeld’s Algorithm Estimator and Thresholding MLѡтн had been examined using Type I error and power of test values in a simulation study. The MLвсн outperformed other modified ML-tests and presents a robust estimator with high breakdown points and affine equivariance with better computational efficiency in resolving the singularity and outlier problems. The validation of this robust modified ML-test was conducted in a real application on the Pakistan microeconomic system and turned out with good performance. The study contributes to improve the performance of the sample covariance matrices in relation to singularity and outlier problem in high dimensional data cases.
format	Thesis
qualification_name	other
qualification_level	Doctorate
author	Hafeez, Ahmad
author_facet	Hafeez, Ahmad
author_sort	Hafeez, Ahmad
title	Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
title_short	Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
title_full	Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
title_fullStr	Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
title_full_unstemmed	Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
title_sort	modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases
granting_institution	Universiti Utara Malaysia
granting_department	Awang Had Salleh Graduate School of Arts & Sciences
publishDate	2021
url	https://etd.uum.edu.my/9512/1/depositpermission-not%20allow_s901078.pdf https://etd.uum.edu.my/9512/2/s901078_01.pdf https://etd.uum.edu.my/9512/3/s901078_02.pdf
_version_	1747828612307877888

Modifying maximum likelihood test for solving singularity and outlier problems in high dimensional cases

Similar Items