Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition

Human Action Recognition (HAR) is critical in video monitoring, human-computer interaction, video comprehension, and virtual reality. While significant progress has been made in the HAR domain in recent years, developing an accurate, fast, and efficient system for video action recognition remains a...

Full description

Saved in:

Bibliographic Details
Main Author:	Nasrul ‘Alam, Fakhrul Aniq Hakimi
Format:	Thesis
Language:	English
Published:	2022
Subjects:	T Technology (General)
Online Access:	http://eprints.utm.my/id/eprint/99599/1/FakhrulAniqHakimiMMJIIT2022.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.99599
record_format	uketd_dc
spelling	my-utm-ep.995992023-03-05T08:24:04Z Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition 2022 Nasrul ‘Alam, Fakhrul Aniq Hakimi T Technology (General) Human Action Recognition (HAR) is critical in video monitoring, human-computer interaction, video comprehension, and virtual reality. While significant progress has been made in the HAR domain in recent years, developing an accurate, fast, and efficient system for video action recognition remains a challenge due to a variety of obstacles, such as changes in camera viewpoint, occlusions, background, and motion speed. In general, the action recognition model learns spatial and temporal features in order to classify human actions. The state-of-the-art approaches to deep learning skeleton-based action recognition rely primarily on Recurrent Neural Networks (RNN) or Convolutional Neural Networks (CNN). RNN-based action recognition methods only model the long-term contextual information in the temporal domain. In return, they neglect the spatial configurations of articulated skeletons where the joints are strongly discriminative. Therefore, it is challenging to extract high-level features. In contrast, action recognition based on CNNs is incapable of modelling long-term temporal dependency. Typically, implementations stack a limited number of frames and convert them into images to represent spatio-temporal information. However, this approach is susceptible to information loss during the conversion process. This study proposes STEM-Coords as pre-processing and features extraction technique, to effectively represent spatio-temporal features using joint coordinates from a human pose. The feature set comprised normalized joint coordinates and their respective speed was represented tabularly as input for the Neural Oblivious Decision Ensemble (NODE) classification model. The proposed STEM-Coords was validated on three benchmark datasets KTH, RealWorld HAR, and MSR DailyActivity 3D. Our method outperformed the state-of-the-art approaches on every dataset with 97.3%, 99.3%, and 97.4% accuracy rates, respectively. The results demonstrated that our proposed method effectively and efficiently represents spatio-temporal information while maintaining robustness to partial occlusion, anthropometrically, and view-invariant. 2022 Thesis http://eprints.utm.my/id/eprint/99599/ http://eprints.utm.my/id/eprint/99599/1/FakhrulAniqHakimiMMJIIT2022.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:150862 masters Universiti Teknologi Malaysia Malaysia-Japan International Institute of Technology (MJIIT)
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	T Technology (General)
spellingShingle	T Technology (General) Nasrul ‘Alam, Fakhrul Aniq Hakimi Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
description	Human Action Recognition (HAR) is critical in video monitoring, human-computer interaction, video comprehension, and virtual reality. While significant progress has been made in the HAR domain in recent years, developing an accurate, fast, and efficient system for video action recognition remains a challenge due to a variety of obstacles, such as changes in camera viewpoint, occlusions, background, and motion speed. In general, the action recognition model learns spatial and temporal features in order to classify human actions. The state-of-the-art approaches to deep learning skeleton-based action recognition rely primarily on Recurrent Neural Networks (RNN) or Convolutional Neural Networks (CNN). RNN-based action recognition methods only model the long-term contextual information in the temporal domain. In return, they neglect the spatial configurations of articulated skeletons where the joints are strongly discriminative. Therefore, it is challenging to extract high-level features. In contrast, action recognition based on CNNs is incapable of modelling long-term temporal dependency. Typically, implementations stack a limited number of frames and convert them into images to represent spatio-temporal information. However, this approach is susceptible to information loss during the conversion process. This study proposes STEM-Coords as pre-processing and features extraction technique, to effectively represent spatio-temporal features using joint coordinates from a human pose. The feature set comprised normalized joint coordinates and their respective speed was represented tabularly as input for the Neural Oblivious Decision Ensemble (NODE) classification model. The proposed STEM-Coords was validated on three benchmark datasets KTH, RealWorld HAR, and MSR DailyActivity 3D. Our method outperformed the state-of-the-art approaches on every dataset with 97.3%, 99.3%, and 97.4% accuracy rates, respectively. The results demonstrated that our proposed method effectively and efficiently represents spatio-temporal information while maintaining robustness to partial occlusion, anthropometrically, and view-invariant.
format	Thesis
qualification_level	Master's degree
author	Nasrul ‘Alam, Fakhrul Aniq Hakimi
author_facet	Nasrul ‘Alam, Fakhrul Aniq Hakimi
author_sort	Nasrul ‘Alam, Fakhrul Aniq Hakimi
title	Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
title_short	Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
title_full	Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
title_fullStr	Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
title_full_unstemmed	Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
title_sort	spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition
granting_institution	Universiti Teknologi Malaysia
granting_department	Malaysia-Japan International Institute of Technology (MJIIT)
publishDate	2022
url	http://eprints.utm.my/id/eprint/99599/1/FakhrulAniqHakimiMMJIIT2022.pdf
_version_	1776100624153182208

Spatio-temporal normalized joint coordinates as features for skeleton-based human action recognition

Similar Items