A study of a vision-based lip movement analysis for hearing-impaired person

The study of vision based lip movement is the interpretation of human lip movement while speaking. Deaf and hard of hearing people often have a problem being able to understand what being talking in the conversation. Sign language may be useful for them to communicate but not everyone may underst...

Full description

Saved in:

Bibliographic Details
Main Author:	Muhamad Kamil Syahid, Talha
Format:	Thesis
Language:	English
Subjects:	Lip movement Lip movement > Analysis Hearing-impared person Vision based lip movement Lip reading
Online Access:	http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/1/Page%201-24.pdf http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/2/Full%20text.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-unimap-61991
record_format	uketd_dc
spelling	my-unimap-619912019-09-25T04:24:57Z A study of a vision-based lip movement analysis for hearing-impaired person Muhamad Kamil Syahid, Talha Dr. Wan Khairunizam Wan Ahmad The study of vision based lip movement is the interpretation of human lip movement while speaking. Deaf and hard of hearing people often have a problem being able to understand what being talking in the conversation. Sign language may be useful for them to communicate but not everyone may understand the sign language. To make the communication more interesting and does not have any obstacle to talk to anybody, the lip reading is the best way to communicate. Deaf and hard hearing person can talk like the normal person, however, they do not hear the word spoken by themselves. On the other hand, there is no sound feedback system for them. This is a reason they could not speak with the correct pronunciations. To teach them pronounce the word correctly, fluently and to understand others in the conversation, the lip reading systems that could train them pronounce the word correctly needs to be developed. Previous researcher proposed many methods to recognize the spoken word based on the lip movement. One of the method is attach the colour marker onto the lip surface and a vision sensor is used to track the moving marker. Other researchers extract the lip region by using Active contour Model (ACM) or a snake method to draw the keypoint around lip edges. In this studies, a system to track and recognize the spoken word based on lip movements is proposed. A camera is used as a vision sensor and an image processing technique is employed to extract and track the lip. The lip horizontal and vertical distances of the lip are used to measure the ellipse while the systems track the lip movements. The trajectories of the ellipse are resampled to 10 point called the features point. The word database contains 10 spoken word frequently speak at the hospital are designed based on the distribution of the 10 feature points. Two types of word database have been designed which are the individual word database and the universal word database. The individual word database is defined as the distributed feature points of each subject by speaks the words with repetition. Meanwhile, the universal word database is defined as the distributed feature point of all word regardless who is the subjects. The recognition experiments are conducted and the system recognizes the unknown words with the recognition rate 92.47% accuracy by using the individual database. Meanwhile, the system recognizes the unknown words with the recognition rate 90.39% accuracy by using the universal database. Universiti Malaysia Perlis (UniMAP) 2014 Thesis en http://dspace.unimap.edu.my:80/xmlui/handle/123456789/61991 http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/1/Page%201-24.pdf feee5b6a116a5fa7a63502b88a535e1a http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/2/Full%20text.pdf 29fda1d01fabb793f34d905a8281bf17 http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/3/license.txt 8a4605be74aa9ea9d79846c1fba20a33 Lip movement Lip movement -- Analysis Hearing-impared person Vision based lip movement Lip reading School of Mechatronic Engineering
institution	Universiti Malaysia Perlis
collection	UniMAP Institutional Repository
language	English
advisor	Dr. Wan Khairunizam Wan Ahmad
topic	Lip movement Lip movement -- Analysis Hearing-impared person Vision based lip movement Lip reading
spellingShingle	Lip movement Lip movement -- Analysis Hearing-impared person Vision based lip movement Lip reading Muhamad Kamil Syahid, Talha A study of a vision-based lip movement analysis for hearing-impaired person
description	The study of vision based lip movement is the interpretation of human lip movement while speaking. Deaf and hard of hearing people often have a problem being able to understand what being talking in the conversation. Sign language may be useful for them to communicate but not everyone may understand the sign language. To make the communication more interesting and does not have any obstacle to talk to anybody, the lip reading is the best way to communicate. Deaf and hard hearing person can talk like the normal person, however, they do not hear the word spoken by themselves. On the other hand, there is no sound feedback system for them. This is a reason they could not speak with the correct pronunciations. To teach them pronounce the word correctly, fluently and to understand others in the conversation, the lip reading systems that could train them pronounce the word correctly needs to be developed. Previous researcher proposed many methods to recognize the spoken word based on the lip movement. One of the method is attach the colour marker onto the lip surface and a vision sensor is used to track the moving marker. Other researchers extract the lip region by using Active contour Model (ACM) or a snake method to draw the keypoint around lip edges. In this studies, a system to track and recognize the spoken word based on lip movements is proposed. A camera is used as a vision sensor and an image processing technique is employed to extract and track the lip. The lip horizontal and vertical distances of the lip are used to measure the ellipse while the systems track the lip movements. The trajectories of the ellipse are resampled to 10 point called the features point. The word database contains 10 spoken word frequently speak at the hospital are designed based on the distribution of the 10 feature points. Two types of word database have been designed which are the individual word database and the universal word database. The individual word database is defined as the distributed feature points of each subject by speaks the words with repetition. Meanwhile, the universal word database is defined as the distributed feature point of all word regardless who is the subjects. The recognition experiments are conducted and the system recognizes the unknown words with the recognition rate 92.47% accuracy by using the individual database. Meanwhile, the system recognizes the unknown words with the recognition rate 90.39% accuracy by using the universal database.
format	Thesis
author	Muhamad Kamil Syahid, Talha
author_facet	Muhamad Kamil Syahid, Talha
author_sort	Muhamad Kamil Syahid, Talha
title	A study of a vision-based lip movement analysis for hearing-impaired person
title_short	A study of a vision-based lip movement analysis for hearing-impaired person
title_full	A study of a vision-based lip movement analysis for hearing-impaired person
title_fullStr	A study of a vision-based lip movement analysis for hearing-impaired person
title_full_unstemmed	A study of a vision-based lip movement analysis for hearing-impaired person
title_sort	study of a vision-based lip movement analysis for hearing-impaired person
granting_institution	Universiti Malaysia Perlis (UniMAP)
granting_department	School of Mechatronic Engineering
url	http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/1/Page%201-24.pdf http://dspace.unimap.edu.my:80/xmlui/bitstream/123456789/61991/2/Full%20text.pdf
_version_	1747836852767817728

A study of a vision-based lip movement analysis for hearing-impaired person

Similar Items