Recognition of Real-Time BISINDO Sign Language-to-Speech using Machine Learning Methods

Muhammad Zulfikar Fauzi*, Riyanarto Sarno, Shintami Chusnul Hidayati

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Citations (Scopus)

Abstract

In this study, a sign language-to-speech system was developed to recognize and convert BISINDO's sign language into speech using a machine learning approach. The speech output will make it easier for the user to communicate with the other person and will make it easier for the other person to understand sign language and will improve the quality of communication. Using the dataset produced in this study and Mediapipe for feature extraction, the model accuracy was able to obtain a score of 98% using the Support Vector Machine method. However, the accuracy score of the model decreased drastically reaching 78% in trials conducted directly on users because the testing exceeded the system effective range. The results of the implementation of Sign Language-to-Speech succeeded in producing an output in form of audio speech without using an internet connection. The system was able to detect both dynamic and static gesture from the user in real-time.

Original languageEnglish
Title of host publicationICCoSITE 2023 - International Conference on Computer Science, Information Technology and Engineering
Subtitle of host publicationDigital Transformation Strategy in Facing the VUCA and TUNA Era
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages986-991
Number of pages6
ISBN (Electronic)9798350320954
DOIs
Publication statusPublished - 2023
Event2023 International Conference on Computer Science, Information Technology and Engineering, ICCoSITE 2023 - Jakarta, Indonesia
Duration: 16 Feb 2023 → …

Publication series

NameICCoSITE 2023 - International Conference on Computer Science, Information Technology and Engineering: Digital Transformation Strategy in Facing the VUCA and TUNA Era

Conference

Conference2023 International Conference on Computer Science, Information Technology and Engineering, ICCoSITE 2023
Country/TerritoryIndonesia
CityJakarta
Period16/02/23 → …

Keywords

  • BISINDO
  • Gesture Recognition
  • Machine Learning
  • Speech

Fingerprint

Dive into the research topics of 'Recognition of Real-Time BISINDO Sign Language-to-Speech using Machine Learning Methods'. Together they form a unique fingerprint.

Cite this