Recognition of Real-Time Angklung Kodály Hand Gesture using Mediapipe and Machine Learning Method

Muhammad Zulfikar Fauzi*, Riyanarto Sarno

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The application of machine learning in the arts and culture can help preserve these arts and culture and, in some cases even facilitate access to the arts to the public. In this study, a hand gesture recognition system was developed for the Kodaly method hand gesture to play angklung directly from a personal computer. Many methods of hand gesture recognition still need an additional hardware such as gloves and expensive sensors that may not be accessible to some people. This study proposes a computer vision-based approach utilizing the personal computer internal webcam without additional hardware and employs multi-layer perceptron classifier for real-time recognition and Mediapipe for feature extraction. The developed system achieves an average accuracy of 99% and runs with low latency between recognition and audio output. The effective range of hand gesture recognition is up to 1.5 meters in environment with a light level as low as 25 LUX.

Original languageEnglish
Title of host publicationICCoSITE 2023 - International Conference on Computer Science, Information Technology and Engineering
Subtitle of host publicationDigital Transformation Strategy in Facing the VUCA and TUNA Era
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages980-985
Number of pages6
ISBN (Electronic)9798350320954
DOIs
Publication statusPublished - 2023
Event2023 International Conference on Computer Science, Information Technology and Engineering, ICCoSITE 2023 - Jakarta, Indonesia
Duration: 16 Feb 2023 → …

Publication series

NameICCoSITE 2023 - International Conference on Computer Science, Information Technology and Engineering: Digital Transformation Strategy in Facing the VUCA and TUNA Era

Conference

Conference2023 International Conference on Computer Science, Information Technology and Engineering, ICCoSITE 2023
Country/TerritoryIndonesia
CityJakarta
Period16/02/23 → …

Keywords

  • Angklung
  • Computer vision
  • Hand gesture recognition
  • Multi-layer Perceptron

Fingerprint

Dive into the research topics of 'Recognition of Real-Time Angklung Kodály Hand Gesture using Mediapipe and Machine Learning Method'. Together they form a unique fingerprint.

Cite this