Handwriting Classification based on Hand Movement using ConvLSTM

Awang Karisma As Ad Adi Asta, Eko Mulyanto Yuniarno, Supeno Mardi Susiki Nugroho, Cries Avian

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The recognition of handwritten text presents challenges due to the variability and complexity of human handwriting, making it difficult to capture subtle nuances through traditional methods. Hand gesture recognition has emerged as an alternative method for predicting handwritten text, using sensors such as Kinect, LeapMotion, gyroscopes, accelerometers, and electromyograms to extract geometric and spatial information. Continuous hand-gesture recognition using cameras is preferred due to its ease of use and low hardware costs. Researchers have proposed different methods for recognizing hand gestures, including fuzzy logic, deterministic finite automata, trajectory-based methods, and dynamic probability long short-term memory (DP-LSTM). However, the latest research has shown that using LSTM can result in spatial information being lost. Therefore, this work proposes an architecture that captures spatial information using Convolutional Neural Network (CNN) and LSTM as Conv-LSTM, achieving high recognition rates in hand gesture trajectories for letters a to e in English captured using MediaPipe. Our results show that our proposed model can achieve high accuracy in classification and attained 0.8438.

Original languageEnglish
Title of host publication2023 International Seminar on Intelligent Technology and Its Applications
Subtitle of host publicationLeveraging Intelligent Systems to Achieve Sustainable Development Goals, ISITIA 2023 - Proceeding
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages341-346
Number of pages6
ISBN (Electronic)9798350313956
DOIs
Publication statusPublished - 2023
Event24th International Seminar on Intelligent Technology and Its Applications, ISITIA 2023 - Hybrid, Surabaya, Indonesia
Duration: 26 Jul 202327 Jul 2023

Publication series

Name2023 International Seminar on Intelligent Technology and Its Applications: Leveraging Intelligent Systems to Achieve Sustainable Development Goals, ISITIA 2023 - Proceeding

Conference

Conference24th International Seminar on Intelligent Technology and Its Applications, ISITIA 2023
Country/TerritoryIndonesia
CityHybrid, Surabaya
Period26/07/2327/07/23

Keywords

  • convolution
  • hand gesture
  • handwriting recognition
  • lstm
  • mediapipe

Fingerprint

Dive into the research topics of 'Handwriting Classification based on Hand Movement using ConvLSTM'. Together they form a unique fingerprint.

Cite this