Combining decision tree and back propagation genetic algorithm neural network for recognizing word gestures in Indonesian Sign Language using Kinect

Sign language is a media for speech and/or hearing problem’s people to communicate. Different kind of sign languages exist in the world such as Indonesian Sign Language (ISL), American Sign Language (ASL), Chinese Sign Language (CSL), British Sign Language (BSL), Brazilian Sign Language (BSL), and France Sign Language (FSL). In Indonesia, the used of ISL was less extensive because not all people understand it. People that do not have understanding on ISL cannot translate it. Therefore an ISL translation system is required. Many researches about sign language translation system had been done for FSL, BSL, FSL, and CSL. However, research on ISL is still limited and still need development. Therefore we proposed a new system for recognizing ISL word gestures. In this research we captured user skeleton by using Kinect. From those skeletons only nine skeletons were used as feature by computing their vector value, angle value, and distance value. Totally 28 features were extracted. Then the combination of Decision Tree and Back Propagation Neural Network (BPGANN) was implemented for classifier. For experiment, eight ISL vocabularies were tested by two people. The recognition accuracy of this system, although evaluated with small vocabulary, presents very promising result with value 96%.