A model of Indonesian dynamic visemes from facial motion capture database using a clustering-based approach

Realistic 3D facial animation is a challenging task in the entertainment industries. One of the efforts is to build a realistic lips animation. This research aims to build a model of Indonesian Dynamic visemes based on the results of the clustering process of the facial motion capture (MoCap) database. The Subspace LDA (Linear Discriminant Analysis) method is used to reduce the dimension. The Subspace LDA method is a combination of the PCA (Principal Component Analysis) and the LDA method. The clustering process is used to make up a natural grouping of data features which its dimensions are reduced into a number of groups. The quality of cluster results is measured by using Sum Square Error (SSE) and a ratio of Between-Class Variation (BCW) and Within-Class Variation (WCV). The measurement shows that the results of the clustering process achieving the best quality occurs at k = 38. In this research, it has been found out that the class structure of Indonesian dynamic visemes consists of 39 classes (38 classes from the clustering process and 1 class for neutral). For the future work, the results of this research can be used as a basis to build Indonesian visual speech synthesis smoother and as a reference to determine a structure of Indonesian dynamic visemes based on linguistic knowledge.