A realistic visual speech synthesis for Indonesian using a combination of morphing viseme and syllable concatenation approach to support pronunciation learning

Aripin*, Hanny Haryanto, Surya Sumpeno

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)

Abstract

This research aims to build a realistic visual speech synthesis for Indonesian so that it can be used to learn Indonesian pronunciation. In this research, We used the combination of morphing viseme and syllable concatenation method. The morphing viseme method is a process of deformation from one viseme to another, so that the animation of the mouth shape looks smoother. This method is used to create the transition of animation between viseme. The Syllable Concatenation method is used to assemble viseme based on certain syllable patterns. We built a syllable-based voice database as a basis for synchronization between syllables, speech and viseme models. The method proposed in this research consists of several stages, namely the formation of Indonesian viseme models, designing facial animation character, development of speech database, a synchronization process and subjective testing of the resulting application. Subjective tests were conducted on 30 respondents who assessed the suitability and natural movement of the mouth when uttering the Indonesian texts. The MOS (Mean Opinion Score) method is used to calculate the average of respondents' scores. The MOS calculation results for the criteria of Synchronization and naturalness are 4,283 and 4,107 on the scale of 1 to 5. This result shows that the level of Synchronization and naturalness of the synthesis of visual speech is more realistic. Therefore, the system can display the visualization of phoneme pronunciation to support learning Indonesian pronunciation.

Original languageEnglish
Pages (from-to)19-37
Number of pages19
JournalInternational Journal of Emerging Technologies in Learning
Volume13
Issue number8
DOIs
Publication statusPublished - 2018

Keywords

  • Morphing viseme
  • Realistic
  • Syllable concatenation
  • Visual speech synthesis for Indonesian

Fingerprint

Dive into the research topics of 'A realistic visual speech synthesis for Indonesian using a combination of morphing viseme and syllable concatenation approach to support pronunciation learning'. Together they form a unique fingerprint.

Cite this