Hybrid approach of structural lyric and audio segments for detecting song emotion

Detecting song emotion is very important, however many studies have been done based on song lyrics and song audio separately. This research proposes a method for detecting song emotion based on integrated song lyrics and audio. Synchronizing the right structural segment and lyrics of song can be used for hybrid approach to detected right emotion. Song emotion can be classified into Thayer emotion label. The features of a song lyric are extracted using Psycholinguistic and Stylistic; whereas the features of a song audio are extracted using analyze audio signal waveform using Fast Fourier Transform (FFT) method. A song can be divided into 5 structural segments, which are intro, chorus, bridge, verse and outro. A preprocessing method of audio uses Correlation Features Selection (CFS) and preprocessing text for lyrics. Six classification methods are used for classifying emotion based on lyrics and audio of song structural segments separately. The aggregate method is used to analyze the results of classification before to obtain structural segments that represent emotions. Then, the final process Hybrid approach used to combine audio and lyrics features in emotion detection. Sum of matrix and Majority Voting Concept are used for Hybrid approach. The value of F-Measure is 0.823.