Multiview Sentiment Analysis with Image-Text-Concept Features of Indonesian Social Media Posts

Social media development makes it possible for everyone to express their opinions and information through text, speech, video, or images. Multiview sentiment analysis in current studies generally combines two modalities, text and image. It seeks to classify social media posts into two or more polarities, such as positive, neutral, or negative. To improve the performance of multiview Sentiment Analysis, we added another modality, which is concepts derived from text and image. Our proposed model integrates three views into a fusion with an ensemble approach by a metaclassifier. We performed text classification with Deep Convolutional Neural Networks. The input feature is Word2Vec for text representation in order to preserve semantic meaning. Additionally, we analyzed concepts from texts with SenticNet 5 as a knowledge base model and extracted concepts from images using the DeepSentiBank model. We obtained 2089 Adjective Noun Pairs and classified it with Multi-Layer Perceptron. Then we combined predicted probabilities from each classifier for Image, Text, and Concept by Ensemble Learning. A meta-classifier was implemented to predict the final sentiment from a fusion of Image-Text-Concept features. The fusion for multiview sentiment analysis works well and could achieve the best accuracy of 70% by applying the ensemble approach with Logistic Regression as the meta-classifier.