Classification of Illustrated Question for Indonesian National Assessment with Deep Learning

Reading literacy is a part of the Indonesian National Assessment. Each question has an illustrative image to help the reader understand the information conveyed. The dataset in this study was collected manually from official government platforms. However, in this study, we encountered a small dataset problem, so data augmentation was needed to increase the variation of the training data. This research focuses on the use of deep learning, namely the Convolutional Neural Network (CNN) to classify illustrative images into information or literature classes. We use five backbones, Visual Geometry Group 16 (VGG-16), MobileNetV2, Residual Network 50 (ResNet-50), Dense Network (DenseNet-121), and EfficientNetB4. Then we compared accuracy, F1 score, loss function to our evaluation metrics. We apply the transfer and fitting learning method to solve small data set problems. The best result in this study is using the ResNet-50 model and the fastest is using the Adam optimizer but it easily causes overfitting.