Online Incremental Learning Based on Crowdsourcing for Indonesian Ontology Relation Extraction

Ontology is a form of structured knowledge representation. Ontology is largely used and developed in the process of information retrieval because of its ability to represent knowledge in a form that is both understandable by machine and human. With the increase of ontology scale and complexity is a greater challenge in extra-logical error identification. Most ontological engineering methods depend on machine learning where there is a risk of overlooking extra-logical error. One way to handle this is by crowdsourcing, that is dividing a large task into several smaller subtasks and employ the mass to complete them online. To utilise crowdsourcing, we change the offline and batch data processing into the online and incremental one. Online incremental learning constructs a model in an iterative manner right after a change is made, ensuring that previously acquired knowledge is maintained. The crowdsourcing participants will be asked to repeatedly validate those relations until the desired accuracy value is reached. From this research, we find that crowdsourcing is able to improve the model used in relation extraction process, from the F1-Score of 87.2 % to 89.8 %. This improvement using crowdsourcing reaches the same score as that using expert. Therefore, crowdsourcing is considered as able to correct extra-logical error accurately, just like expert. Besides, we also discover that offline incremental learning using Random Forest produces a model with higher accuracy than online incremental learning using Mondrian Forest. Random Forest model has the final accuracy value of 90.6 % while Mondrian Forest model has 89.7 %. From this result, we conclude that online incremental learning is unable to produce a better result than offline incremental learning in improving meronymy relation extraction process.