Expert Annotation Tools for Labeling Student Capstone Project based on ACM CCS Ontology

Students enrolled in an undergraduate computer science program must complete a capstone project as their final project. The selection of a capstone topic that is not appropriate can cause delays in the completion of this activity. Consequently, a thorough comprehension of the disciplines represented in each capstone title is required as a reference point for students when selecting topics that are appropriate for their academic skills. The popular ontology produced by the Association of Computing Machinery (ACM) can be used to learn about different computer science disciplines. Capstone title labeling on this ontology is applicable in topic recommendation systems that use clustering techniques to group titles into groups of similar topics. This technique requires a manual labeling process carried out by an expert, which has proven to be superior in other studies. However, this process is cumbersome and prone to errors. As a result, we present annotation tools that can assist experts in labeling based on the rules of the ACM Computing Classification System (CCS) ontology. Such tools have never been developed before in the education domain, despite the fact that educational datasets are widely available. In order to evaluate the efficacy, reliability, and accuracy of our system, we employ expert services to make manual annotations with and without the use of tools. The results of the comparison show that the manual labeling process with our tools are able to speed up the annotation process and increase the accuracy of the results achieved, as well as assist the resolution process of labeling disagreements among experts.