Exploring the Psychometric Properties of Computational Thinking Assessment in Introductory Programming

Computational thinking (CT) is considered as the skill of 21st century. The fundamental CT concepts include abstraction, algorithm design, decomposition, pattern recognition, and data representation or generalization. CT assessment is required to improve the understanding of the cognitive abilities and to relate them in related areas, such as introductory programming in computer science. Assessing computational thinking skills however is a challenging issue since it measures latent variables that cannot be directly observed. In addition, according to psychometrics, appropriate test requires a validation process before it can be effectively used as a measuring instrument. Thus, high quality tools for measuring student learning in Introductory Programming have been under-developed and under-researched. The objective of this work is to determine the psychometric properties (item validity, reliability, discrimination, difficulty, and distractors) of the developed multiple-choice questions of computational thinking in introductory programming by exploring classical test theory and item response theory which has not been deeply investigated by previous studies. The analysis results reveal that most of items are valid and the items are generally adequate reliable. In spite of the fact that some items are suggested to be revised since the item discrimination values, the distribution of difficulties, and distractor points are less than the expected threshold.