Similarities between human and artificial intelligence evaluations applied on engineering students on first-year kinematics learning through an argumentative item
Academic Article in Scopus
The scope of an argumentative evaluation allows for evaluating comprehension, conceptual understanding, and similarity with the answer of an expert in the field, in this case, STEAM. Teachers have assessment resources, with some risk of bias. An automated assessment tool, based on AI can objectively identify relevant didactic aspects, previously established. This paper investigates how the argued responses of 244 engineering students to an acceleration problem are formed, analyzed from 1) their similarity with the response of an expert by the AI tool and 2) their level of understanding (I-IV), in the light of SOLO taxonomy, by a group of expert teachers in the teaching of Physics. The findings of this work show that: 1) the tool calculated that the average percentage of response similarity with an expert is 44%, 2) the group of teachers reported that the prestructural (I) and relational (IV) levels of understanding obtained the highest percentage in the evaluation of the average and instantaneous acceleration, followed by the prestructural and unistructural levels in the characterization of the acceleration. The integration of results broadens the perspective of evaluation and analysis of the conceptual understanding achieved by the students since it quantifies how much their answers are aligned with that of an expert. Moreover, the levels of understanding of the concept of acceleration are displayed. In this way, the use and combination of this AI tool contribute to a better understanding gaps and conceptual difficulties in first-year engineering students.
