GAME's marks agree with a human marker's marks on student programming exercises and assignments¶

Claim · All claims
Evidence · 1 study · design r? · q2
Currency · newest study 2008 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Statistical comparison of GAME's marking against a human marker on student programming exercises and assignments shows encouraging agreement. → Blumenstein 2008

Evidence¶

Blumenstein 2008¶

Blumenstein, Michael; Green, Steve; Fogelman, Shoshana; Nguyen, Ann; Muthukkumarasarny, Vallipuram. (2008). Performance analysis of GAME: A generic automated marking environment. Computers and Education, 50(4). https://doi.org/10.1016/j.compedu.2006.11.006

q2 · i? · design · r?

The article reports testing GAME on student programming exercises and assignments and comparing its performance with a human marker, presenting an "in-depth statistical analysis of the comparison" with "encouraging results". No effect size or test statistic is printed in the supplied text.

"GAME has been tested on a number of student programming exercises and assignments and its performance has been compared against that of a human marker. An in-depth statistical analysis of the comparison is presented, providing encouraging results"

Discussion¶

Related Claims¶

• Automated assessment marks show very strong positive correlation with expert tutor marks on search algorithm exercises — related
• Kappa agreement with human experts shows a system is neither much worse nor much better than experts, but not that experts are a desirable comparison basis — related
• Peer marks and grades show acceptably high reliability with teachers' marks in most comparison studies, but a substantial minority report low reliability — related