The final PHP ITS version correctly analysed 96.31% of student solutions to programming exercises¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The final version of the PHP ITS achieved 96.31% accuracy in correctly analysing student-submitted PHP programs. → Weragama 2014

Evidence¶

Weragama 2014¶

Weragama, D. & Reye, J. (2014). Analysing Student Programs in the PHP Intelligent Tutoring System. Int J Artif Intell Educ 24:162–188. https://doi.org/10.1007/s40593-014-0014-z

q2 · i? · design · r2

Evaluation with two groups of postgraduate students at Queensland University of Technology across two semesters; 678 non-duplicate solutions analysed. The final version achieved "an analysis accuracy of 96.31 %", up from 80.53% in earlier versions.

"Although the final version of the PHP ITS showed an analysis accuracy of 96.31 %, the results obtained from the two earlier versions were somewhat less accurate. Of the 678 solutions analysed, 546 were analysed correctly by those versions. This is 80.53 %"

Discussion¶

Related Claims¶

• Per-exercise analysis accuracy improved from 15 of 27 exercises above 80% to all 27 above 80% in the third version — related
• Two main causes of incorrect analysis were unhandled constructs and string-literal quote inconsistencies — related
• PHP ITS identifies semantically equivalent solutions without bug libraries, ideal solutions or heuristics — related
• 81.54% of incorrectly analysed programs were incorrect programs analysed as correct — related
• 16.08% of student programs contained unnecessary statements — related