In log analysis of 3466 interactions, 56.1% of interactions and 46.3% of attempts were recognised as correct model solutions¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Analysis of 5950 log entries (3466 diagnosis interactions from 116 of 285 students) showed 56.1% of interactions and 46.3% of attempts were recognised as matching a model solution, with 82.9% of interactions classified. → Gerdes 2016
Evidence¶
Gerdes 2016¶
Gerdes, A., Heeren, B., Jeuring, J., & van Binsbergen, L. T. (2016). Ask-Elle: an Adaptable Programming Tutor for Haskell Giving Automated Feedback. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-015-0080-x
q2 · i? · design · r2
Log-file analysis of Ask-Elle usage in a second-year university class at Utrecht University (September 2013), with 3466 diagnosis interactions from 116 of 285 registered students across 26 exercises. "Currently, 56.1% of the interactions (and 46.3% of the attempts) are recognised to be correct." The authors note this ratio is a lower bound due to possible undetected incorrect solutions in the Tests passed and Discarded classes.
"Currently, 56.1% of the interactions (and 46.3% of the attempts) are recognised to be correct."
Discussion¶
Related Claims¶
- Compiler errors accounted for over half of diagnosis interactions in the log analysis — related
- Strategy-based assessment statically determines which solution strategy a student used and guarantees correctness of recognised programs — related
- Students rated the tutor 3.4 out of 5 overall and particularly liked the worked-out solution feedback
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
ask-elle-log-analysis-recognition-rates |
title |
In log analysis of 3466 interactions, 56.1% of interactions and 46.3% of attempts were recognised as correct model solutions |
description |
In log analysis of 3466 interactions, 56.1% of interactions and 46.3% of attempts were recognised as correct model solutions |
status |
review |
evidence_strength |
weak |
newest_evidence |
2016 |
generated |
by process:wiki-ingest on 2026-10-28 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — gerdes-2016 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.