Automated analysis accuracies were on average 10% lower than the accuracies of manual analysis¶
Claim · All claims
Evidence · 1 study · designr2·q3
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? The accuracies of the automated analysis were on average 10% lower than the accuracies of the manual analysis. → Thomas Daniel Ullmann 2019
Evidence¶
Thomas Daniel Ullmann 2019¶
Thomas Daniel Ullmann. (2019). Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches. International Journal of Artificial Intelligence in Education (2019) 29:217–257. https://doi.org/10.1007/s40593-019-00174-2
q3 · i? · design · r2
Performance comparison in the standardized evaluation between machine learning predictions and coder annotations across the eight category data sets; the article reports the machine models were "on average 10% lower" than manual coding accuracies.
"The accuracies of the automated analysis were on average 10% lower than the accuracies of the manual analysis."
Discussion¶
Related Claims¶
- Praxical teacher-AI grading teaming decreased grading time by 44% and increased accuracy by 6% versus manual grading — reports the opposite
- Automatic DAC coding of dialogue acts agrees substantially with manual coding (79.0% agreement, kappa 0.75) — related
- Automatically coded dialogue acts correlate with manually coded collaborative activities as expected — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
ml-accuracy-ten-percent-below-manual-coding |
title |
Automated analysis accuracies were on average 10% lower than the accuracies of manual analysis |
description |
Automated analysis accuracies were on average 10% lower than the accuracies of manual analysis |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2019 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — thomas-daniel-ullmann-2019 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.