A theory-driven trace-based SRL measurement achieved a match rate of 38.97% with think aloud data when evaluated against think aloud as reference point¶
Claim · All claims
Evidence · 1 study · designr2·q3
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2025: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? The theory-driven process library achieved a match rate of 38.97% (median) with think aloud codes in the first alignment, indicating low construct validity of the initial trace-based SRL measurement. → Fan 2022
Evidence¶
Fan 2022¶
Fan, Y., van der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., Moore, J., Molenaar, I., Bannert, M., & Gašević, D. (2022). Towards investigating the validity of measurement of self-regulated learning based on trace data. Metacognition and Learning. https://doi.org/10.1007/s11409-022-09291-1
q3 · i? · design · r2
First alignment between theory-driven SRL processes extracted from trace data and think aloud codes for 32 participants in the training set of the laboratory study. The match rate was 38.97% (median), with low sensitivity and specificity, indicating the need to improve validity of the trace-based measurement relative to the reference point.
"As shown in Table 6, we only achieved a match rate of 38.97% (median) between the theory-driven SRL processes and think aloud codes at the first alignment. The sensitivity and specificity are also low based on this first alignment."
Discussion¶
Related Claims¶
- The largest source of mismatch between trace-based and think aloud SRL processes was confusion between elaboration/organisation and first-reading processes — related
- The combined theory-driven and data-driven validation approach improved the match rate between trace-based SRL processes and think aloud data from 38.97% to 54.24% (training) and 34.54% to 55.09% (testing) — related
- Substantial differences existed between the time distribution of trace-extracted and think aloud-coded SRL processes: 41.66% of session duration had no think aloud code while only 3.19% had no trace process — related
- Only nine of 66 reviewed studies specified a theoretical basis, with seven using regulation of learning theory and two using Community of Inquiry — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
theory-driven-srl-trace-measurement-low-match-rate |
title |
A theory-driven trace-based SRL measurement achieved a match rate of 38.97% with think aloud data when evaluated against think aloud as reference point |
description |
A theory-driven trace-based SRL measurement achieved a match rate of 38.97% with think aloud data when evaluated against think aloud as reference point |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2022 |
recheck_by |
2025 |
generated |
by process:wiki-ingest on 2026-10-16 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — fan-2022 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.