The combined theory-driven and data-driven validation approach improved the match rate between trace-based SRL processes and think aloud data from 38.97% to 54.24% (training) and 34.54% to 55.09% (testing)¶
Claim · All claims
Evidence · 1 study · designr2·q3
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2025: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? The validation approach improved the match rate between SRL processes extracted from trace data and think aloud data in both training and testing sets. → Fan 2022
Evidence¶
Fan 2022¶
Fan, Y., van der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., Moore, J., Molenaar, I., Bannert, M., & Gašević, D. (2022). Towards investigating the validity of measurement of self-regulated learning based on trace data. Metacognition and Learning. https://doi.org/10.1007/s11409-022-09291-1
q3 · i? · design · r2
Laboratory study with 44 learners studying about artificial intelligence in education in a technology-enhanced learning environment for 45 minutes. The improved match rate quantifies the extent to which validity improved by combining empirical evidence from think aloud data and rationale from the theoretical framework of SRL.
"we achieved an improved match rate between SRL processes extracted from trace-data and think aloud data (training set: 54.24%; testing set: 55.09%) compared to the match rate before applying the validation approach (training set: 38.97%; test set: 34.54%)."
Discussion¶
Related Claims¶
- The largest source of mismatch between trace-based and think aloud SRL processes was confusion between elaboration/organisation and first-reading processes — related
- A theory-driven trace-based SRL measurement achieved a match rate of 38.97% with think aloud data when evaluated against think aloud as reference point — related
- Substantial differences existed between the time distribution of trace-extracted and think aloud-coded SRL processes: 41.66% of session duration had no think aloud code while only 3.19% had no trace process — related
- Adding think-aloud data to theory prompts improves GPT-4o codebook completeness and alignment with the data — related
- Only nine of 66 reviewed studies specified a theoretical basis, with seven using regulation of learning theory and two using Community of Inquiry — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
validation-approach-improves-match-rate |
title |
The combined theory-driven and data-driven validation approach improved the match rate between trace-based SRL processes and think aloud data from 38.97% to 54.24% (training) and 34.54% to 55.09% (testing) |
description |
The combined theory-driven and data-driven validation approach improved the match rate between trace-based SRL processes and think aloud data from 38.97% to 54.24% (training) and 34.54% to 55.09% (testing) |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2022 |
recheck_by |
2025 |
generated |
by process:wiki-ingest on 2026-10-16 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — fan-2022 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.