Stealth assessment facets show good internal consistency (Cronbach's alpha = 0.76) while MicroDYN reliabilities were low in this middle-school sample¶
Claim · All claims
Evidence · 1 study · designr2·q3
Currency · newest study 2016 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2019: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Cronbach's alpha was 0.76 for the problem solving stealth assessment, but 0.67 for MicroDYN rule acquisition and 0.43 for rule application. → Shute 2016
Evidence¶
Shute 2016¶
Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106-117. https://doi.org/10.1016/j.chb.2016.05.047
q3 · i? · design · r2
Internal consistency analysis of the validation study measures. The article reports alpha = 0.76 for the stealth assessment facets and lower alphas for MicroDYN, attributing the low MicroDYN reliabilities to middle school students' difficulty with the tasks and noting acceptable reliabilities in older samples.
"For the problem solving stealth assessment, Cronbach 's a ¼ 0.76, which indicates good internal reliability among the four problem solving facets. However, Cronbach 's a ¼ 0.67 for rule acquisition, and Cronbach's a ¼ 0.43 for rule application relative to MicroDYN, were less than ideal."
Discussion¶
Related Claims¶
- MicroDYN is a stronger predictor of the in-game problem solving estimates than Raven's, with the rule-application facet more relevant than rule acquisition — related
- Adding MicroDYN rule application significantly improves prediction of overall problem solving, planning, tool usage, and evaluating progress, but not analyzing givens — related
- Stealth assessment estimates of problem solving in Use Your Brainz correlate significantly with external measures (Raven's and MicroDYN) in middle-school students — related
- An exam with low internal consistency cannot be used to verify peer assessment validity — related
- Facet difficulty and intercorrelation patterns: analyzing givens was easiest, evaluating progress hardest, and planning correlated 0.98 with the overall estimate — related
- Alpha of knowledge tests is higher for language and mathematics than for science and social sciences/humanities — related
- Raven's and MicroDYN jointly explain 22% of variance in the stealth assessment estimate, with MicroDYN adding 5.7% unique variance beyond Raven's — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
internal-consistency-stealth-versus-microdyn |
title |
Stealth assessment facets show good internal consistency (Cronbach's alpha = 0.76) while MicroDYN reliabilities were low in this middle-school sample |
description |
Stealth assessment facets show good internal consistency (Cronbach's alpha = 0.76) while MicroDYN reliabilities were low in this middle-school sample |
status |
review |
evidence_strength |
weak |
newest_evidence |
2016 |
source_checked |
2026-10-20 |
recheck_by |
2019 |
generated |
by process:wiki-ingest on 2026-10-16 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — shute-2016 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.