Retest reliability may be more appropriate than alpha for knowledge tests, but requires a testing interval without learning¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r? · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Retest reliability, not alpha, predicted validity indices for a personality inventory in a large study, suggesting retest reliability may better suit heterogeneous constructs like knowledge. → Peter A. Edelsbrunner 2025 q2 i? Estimating retest reliability of a knowledge test requires an interval during which no learning occurs, since knowledge changes could lower test-retest correlations even with low measurement error. → Peter A. Edelsbrunner 2025 (2)

Evidence¶

Peter A. Edelsbrunner 2025¶

Peter A. Edelsbrunner, Bianca A. Simonsmeier, Michael Schneider. (2025). The Reliability, But Not the Cronbach's Alpha, of Knowledge Tests Matters: Response to Zitzmann and Orona (2025). Educational Psychology Review. https://doi.org/10.1007/s10648-025-10023-5

q2 · i? · associational · r?

The commentary reports, citing McCrae et al. (2011), a study of 34,108 participants in which "only retest reliability, not alpha, predicted validity indices for the NEO Personality Inventory." The reviewed study's design is not described beyond this.

"A study of 34,108 participants found that only retest reliability, not alpha, predicted validity indices for the NEO Personality Inventory (McCrae et al., 2011)."

Peter A. Edelsbrunner 2025 (2)¶

Peter A. Edelsbrunner, Bianca A. Simonsmeier, Michael Schneider. (2025). The Reliability, But Not the Cronbach's Alpha, of Knowledge Tests Matters: Response to Zitzmann and Orona (2025). Educational Psychology Review. https://doi.org/10.1007/s10648-025-10023-5

q2 · i? · theoretical · r3

The authors' stated boundary condition: knowledge changes through instruction or development could lower test-retest correlations even when measurement error is low, so retest reliability "is no panacea."

"However, estimating the retest reliabil- ity of a knowledge test requires a testing interval during which no learning occurs."

Discussion¶

Related Claims¶

• Confidence intervals around alpha do not resolve the problem because they inherit alpha's failure to indicate knowledge-test reliability — related
• A high Cronbach's alpha across items measuring distinct knowledge aspects indicates the test is not working as intended — related
• Validation evidence across the 15 RCR measures is partial and heterogeneous, with no test–retest, criterion validity or factor-analytic evidence reported — related
• Validity coefficients with external criteria provide a lower bound for a test's reliability — related