A high Cronbach's alpha across items measuring distinct knowledge aspects indicates the test is not working as intended¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q1
Currency · no dated study recorded · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q1 i? When a domain knowledge test contains separate items designed for distinct, separable aspects, finding a high alpha coefficient indicates the test is failing its design intent. → alpha-1

Evidence¶

Stadler 2021¶

Stadler, Matthias, Michael Sailer, and Frank Fischer. 2021. “Knowledge as a formative construct: a good alpha is not always better.” New Ideas in Psychology 60: 100832. https://doi.org/10.1016/j.newideapsych.2020.100832.

q1 · i? · theoretical · r3

Analytical argument in the article's section on inadequate selection criteria. Formative indicators need not correlate highly; the article says "correlations among formative indicators should be low enough so that they are not redundant with each other in contributing to the latent construct."

"Finding that there is a high alpha coefficient across these items would actually indicate that the test is not working as intended."

Discussion¶

Related Claims¶

• Domain knowledge is theoretically a formative construct formed by distinct, not necessarily associated, aspects — related
• The 16-item CT–PCK Survey shows internal consistency reliability around .71–.72 and person separation reliability of .69 — related
• The post-study survey instrument showed high internal consistency (alpha = 0.90) — related
• A pilot study (N = 10) established timed-administration parameters, showing high internal consistency and minimal speededness for the SCS1 12-item subset — related
• The post-interaction reflection instrument showed satisfactory internal consistency (Cronbach's α = .863) — related
• CTAT shows robust psychometric properties for higher education CS/AI students, with reliable internal consistency and adequate IRT item parameters — related
• Questionnaire constructs showed acceptable to strong internal consistency (α = 0.740 to 0.945) — related
• TAILS dimensions show high internal consistency, with Cronbach's alpha values above 0.90 — related
• Reflective modeling of formative knowledge constructs risks misrepresenting domain knowledge as a general factor such as intelligence — related
• The eight Barkley Current Behavior Scale items chosen for DESR show high internal consistency — related
• Detection accuracy is more internally consistent within phishing and within legitimate emails (α = 0.77 each) than across all emails (α = 0.59) — related
• The MAKQ dimensions show acceptable internal consistency across both studies (alphas 0.77-0.93) — related
• Cronbach's alpha equals the reliability of the scale score under classical test theory, domain sampling, and generalizability theory, and should be interpreted as a reliability coefficient — related
• A low alpha does not necessarily indicate high content validity, because reliability and validity can both be high — related
• Adding items to a knowledge test can increase reliability without requiring the added items to be redundant with existing items — related
• An exam with low internal consistency cannot be used to verify peer assessment validity — related
• The TQsrl and SQsrl questionnaires are validated instruments measuring SRL sub-processes with emphasis on reflection — related
• Single-item effort ratings show high internal consistency but their reliability is difficult to establish — related
• Cronbach's alpha does not generally indicate the reliability of knowledge tests because knowledge is a heterogeneous construct — related
• Retest reliability may be more appropriate than alpha for knowledge tests, but requires a testing interval without learning — related