Reflective modeling of formative knowledge constructs risks misrepresenting domain knowledge as a general factor such as intelligence¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q1
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? A high alpha on a domain knowledge test may reflect general learner-related factors, so the latent factor could be misinterpreted as general intelligence rather than domain knowledge. → misrep-1

Evidence¶

Stadler 2021¶

Stadler, Matthias, Michael Sailer, and Frank Fischer. 2021. “Knowledge as a formative construct: a good alpha is not always better.” New Ideas in Psychology 60: 100832. https://doi.org/10.1016/j.newideapsych.2020.100832.

q1 · i? · theoretical · r3

Analytical argument from the article's misrepresentation section. It reasons that high alpha may indicate "a lot of the variance is due to general learner-related factors (e.g., intelligence, study diligence, interest)" so the instrument does not differentiate conceptual material.

"In this case, a measure of domain knowledge would be mis- represented as a measure of, for instance, general intelligence."

Discussion¶

Related Claims¶

• Domain knowledge is theoretically a formative construct formed by distinct, not necessarily associated, aspects — related
• Research practice misaligns with theory: domain knowledge is usually assessed with reflective models without argument for that choice — a broader claim this one bears on
• A high Cronbach's alpha across items measuring distinct knowledge aspects indicates the test is not working as intended — related
• A low alpha does not necessarily indicate high content validity, because reliability and validity can both be high — related
• Cronbach's alpha does not generally indicate the reliability of knowledge tests because knowledge is a heterogeneous construct — related