Reliability of a test is a prerequisite for valid comparison, experimentation, classification and diagnosis¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q2
Currency · newest study 1921 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Comparisons of group averages or variability, experimental conclusions, classifications and diagnoses are impermissible or faulty when test reliability is unknown or low. → Franzen 1921

Evidence¶

Franzen 1921¶

Franzen, R., & Knight, F. B. (1921). Criteria to Employ in Choice of Tests. The Journal of Educational Psychology. https://doi.org/10.1037/h0075312

q2 · i? · theoretical · r2

Analytical argument that the reliability of averages and spread measures are functions of test reliability, so group comparisons are 'not permissible unless we know the reliability of the tests'. Support is the authors' reasoning, illustrated by the telephone-diagnosis analogy; no data reported.

"the use for experimentation of a test with a low reliability leads to faulty conclusions and classification; diagnoses and definition of goals made on a basis of unreliable material is tantamount to prescriptions of a doctor who has made a diagnosis over a telephone."

Discussion¶

Related Claims¶

• The prerequisite-violating knowledge-state pattern is primarily attributable to Q-matrix misspecification, and the original hierarchy shows superior predictive efficiency — related
• Validity is the only test criterion that applies to all five administrative uses of tests — related
• A semi-supervised ANN method combining DINA and DINO achieves appreciated classification performance across test conditions, especially when diagnostic quality is not high or the Q-matrix contains misspecified elements — related
• Scaling is a sine qua non for experimental use of tests, and unscaled tests interpreted as scaled do harm — related
• Misspecification of theoretical diagnostic classification models and inaccurate Q-matrices impact classification accuracy — related