Skip to content

Messick's six-source unified validity framework for AI evaluation

Research Method · All research methods
Evidence · 1 claim (1 for) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description

The article anchors its validity discussion in Messick's unified theory of validity, which holds that validity is a property of inferences and decisions based on scores, not of instruments. It enumerates six interlocking evidence sources (content, substantive, structural, generalizability, external, and consequential) and gives each an AI analogue, such as item-level factor analysis for structural evidence and out-of-distribution testing for generalizability evidence.

Accounts

  • Messick's unified validity framework applied to AI evaluation, with six sources of construct-validity evidence: The article anchors its validity discussion in Messick's unified theory of validity, which holds that validity is a property of inferences and decisions based on scores, not of instruments. It enumerates six interlocking evidence sources (content, substantive, structural, generalizability, external, and consequential) and gives each an AI analogue, such as item-level factor analysis for structural evidence and out-of-distribution testing for generalizability evidence. (Peng Wang (2026))

Claims

-

Key Sources

  • Peng Wang, Fred Paas. (2026). Five Educational-Psychology Lenses for Training and Evaluating AI Models. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10195-8
Page metadata
Field Value
type research-method
id messicks-six-source-unified-validity-framework-for-ai-evaluation
title Messick's six-source unified validity framework for AI evaluation
description The article anchors its validity discussion in Messick's unified theory of validity, which holds that validity is a property of inferences and decisions based on scores, not of instruments.
status review
generated by process:settle-candidates on 2026-10-21
trust unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.