A 200-student pilot showed strong content validity across relevance, comprehensiveness, comprehensibility, and face validity¶

Claim · All claims
Evidence · 1 study · design r3 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Pilot ratings on six content-validity questions were strong across all four assessed aspects. → Yueqiao Jin 2024

Evidence¶

Yueqiao Jin 2024¶

Yueqiao Jin, Roberto Martinez-Maldonado, Dragan Gašević and Lixiang Yan. (2024). GLAT: The Generative AI Literacy Assessment Test. https://arxiv.org/abs/2411.00283 doi:10.48550/arxiv.2411.00283

q2 · i? · design · r3

Pilot study with 200 higher education students rated six content-validity questions on five-point Likert scales (Cronbach's alpha = 0.81); Table 3 reports means from 4.22 to 4.62 across relevance, comprehensiveness, comprehensibility, and face validity.

"The content validity of the GLAT was assessed in a pilot study with a sample of 200 higher education students through six questions (Table 3) based on the Standards for Educational and Psychological Testing"

Discussion¶

Related Claims¶

• Pilot test events represented fewer grade levels overall, with more grade levels represented for low- and high-performing students, while the comparison group showed the opposite pattern — related
• The Content Proximity spring 2022 pilot study examined validity, reliability, and test score comparability of MAP Growth assessments using the new item-selection algorithm — related
• GLAT scores predicted performance on GenAI-supported tasks better than perceived ChatGPT proficiency — related
• Adapting the scoring mechanism substantially reduced ceiling effects found in the pilot study — related