GLAT scores predicted performance on GenAI-supported tasks better than perceived ChatGPT proficiency¶

Claim · All claims
Evidence · 2 studies · 1 quant-synthesis r1, 1 associational r2 · q2–q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? GLAT, a 20-item objective GenAI literacy test validated with 355 higher-education students, predicted performance on GenAI-supported tasks better than perceived ChatGPT proficiency. → Daniele Verí 2026

q2 i? GLAT scores were significant predictors of learners' performance in GenAI-supported tasks and outperformed self-reported measures such as perceived ChatGPT proficiency. → Yueqiao Jin 2024

Evidence¶

Daniele Verí 2026¶

Daniele Verí. (2026). Beyond AI Literacy: A Structured Review and Exploratory Meta-Analysis of Measures for Competent Generative-AI Use. arXiv. https://arxiv.org/abs/2609.15624 doi:10.48550/arxiv.2609.15624

q3 · i? · quant-synthesis · r1

Criterion evidence from the GLAT validation study (Jin et al., 2025) as reported in the review's results section. The test showed internal consistency of α = .80 and ω = .81 with good 2PL IRT structural fit; the review reports no effect size for the criterion comparison.

"GLAT scores predicted performance on GenAI-supported tasks better than perceived ChatGPT proficiency [Jin et al., 2025]."

Yueqiao Jin 2024¶

Yueqiao Jin, Roberto Martinez-Maldonado, Dragan Gašević and Lixiang Yan. (2024). GLAT: The Generative AI Literacy Assessment Test. https://arxiv.org/abs/2411.00283 doi:10.48550/arxiv.2411.00283

q2 · i? · associational · r2

The article reports that GLAT scores predicted performance on a GenAI chatbot-supported visual-analytics comprehension task better than the self-reported ChatGPT Literacy Scale, in an OLS regression controlling for baseline score and visualisation literacy (mini-VLAT) with 83 students. Detailed regression coefficients appear beyond the provided text.

"GLAT scores were found to be significant predictors of learners’ performanceinGenAI-supportedtasks,outperformingself-reportedmeasuressuchasperceived ChatGPT proficiency and demonstrating external validity."

Discussion¶

Related Claims¶

• Existing AI literacy assessment instruments poorly cover Stages 0–1 and Stage 4 of the continuum — related
• Instruction type predicts post-test performance with a large effect size after controls (linear and quantile regression) — related
• GLAT shows robust structural validity and reliability under a 2-parameter logistic IRT model — related
• Off-line self-report instruments show low concurrent validity with on-line measures and do not predict learning outcomes very well — a broader claim this one bears on
• CTT item selection dropped five low-discrimination items, retaining 20 items of moderate difficulty — related
• A 200-student pilot showed strong content validity across relevance, comprehensiveness, comprehensibility, and face validity — related
• Post-2024 instrument development expanded objective AI-literacy measurement, which had been dominated by self-report scales — a broader claim this one bears on