Generative AI Literacy Assessment Test (GLAT)

Product or Programme · All products and programmes
Evidence · 5 claims (4 for, 1 mixed) · 4 studies (1 causal, 1 quant-synthesis, 1 associational, 1 design), q2–q3 · 2 of 4 report an effect size · 5 claims rest on one study

Description

A 20-item multiple-choice performance assessment developed by Jin et al. to measure demonstrated generative AI knowledge, application, evaluation and creation skills, and ethical understanding among learners.

Components

• Generative AI Literacy Assessment Test (GLAT) as an objective AI literacy measure: The GLAT (Jin et al., 2025) is "a 20-item multiple-choice performance assessment of demonstrated generative AI knowledge, skills, and ethical understanding", with domains covering Know & understand (8 items), Use & apply (3 items), Evaluate and create (4 items), and Ethics (5 items). Because the study outcome also assessed ethical judgment, the authors computed a 15-item Non-Ethics GLAT score (KR-20 = 0.65) for primary analyses to reduce criterion contamination; the full 20-item score showed KR-20 = 0.72. (Reed et al. (2026))
• Generative AI Literacy Assessment Test (GLAT), a 20-item performance-based instrument: The GLAT is a performance-based 20-item multiple-choice instrument measuring generative AI literacy across four dimensions: understanding, applying, creating with, and making ethical judgements about GenAI. Each item has four response options, and a corrected total score (with correction for guessing) served as a standardised continuous predictor in the study's regression and mediation models. It was chosen over self-report confidence scales because it assesses demonstrated knowledge relevant to prompting, evaluating, and using generative systems. (Yueqiao Jin et al. (2025))
• GLAT: a 20-item multiple-choice performance-based assessment of GenAI literacy for higher education: GLAT is a performance-based multiple-choice instrument designed to measure generative AI literacy among higher education students, targeting technical proficiency, ethical awareness, and critical evaluation of GenAI outputs. It was built through blueprint construction, item generation, expert review, and CTT-based item selection, yielding a final 20-item set. The article reports it as filling a gap in self-report-dominant AI literacy assessment. (Yueqiao Jin et al. (2024))
• Four-dimension GenAI literacy blueprint: Know & Understand, Use & Apply, Evaluate & Create, Ethics: The GLAT blueprint operationalises GenAI literacy using the four AI literacy dimensions proposed by Ng et al. (2021b), specialised to generative AI: "1) Know & Understand, 2) Use & Apply, 3) Evaluate & Create, and 4) Ethics". Two researchers independently reviewed 19 source documents, consolidated a refined set of 25 GenAI concepts across these dimensions (e.g., foundation models, RAG, hallucination cross-checking, model biases, privacy), and a panel of three GenAI researchers validated coverage. Items were then aligned to these concepts. (Yueqiao Jin et al. (2024))

Claims

• AI literacy shows a small positive association with ethical scenario classification accuracy [+M]
• Each additional correct GLAT response is associated with a 13.9% increase in modeled odds of correctly classifying a scenario [+M]
• GenAI literacy predicted immediate independent writing performance after AI removal for Visual Data Integration, Critical Thinking, and overall Composite Score [+M]
• GLAT shows robust structural validity and reliability under a 2-parameter logistic IRT model [+W]
• Perceived and objectively demonstrated AI literacy show only limited alignment in the same samples (pooled r = .055, interval including zero) [~W]

Related Products and Programmes

• Generative AI Student Feedback Literacy Scale (GenAI-SFLS)
• Multi-strategy test (MST)

Key Sources

• Reed, J. M., Ferdig, R. E., Dodson, T. M., Gunstad, J., & Hughes, J. W. (2026). AI Literacy and Scenario-based Ethical Judgment About Generative AI Among Undergraduate Students. Journal of Academic Ethics. https://doi.org/10.1007/s10805-026-09757-w
• Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević and Lixiang Yan. (2025). The Agency Gap in AI-Supported Writing: How Reactive and Proactive Agent Designs Shape Multimodal Reasoning. https://arxiv.org/abs/2507.04398
• Yueqiao Jin, Roberto Martinez-Maldonado, Dragan Gašević and Lixiang Yan. (2024). GLAT: The Generative AI Literacy Assessment Test. https://arxiv.org/abs/2411.00283

Page metadata

Field	Value
type	product
id	generative-ai-literacy-assessment-test-glat
title	Generative AI Literacy Assessment Test (GLAT)
description	A 20-item multiple-choice performance assessment developed by Jin et al.
status	review
product_kind	assessment
generated	by process:settle-candidates on 2026-10-11
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.