CTAT shows robust psychometric properties for higher education CS/AI students, with reliable internal consistency and adequate IRT item parameters¶

Claim · All claims
Evidence · 1 study (2 entries) · design r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? CTAT demonstrated reliable internal consistency (Cronbach's alpha 0.849) in a field test with 461 higher education students. → Shile Zhang and Shuhan Zhang 2026 q3 i? Under the 3PL1p IRT model, CTAT items showed adequate difficulty, discrimination above 0.5, and guessing at or below 0.35. → Shile Zhang and Shuhan Zhang 2026 (2)

Evidence¶

Shile Zhang and Shuhan Zhang 2026¶

Shile Zhang and Shuhan Zhang. (2026). Integrating AI into computational thinking: development and validation of an assessment tool for higher education students. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00623-0

q3 · i? · design · r3

Main-study field test of CTAT with 461 CS/AI students in two colleges and two vocational schools in northern China. The article reports Cronbach's alpha of "0.849", exceeding the 0.7 threshold, indicating reliable performance.

"Internal consistency was examined using Cronbach's alpha, which yielded a value of 0.849. This result exceeds the recommended threshold of 0.7 (Kline, 2023), indicating that the scale demonstrated reliable performance."

Shile Zhang and Shuhan Zhang 2026 (2)¶

Shile Zhang and Shuhan Zhang. (2026). Integrating AI into computational thinking: development and validation of an assessment tool for higher education students. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00623-0

q3 · i? · design · r3

IRT item-parameter analysis of the main-study data using the 3PL1p model. Mean difficulty was 0.634, mean discrimination 2.412 with all items above the 0.5 threshold, and mean guessing parameter 0.221, indicating adequate item parameters.

"CTAT had a mean difficulty of 0.634 (range -0.887 to 2.181), showing reasonable difficulty with broad coverage. The mean discrimination was 2.412 (range 0.503 to 5.609), all items exceeded the 0.5 threshold"

Discussion¶

Related Claims¶

• College students scored significantly higher on CTAT than vocational school students — related
• The AEEGS software demonstrated internal consistency of Cronbach alpha 0.86 in trial testing and was validated for face and content validity by three software experts — related
• Exploratory factor analysis of survey items yields a unidimensional 9-item AI pedagogical orientation scale with strong reliability — related
• The 16-item CT–PCK Survey shows internal consistency reliability around .71–.72 and person separation reliability of .69 — related
• A 12-item short form of the survey retains acceptable-to-good reliability (alpha = 0.85; test-retest = 0.75) — related
• Questionnaire constructs showed acceptable to strong internal consistency (α = 0.740 to 0.945) — related
• TAILS dimensions show high internal consistency, with Cronbach's alpha values above 0.90 — related
• The post-interaction reflection instrument showed satisfactory internal consistency (Cronbach's α = .863) — related
• A high Cronbach's alpha across items measuring distinct knowledge aspects indicates the test is not working as intended — related
• The MAKQ dimensions show acceptable internal consistency across both studies (alphas 0.77-0.93) — related
• Meta-analytic average Cronbach's Alpha of domain-specific knowledge tests is 0.85 with expected item intercorrelation of 0.22 — related
• The ten-construct instrument showed strong psychometric quality, with the four ARCS dimensions forming a psychometrically sound higher-order motivation construct — related
• CSEBench items show adequate psychometric quality, with satisfactory difficulty, discrimination, and reliability for both diagnostic forms — related