In a retrospective higher-education evaluation, instructors using TEA found low CIC values and unbalanced True Weights in three of the ten tests and marked them for further analysis¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2021 · checked against its sources 2026-10-28 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In the retrospective analysis of the Fundamentals of Computer Science module, most tests showed adequate CIC and homogeneous True Weights, but three tests showed inconsistencies and were marked for thorough analysis. → Josu Arruarte 2021
Evidence¶
Josu Arruarte 2021¶
Josu Arruarte, Mikel Larrañaga, Ana Arruarte & Jon A. Elorriaga. (2021). Measuring the Quality of Test-based Exercises Based on the Performance of Students. International Journal of Artificial Intelligence in Education, 31, 585–602. https://doi.org/10.1007/s40593-020-00208-0
q2 · i? · design · r2
Retrospective evaluation in which instructors used TEA on Moodle-recorded answers from 50 students across 10 computer-session tests of 5 items each, with final grades as ground truth. The article reports "inconsistencies, i.e., low CIC values and unbalanced True Weights of the items, were observed in three of the tests." No effect sizes are printed.
"Most of the tests presented adequate values for the Coefficient of Internal Consistency (CIC), which determines how consistent the answers of the students to the test items are. In addition, most of the tests presented quite homogeneous True Weights in their items. However, inconsistencies, i.e., low CIC values and unbalanced True Weights of the items, were observed in three of the tests."
Discussion¶
Related Claims¶
- Boxplot visualizations enriched with correlation to final marks identified the same suspicious tests as the consistency metrics, and instructors modified two of them for the next academic year — a narrower finding that bears on this claim
- Item-level visualizations of discrimination index and discrimination efficiency led instructors to identify troublesome items and replace or modify them — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
tea-retrospective-inconsistent-tests-three |
title |
In a retrospective higher-education evaluation, instructors using TEA found low CIC values and unbalanced True Weights in three of the ten tests and marked them for further analysis |
description |
In a retrospective higher-education evaluation, instructors using TEA found low CIC values and unbalanced True Weights in three of the ten tests and marked them for further analysis |
status |
review |
evidence_strength |
weak |
newest_evidence |
2021 |
source_checked |
2026-10-28 |
generated |
by process:wiki-ingest on 2026-10-28 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — josu-arruarte-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.