Self Assessment Accuracy Is Low Without Training¶

Claim · All claims
Evidence · 2 studies · 1 causal r?, 1 quant-synthesis r? · q3–q4 · i2 medium
Currency · newest study 2012 · 1 of 2 entries checked against their sources (automated) · not reviewed by a person

Untrained learners systematically misjudge their own performance — typically overestimating it — and self-assessment only becomes reasonably accurate after explicit training, practice with feedback, and access to clear criteria.

Subclaims¶

q3 i2 In a randomized 2×2 experiment, secondary-education students who studied worked modeling examples of self-assessment produced significantly more accurate self-assessments of their own problem-solving performance than students who had not, a medium-sized effect. → Kostons et al. 2012

q4 i? Across a meta-analysis of 51 quantitative studies, the correspondence between students' self-assessed marks and teachers' marks in higher education was inconsistent, and was closer in better-designed studies, in more advanced (vs. introductory) courses, and in science compared to other fields — i.e., untrained self-assessment accuracy is not a stable baseline but depends heavily on experience and assessment design. → Falchikov & Boud 1989

Evidence¶

Kostons et al. 2012¶

Kostons, D., van Gog, T., & Paas, F. (2012). Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning. Learning and Instruction, 22(2), 121–132. doi:10.1016/j.learninstruc.2011.08.004

q3 · randomized experiment (not pre-registered) · i2 · medium effect, ηp²=.10 · n=80 (secondary-education students, Experiment 1) · causal · r?

80 Dutch fourth-year pre-university students (age M=15.23) were randomly assigned to one of four conditions in a 2×2 design (self-assessment modeling examples: yes/no; task-selection modeling examples: yes/no) and solved genetics problems before and after studying modeling examples. Self-assessment accuracy was scored as the absolute difference between each student's self-rated performance and their actual (objectively scored) performance on the posttest, so a lower score means more accurate self-assessment. Students who had observed a human model demonstrating self-assessment were significantly more accurate (M=.81) than those who had not (M=1.23), F(1,76)=8.04, p=.006, ηp²=.10 — direct experimental evidence that untrained self-assessment is measurably worse than trained self-assessment on the same task.

Falchikov & Boud 1989¶

Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research, 59(4), 395. doi:10.2307/1170205

q4 · meta-analysis of 51 studies · i? · no pooled effect size in the abstract · n=51 studies · quant-synthesis · r?

A meta-analysis of 51 quantitative studies comparing students' self-assessed marks with teachers' marks in higher education. Correspondence between the two was not uniformly high: it was closer in better-designed studies than in poorly designed ones, closer for students in advanced courses than in introductory courses, and closer in science than in other fields of study — i.e., raw self-assessment accuracy varies widely and cannot be assumed without attention to design and learner experience. Read at abstract level only (full text is paywalled/JSTOR-gated); no pooled correlation or effect-size statistic appears in the abstract, so no quantitative i code beyond i? can be supported from what was read.

Studies bearing on this claim¶

Can learners' own monitoring be trusted, and can it be trained? This table gathers the studies behind this claim and the 25 narrower claims that bear on it: 25 distinct studies (14 causal, 4 review, 3 quant-synthesis, 3 associational, 1 theoretical), q1–q4, 8 of 25 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Direction of the bias. The dominant finding in the self-assessment literature is overestimation: low performers lack the domain knowledge to recognize their own errors, a pattern closely related to the Dunning–Kruger effect. Weaker students show the largest miscalibration, while stronger students are sometimes slightly underconfident. This means raw self-assessment data cannot be treated as a valid proxy for achievement, especially in early stages of learning.

What training changes. Accuracy improves when learners are given explicit evaluation criteria (rubrics, exemplars of different quality levels), opportunities to compare their judgments against expert or peer judgments, and iterative feedback on the accuracy of their self-ratings themselves — not just on the work. Self-assessment embedded in assessment for learning cycles, where judgments feed directly into revision, tends to be more accurate than one-off self-grading.

Boundary conditions. Self-assessment accuracy is domain- and task-dependent: learners calibrate better on well-structured tasks with unambiguous correctness than on open-ended work like essays or design projects. It also develops with expertise — the same expertise reversal dynamics seen in scaffolding research suggest that supports needed for accurate novice self-assessment (detailed rubrics, guided checklists) should be faded as learners gain experience.

Relation to self-regulated learning. Accurate self-monitoring is a core component of self-regulated learning: learners who misjudge their performance select inappropriate study strategies and terminate practice prematurely. Training calibration is therefore not merely an assessment concern but a prerequisite for effective self-regulated study decisions.

Design implication. The claim is a caution, not a prohibition: untrained self-assessment is unreliable as a measure, but the act of trained self-assessment can still benefit learning by directing attention to criteria and gaps. Designers should not ask learners to self-assess without first teaching them how, and should treat self-ratings from novices as noisy data requiring triangulation with assessment evidence.

Open questions. Two studies are recorded: one randomized experiment in which secondary students trained with modeling examples self-assessed more accurately than untrained students (Kostons et al. 2012, n = 80), and one meta-analysis of 51 higher-education studies, read at abstract level, which found student–teacher agreement inconsistent and closer in advanced courses and better-designed studies but reports no pooled accuracy estimate there (Falchikov & Boud 1989). Still missing: a pooled estimate of how low untrained accuracy is, and replications of the training effect beyond one experiment.

Related Claims¶

• In two secondary-school writing studies, peer reviewers' own performance predicted how helpful their feedback was but not how accurately it identified problems — the general claim this finding bears on, with the studies behind it

• Self Assessment Improves Self Regulated Learning — the general claim this finding bears on, with the studies behind it

• Assessment for learning improves achievement. — feedback-rich assessment cycles are the context in which self-assessment accuracy develops

• Self-Regulated Learning — accurate self-monitoring is a core component of self-regulation
• Expertise Reversal Effect — self-assessment supports needed by novices should be faded as expertise grows
• Prior Knowledge Needed For Accurate Self Assessment — related
• Metacognitive Strategies Improve Learning — a broader claim this one bears on
• Self-regulated learning improves achievement — a broader claim this one bears on
• Learners who could decide after a trial whether to receive knowledge of results estimated their own movement outcomes more accurately in retention than Self-Before learners and their yoked counterparts. — related
• Sequencing worked examples with practice problems improves learning for novices — related
• A theoretical article, citing two simulation studies, reports that novice clinical learners overestimated their performance and that their confidence exceeded their accuracy — a narrower finding that bears on this claim
• Students' expertise on the task may be an important prerequisite for accurate self-assessment — related
• ChatGPT use does not change absolute self-evaluation accuracy, but self-evaluation judgments fail to correlate with performance under ChatGPT assistance — related
• Poor mathematics performers are less intrinsically motivated, less metacognitively accurate, and overestimate their performances more often than well-performing peers in all grades — related
• Secondary school students overestimate their performance on learning tasks by about one step out of five — related
• Students lack metacognitive awareness of the benefits of errorful learning: the three learning groups did not differ in perceived competence, judgments of learning, or rated method effectiveness even after tests — related
• Students with good-quality or moderately positive profiles self-assessed more accurately after studying video modeling examples than students with poor-quality or moderately negative profiles — related
• The benefits of rubrics on self-assessment accuracy do not depend on the inferential level of the assessment criteria — related
• SRSD instruction improved students' accuracy in self-evaluating their texts' quality, although overestimation persisted — related
• The tribute reports students tend to overestimate their sport performance, and four-level model instruction may improve calibration accuracy — a narrower finding that bears on this claim
• Directly after training, students who observed a modeling example referred to assessment criteria in their feedback significantly more often than students who practiced — related
• Across both experiments, self-assessment feedback failed to improve self-assessment accuracy in the absence of feedback, and learners' self-assessments did not track practice-driven performance gains — reports the opposite
• Video modeling training of self-assessment and task-selection skills improves secondary students' problem-solving performance after a self-regulated learning phase — related
• Underuse of effective strategies may stem from false beliefs, lack of awareness, and the counter-intuitive nature of the strategies — related
• Solving an isomorphic practice problem after studying a worked example improves prospective monitoring accuracy, with mixed effects on regulation accuracy — related
• Model answers combined with self-assessment instructions interactively improve monitoring-based regulation accuracy, with two model answers most effective — related