Expert example standards improve university students' judgment accuracy (bias, absolute accuracy, and relative accuracy) when evaluating self-generated examples¶

Claim · All claims
Evidence · 1 study (3 entries) · causal r2–r3 · q3 · i2 medium
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 Providing expert example standards during self-evaluation significantly reduced bias in judging self-generated examples. → Linda Froese 2023 q3 i2 Expert example standards significantly improved absolute accuracy of self-evaluations of generated examples. → Linda Froese 2023 (2) q3 i2 Expert example standards significantly enhanced relative accuracy (ability to rank examples by quality). → Linda Froese 2023 (3)

Evidence¶

Linda Froese 2023¶

Linda Froese, Julian Roelle. (2023). Expert example but not negative example standards help learners accurately evaluate the quality of self-generated examples. Metacognition and Learning. https://doi.org/10.1007/s11409-023-09347-w

q3 · i2 · causal · r2

In the 2×2 between-subject experiment (N = 128, one outlier excluded), an ANCOVA controlling for prior knowledge showed a significant main effect of expert example standards on bias, "F(1, 122) = 5.35, p = .022, ηp2 = .04", a medium-sized effect.

"We found a statistically significant main effect of expert example standards, F(1, 122) = 5.35, p = .022, ηp2 = .04, indicating that the learners who received expert example standards showed lower bias than the learners without expert example standards."

Linda Froese 2023 (2)¶

Linda Froese, Julian Roelle. (2023). Expert example but not negative example standards help learners accurately evaluate the quality of self-generated examples. Metacognition and Learning. https://doi.org/10.1007/s11409-023-09347-w

q3 · i2 · causal · r3

The same experiment showed a significant main effect of expert example standards on absolute accuracy, "F(1, 122) = 11.42, p = .001, ηp2 = .08", a medium effect; learners with expert standards reached values closer to zero.

"Like for bias and confirming Hypothesis 1a, we found a statistically significant main effect of expert example standards, F(1, 122) = 11.42, p = .001, ηp2 = .08."

Linda Froese 2023 (3)¶

Linda Froese, Julian Roelle. (2023). Expert example but not negative example standards help learners accurately evaluate the quality of self-generated examples. Metacognition and Learning. https://doi.org/10.1007/s11409-023-09347-w

q3 · i2 · causal · r3

For relative accuracy (intra-individual gamma correlations; n = 8 participants excluded due to indeterminate gammas), expert example standards significantly enhanced ranking accuracy, "F(1, 114) = 4.82, p = .030, ηp2 = .04", a medium-sized effect.

"In line with Hypothesis 1a, we found that expert example standards enhanced relative accuracy, F(1, 114) = 4.82, p = .030, ηp2 = .04."

Discussion¶

Related Claims¶

• Example quality does not significantly moderate the effects of the standards on judgment accuracy when aggregated across concepts — related
• Providing expert example standards enhances absolute judgment accuracy in evaluating self-generated examples — related
• Idea unit standards do not enhance judgment accuracy in evaluating self-generated examples — related
• Negative example standards do not reliably improve judgment accuracy and significantly decrease relative accuracy in evaluating self-generated examples — related
• Providing expert example standards reduces bias in learners' evaluations of self-generated examples — a narrower finding that bears on this claim
• Benefits of expert example standards depend on example quality: largest for incorrect examples, absent for correct ones — a narrower finding that bears on this claim
• Providing a rubric reduces bias in self-assessment, moving judgments closer to expert ratings — related
• Expert example standards did not significantly enhance relative accuracy in evaluating self-generated examples — reports the opposite
• ChatGPT use does not change absolute self-evaluation accuracy, but self-evaluation judgments fail to correlate with performance under ChatGPT assistance — related