After the design iteration, no student deleted an evaluation step, in contrast to frequent step deletion in the first deployment¶

Claim · All claims
Evidence · 1 study · design r3 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In the second deployment no student ever specifically deleted an evaluation step, whereas 33%, 50%, and 0% of students did so across the first deployment's inductions. → Matthew Keenan 2026

Evidence¶

Matthew Keenan 2026¶

Matthew Keenan, Nishant Kheterpal, Jean-Baptiste Jeannin, Cyrus Omar. (2026). Hazel Prover: A Classroom Proof Assistant for Learning Structural Induction. https://arxiv.org/abs/2608.23309 doi:10.48550/arxiv.2608.23309

q2 · i? · design · r3

Log-replay comparison of evaluation-stepping behavior between deployments (Table 1(c) and 1(d)). The authors attribute the change to the greater work required to create a step, after which students were clearly more confident in their written step; no effect size is printed.

"By complete contrast, in the second deployment, no student ever specifically deleted an evaluation step (Table 1(d))."

Discussion¶

Related Claims¶

• Student backtracking before setting up the correct induction decreased across exercises in both deployments, indicating a learning curve — related
• Requiring students to write out evaluation steps manually improved transfer to pen-and-paper exam performance between the two deployments — related
• Novice designers deleted ideas only when confronted with stakeholder friction; unsupported self-reflection produced no deletions — related
• Students backtracked far more before applying the inductive hypothesis than after, suggesting difficulty finding where to place it — related
• In the first deployment, exam scores on the induction question were visibly lower than in previous semesters taught without the tool — related