Retrospective pre/post confidence rose on all four course learning objectives with large rank-biserial effects (N=159)¶

Claim · All claims
Evidence · 1 study · associational r1 · q2 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i3 On all four ME 373 learning objectives, retrospective pre/post confidence rose by more than one Likert point, with Mann–Whitney U p < 10−27 and rank-biserial effect sizes rrb from 0.688 (PDEs) to 0.825 (BVPs). → Jesse A. Rodríguez 2026

Evidence¶

Jesse A. Rodríguez 2026¶

Jesse A. Rodríguez. (2026). LaTA: A Drop-In, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework. arXiv preprint. https://arxiv.org/abs/2605.05410 doi:10.48550/arxiv.2605.05410

q2 · i3 · associational · r1

Anonymous post-term Canvas survey of N=159 students (~80% response rate) using a retrospective pre/then-post design; pre/post distributions treated as independent for Mann–Whitney U testing. Printed deltas were +1.57 to +1.88 Likert points.

"All four comparisons achieved Mann–Whitney U p <10 −27 uncorrected (well below the threshold any standard multiple-comparisons correction would impose at q= 0.05 across only four tests) with rank-biserial effect sizes rrb ranging from 0.688 (PDEs) to 0.825 (BVPs), conventionally "large" effects."

Discussion¶

Related Claims¶

• In an ARC trial deployment, undergraduate mentors' retrospective pre-then-post ratings of teaching confidence and community connection increased substantially — related
• A half-day workshop featuring the PPH framework significantly increased dietitians' confidence to undertake pro-environmental change in their personal and professional lives — related
• Faculty confidence in guiding student AI use rose markedly after the six-week institute, with high-confidence responses increasing from 0% to 60% — related
• Paired pre/post AI readiness scores rose descriptively from 82.5% to 89.9% (mean gain 7.4 percentage points) among seven co-creation clients, treated as descriptive context only — related
• The between-cohort exam gain is attributable only to a composite intervention (autograding plus corrections plus expanded office hours), not to LaTA alone — related
• Students' Domain interest increased largely across a short introductory teaching course (Cohen's d = 2.07). — related