A research-grounded prompt revision improved scores for 10 of 11 models, with gains of 6.6 to 16.2 percentage points¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Moving from the initial prompt (Trial 1) to the revised prompt (Trial 2) improved 10 of 11 models by 6.6–16.2 percentage points (mean 11.2); qwen3-coder:30b was the sole exception, decreasing by 1.5 points. → H. Chad Lane 2026

Evidence¶

H. Chad Lane 2026¶

H. Chad Lane, Bryson Kageler. (2026). CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming. SLM4ED'26: The 1st Workshop of Small Language Models for Education. https://github.com/InviteInstitute/CSTutorBench doi:10.1145/3770761.3777333

q2 · i? · design · r2

Trial 1 vs. Trial 2 comparison of overall scores for 11 models on the same 17 benchmark questions. The revision primarily improved the type-specific criteria along with tone, formatting, and targetedness, while accuracy, actionability, and vocabulary saw minimal change. No effect sizes are printed.

"10 of 11 models improved from Trial 1 to Trial 2, with gains ranging from 6.6 to 16.2 percentage points (mean: 11.2); qwen3-coder:30b was the sole exception, decreasing by 1.5 points."

Discussion¶

Related Claims¶

• Current LLMs fall short of human experts in predicting classroom intervention outcomes — related
• Model family and instruction-tuning approach appear better predictors of tutoring quality than parameter count alone — related
• Revising the feedback prompt toward essay-grounded guidance was associated with more specific feedback in an observational before–after comparison — related