Both pedagogy-expertise-weighted and unanimous-vote ensembles frequently worsen LLM alignment with student learning¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Neither benchmark-weighted ensembling nor unanimous-vote consensus improves alignment with value-added learning; both frequently worsen it on core instructional dimensions. → Michael Hardy 2026

Evidence¶

Michael Hardy 2026¶

Michael Hardy, Yunsung Kim. (2026). Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact. arXiv. https://arxiv.org/abs/2603.00883 doi:10.48550/arxiv.2603.00883

q2 · i? · causal · r2

Ensemble evaluation in Sec. 5.3 comparing pedagogy-expertise-weighted ensembles and unanimous-vote ensembles on all model-prompt combinations (Fig. 4). Worsening was "especially on core instructional dimensions such as remediation of student errors and behavior management."

"Neither strategy improves alignment with stu- dent learning. Instead, both frequentlyworsen"

Discussion¶

Related Claims¶

• LLM accuracy on the CDPK pedagogy benchmark ranges from 28% to 89%, with closed-weight reasoning models dominating the top of the leaderboard — related
• Rank-averaged ensembling stabilized predictions but did not outperform the single base model — related
• A stacking ensemble of LG, SVM, NB, KNN and XGB achieved the best fine-tuning F1-score (0.88), outperforming individual models such as SVM (0.86) — related
• Ensemble modeling of multiple LLM scorers yields performance gains in automated divergent thinking scoring — reports the opposite
• Majority-vote ensembling across three runs produced only modest accuracy gains (1% to 5%) with no model showing degradation — reports the opposite
• Ensembles of operational raters outperform the LLMs: two raters suffice on average across aspects, but larger ensembles are needed for individual aspects — related
• Current GenAI tools are not designed to support active learning and their apparent domain expertise is brittle — related
• Model self-confidence, weighted probabilistic scoring, and ensemble modeling each significantly improve LLM-based divergent thinking scoring, raising correlation with human judges from r = 0.781 to r = 0.823 — related
• Coaching effects on instruction and achievement are positively correlated, but large instructional improvements are needed for modest achievement gains — related