Stronger problem-solving capability predicts lower misconception faithfulness: flip rate under generic feedback correlates negatively with SFS (Pearson r = -0.94, p = 0.002)¶

Claim · All claims
Evidence · 1 study · causal r2 · q2 · i3 large
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i3 Across the seven evaluated models, capability proxied by flip rate under generic feedback (FG) is strongly negatively correlated with SFS, with reasoning-augmented models showing higher flip rates but lower sensitivity. → Heejin Do 2026

Evidence¶

Heejin Do 2026¶

Heejin Do, Shashank Sonkar, Mrinmaya Sachan. (2026). Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators. Preprint. https://arxiv.org/abs/2605.12748 doi:10.48550/arxiv.2605.12748

q2 · i3 · causal · r2

Correlational analysis across the seven evaluated LLM simulators, averaged over both datasets and prompting strategies; the figure reports Pearson r = -0.94 (p = 0.002), and the text states "Models that can solve the problem without guidance exhibit minimal sensitivity."

"Figure 3(b)shows a strong negative correlation between model capability—proxied by the flip rate under generic feedback (FG)—and SFS. Models that can solve the problem without guidance exhibit minimal sensitivity, collapsing all feedback conditions into uniformly high flip rates."

Discussion¶

Learner Variables¶

• General Cognitive Ability — predictor: learners who differ on it differ in outcomes

Related Claims¶

• LLM student simulators exhibit near-zero Selective Flip Score, flipping at similarly high rates under targeted, misaligned, and generic feedback — related
• Sycophantic failure manifests as indiscriminate correct flips rather than explicit sycophantic agreement, and SFT shifts behavior toward pushback and maintain under non-targeted feedback — related
• Multi-turn reflective interaction does not restore selective updating and destabilizes weaker simulators — related
• Simulating higher-cognitive-ability students is easier: simulation quality correlates positively with student cognitive score — related
• The competence paradox: broadly capable LLMs cannot genuinely unknow expert knowledge when simulating novices — related
• Knowledge, Skills, and Attitude scores are largely independent across frontier LLMs, with Knowledge–Skills correlation r=−0.51 and Knowledge–Attitude r=−0.63 — related