Role-playing participants prefer LearnLM over GPT-4o but express no substantial preference versus Gemini 1.5 Pro or Claude 3.5 Sonnet¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Participants role-playing as learners preferred LearnLM over GPT-4o in all four comparative assessment categories, strongest for overall pedagogy and similarity to a very good human tutor. → LearnLM Team 2024 q3 i? These same participants indicated no substantial preference between LearnLM and Gemini 1.5 Pro or between LearnLM and Claude 3.5 Sonnet. → LearnLM Team 2024 (2)

Evidence¶

LearnLM Team 2024¶

LearnLM Team, Google. (2024). LearnLM: Improving Gemini for Learning. arXiv. https://arxiv.org/abs/2412.16429 doi:10.48550/arxiv.2412.16429

q3 · i? · design · r2

Appendix A.1 reports preference ratings from the Stage 2 participants who role-played learners, on four comparative categories (Figure 7). The text states these participants "revealed a preference toward LearnLM over GPT-4o for allfour comparative assessment categories", strongest for overall pedagogy and similarity to a quality human tutor.

"The participants role-playing as learners revealed a preference toward LearnLM over GPT-4o for allfour comparative assessment categories (Figure 7)."

LearnLM Team 2024 (2)¶

LearnLM Team, Google. (2024). LearnLM: Improving Gemini for Learning. arXiv. https://arxiv.org/abs/2412.16429

q3 · i? · design · r2

The same Appendix A.1 analysis of role-player preferences reports that "These participants indicated no substantial preference between LearnLM and Gemini 1.5 Pro or between LearnLM and Claude 3.5 Sonnet"; equivalence was not formally tested, so this is an absence of detected preference.

"These participants indicated no substantial preference between LearnLM and Gemini 1.5 Pro or between LearnLM and Claude 3.5 Sonnet."

Discussion¶

Related Claims¶

• Pedagogy experts prefer LearnLM over contemporaneous flagship models across comparative assessment categories — related
• LearnLM receives the highest expert pedagogy-rubric marks across rubric categories, with the largest leads in active learning, metacognition, and curiosity — related
• Preference explanations favoring LearnLM more often cite staying on topic, challenging the learner, and not giving away answers — related
• Role-playing learners report weaker experiences with GPT-4o than with LearnLM, Gemini 1.5 Pro, and Claude 3.5 Sonnet — possibly the same claim (merge candidate)