Trust in personalized analogies moved in no single direction, while critical scrutiny tracked domain expertise more consistently than personal interest¶

Claim · All claims
Evidence · 2 studies · 2 qualitative r3 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Some participants trusted tailored analogies more while others scrutinized them more closely or distrusted the tailoring itself. → trust-mixed q2 i? What tracked scrutiny across participants was domain expertise rather than personal interest per se. → expertise-scrutiny

Evidence¶

trust-mixed¶

Seth Bernstein and Naaz Sibia. 2026. Flawed but Memorable: Student Critical Reception of Interest-Personalized GenAI Analogies in Computing Education. Koli Calling '26. https://arxiv.org/abs/2609.06095 doi:10.48550/arxiv.2609.06095

q2 · i? · qualitative · r3

RQ3 interview accounts in the ten-participant qualitative study. The article reports the pattern was not uniform: P1 became less critical of tailored analogies while P7 trusted them less and shared the bare minimum about herself.

"P1 reported exactly the reduction in scrutiny our framework anticipated, saying that knowing the analogy was tailored to them left them "less critical about it because I could trust it more." But P7 reported the opposite association between personal relevance and trust, treating heavy personalization as a reason for suspicion rather than comfort"

expertise-scrutiny¶

Seth Bernstein and Naaz Sibia. 2026. Flawed but Memorable: Student Critical Reception of Interest-Personalized GenAI Analogies in Computing Education. Koli Calling '26. https://arxiv.org/abs/2609.06095

q2 · i? · qualitative · r3

The article reports that domain expertise, not personal interest, consistently tracked scrutiny, quoting P9 that knowing the concepts let her tell when an analogy went wrong without increasing trust. The authors treat this as an emergent qualitative interpretation, not a causal effect.

"What tracked scrutiny consistently, across participants, was domain expertise rather than personal interest per se: P9 noted that their ability to audit an analogy came from prior knowledge of the target concept, not affection for the source domain"

Discussion¶

Learner Variables¶

• Prior Knowledge — moderator: an instructional effect differs with it

Related Claims¶

• Prior Knowledge Needed For Accurate Self Assessment — the general claim this finding bears on, with the studies behind it
• Most participants described interest-personalized GenAI analogies as more engaging or memorable than generic technical explanations — related
• Students describe concrete verification behaviors for GenAI explanations, treating evaluation as an iterative filter rather than a single accept-or-reject judgment — related
• Undergraduate computing students evaluate GenAI analogies by structural correspondence between source and target domain, flagging specific mapping breakdowns rather than judging holistically — related
• Students evaluate GenAI explanations for assumed background and audience, reading unfamiliar cultural references and excessive scaffolding as signals about who the explanation was written for — related
• Students described the verification label as a reason to engage with AI-generated material that did not end their own scrutiny of it — related
• Open-ended student feedback highlighted relevance and conciseness as benefits of personalized AI videos, alongside concerns about unnaturalness, uncanny-valley discomfort, and trust — related