Undergraduate computing students evaluate GenAI analogies by structural correspondence between source and target domain, flagging specific mapping breakdowns rather than judging holistically¶

Claim · All claims
Evidence · 1 study · qualitative r3 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Students with sufficient domain expertise identified structural mismatches between an analogy's source domain and the computing concept, holding analogies to whether their structure matched the concept. → structural-audit

Evidence¶

structural-audit¶

Seth Bernstein and Naaz Sibia. 2026. Flawed but Memorable: Student Critical Reception of Interest-Personalized GenAI Analogies in Computing Education. Koli Calling '26. https://arxiv.org/abs/2609.06095 doi:10.48550/arxiv.2609.06095

q2 · i? · qualitative · r3

Reflexive thematic analysis of think-aloud and interview data from ten CS2-completing participants evaluating personalized and generic explanations. The article reports that a recurring failure was "a structural mismatch between the analogy and the real-world domain it borrowed from"; e.g., P6 corrected a One Piece linked-list analogy as implying a circular list.

"A recurring failure participants identified was a structural mismatch between the analogy and the real-world domain it borrowed from, echoing Halasz and Moran's [25] concern that analogical models are only ever partially structure-preserving."

Discussion¶

Related Claims¶

• Students evaluate GenAI explanations for assumed background and audience, reading unfamiliar cultural references and excessive scaffolding as signals about who the explanation was written for — related
• Most participants described interest-personalized GenAI analogies as more engaging or memorable than generic technical explanations — related
• Students apply precision-of-language as a distinct evaluation criterion, flagging hedging phrases as open to interpretation and proposing concrete substitutions — related
• CS/SE educators rate ANVIL analogies highly and animations as generally faithful, with disagreement concentrated in borderline and visual-clarity judgments — related
• Trust in personalized analogies moved in no single direction, while critical scrutiny tracked domain expertise more consistently than personal interest — related
• Educators identify a risk of pedagogical mismatch when generated animations fail to preserve key constraints of the target concept — related
• Concrete, outcome-oriented eco-feedback visualizations with real-world analogies are rated more favorably than abstract metrics — related