Ethnographically-oriented methods surface five dimensions of classroom context that text-only transcripts omit, including multimodal communication and relationships¶

Claim · All claims
Evidence · 1 study (2 entries) · qualitative r3 · q2
Currency · no dated study recorded · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Participant observations, interviews, and focus groups recovered contextual dimensions—environment, lesson structure, multimodal communication, language ideologies, and relationships—that transcripts lose. → Liliana Santos-Deonizio 2027 q2 i? Coding schemes built around spoken academic talk risk mischaracterizing differences in preferred mode of engagement as gaps in ability or attention. → Liliana Santos-Deonizio 2027 (2)

Evidence¶

Liliana Santos-Deonizio 2027¶

Liliana Santos-Deonizio, James Malamut, Ramón Antonio Martínez, and Dorottya Demszky. (2027). When Youth Enter The Chat: An Epistemic Shift in the Validation of LLM-Based Measures of Student Talk. In Proceedings of TBD (CSCW '27). ACM, New York, NY, USA. https://arxiv.org/abs/2608.23780 doi:10.48550/arxiv.2608.23780

q2 · i? · qualitative · r3

Qualitative case study in one 8th-grade math classroom using seven participant observations, four interviews, two focus groups, and four member checks with four focal students; field notes were thematically grouped into contextual factors LLM-based measures miss.

"By using ethnographically-oriented methods, we recover much of what transcripts lose by treating student talk as a situated, embodied, social, and cultural experience."

Liliana Santos-Deonizio 2027 (2)¶

Liliana Santos-Deonizio, James Malamut, Ramón Antonio Martínez, and Dorottya Demszky. (2027). When Youth Enter The Chat: An Epistemic Shift in the Validation of LLM-Based Measures of Student Talk. In Proceedings of TBD (CSCW '27). ACM, New York, NY, USA. https://arxiv.org/abs/2608.23780

q2 · i? · qualitative · r3

Observation-based analysis contrasting two students: Carlos rarely spoke but worked diligently, while Drake was charismatic and frequently contributed; interpreted only through spoken-talk codes they look like opposite ends of a participation scale, though the difference may reflect preferred mode of engagement.

"Coding schemes built around spoken academic talk risk mischaracterizing those differences as gaps in ability or attention."

Discussion¶

Related Claims¶

• Recall-oriented anchor selection achieved a 17.93% anchor rate and 94.63% non-empty OCR rate, which the authors interpret as anchors frequently capturing visible educational content that transcript chunks omit — related
• During member checks, multilingual 8th-grade students contested both LLM classifications of their talk and the coding scheme itself, especially the Off-task and Claim codes — related
• Classroom implementation of the GoldList Method encountered pitfalls: forgotten notebooks, omitted dates, and unclear initial steps — related
• Contemporary SRL models largely omit the broader social context and implicitly assume all learners are available to learn — related