Pilot engagement was strong with month-over-month user retention, and teachers used the AI primarily for consumption rather than creation¶

Claim · All claims
Evidence · 2 studies · 2 design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Engagement metrics indicated strong long-term retention at CICS Northtown Academy, with users returning month-over-month. → engagement-1 q2 i? Adoption funnel data showed teachers used the AI primarily for consumption (reading profiles to understand students) rather than creation (logging plans). → engagement-2

Evidence¶

engagement-1¶

Cocuzza, B., Noakes, S., Aso, C., Vollavanh, A., Goldschmidt, L., and Bennett, K. (2026, August). From Data Silos to Student Success: The CICS and Innovare Co-Design Approach to AI-Driven Intervention. Digital Promise. https://doi.org/10.51388/20.500.12265/304

q2 · i? · design · r2

Analysis of an engagement-metrics dashboard Innovare prepared, indicating CICS is actively using the tool and generating measurable adoption data (Figure 5). The report treats this as descriptive usage telemetry, not an outcome evaluation.

"One insight that can be gleaned from the data is that CICS Northtown Academy shows strong long-term retention (users return month-over-month)."

engagement-2¶

Cocuzza, B., Noakes, S., Aso, C., Vollavanh, A., Goldschmidt, L., and Bennett, K. (2026, August). From Data Silos to Student Success: The CICS and Innovare Co-Design Approach to AI-Driven Intervention. Digital Promise. https://doi.org/10.51388/20.500.12265/304

q2 · i? · design · r2

Adoption funnel analysis from the same engagement dashboard (Figure 5). The authors state "This information can lead to further investigation as to why that is the case," leaving the consumption-over-creation pattern unexplained.

"the adoption funnel data in Figure 5 demonstrates that teachers here use the AI primarily for consumption (reading profiles to understand students) rather than creation (logging plans)."

Discussion¶

Related Claims¶

• Students primarily used the Socratic-tutor chatbot as a study aid aligned with existing study habits rather than for the intended Socratic reflection — related
• Chatbot engagement was highly uneven: 8 of 15 voluntary users interacted continuously while 7 interacted superficially — related
• Dialogic competence is a prerequisite for meaningful engagement with LLM-based chatbots — related
• Telegram-based chatbot use declined after initial novelty, with platform friction cited as a barrier — reports the opposite
• Fragmented data systems made intervention planning slow and burdensome, with educators spending 2-3 hours building a single student intervention plan — related
• Primary metrics improved during US holidays, attributed to more motivated voluntary users — related
• Voluntary StatBot use was highly uneven and deadline-driven, with about 27% of students never using the chatbot — related