Conduct ed-tech pilots by articulating the instructional need, setting clear product-review criteria, involving educators in planning, supporting educators throughout, gathering data from all users, deciding from data, negotiating purchasing options, and sharing results¶

Strategy · All strategies
Evidence · no claims cited

Description¶

This strategy packages the eight best practices Digital Promise identified for district ed-tech pilots: articulate the instructional need, use clear criteria to review products, involve educators in planning, support educators throughout, gather data from all users, make decisions based on data, discuss purchasing options with developers, and share results. The article states, "we identified the following best practices for conducting ed-tech pilots," and recommends districts "use these recommendations to generate meaningful evidence to support decision making." Each practice is illustrated with a district example, such as Mineola starting small to verify infrastructure.

Design Implications¶

Context¶

Requirements¶

• Districts must first articulate the specific instructional problem they hope the ed-tech product will solve, so pilot evidence can be judged against that need.
• Cooperation and communication among educators, school leaders, product companies, and researchers is essential to ensure common goals and expectations.

Constraints¶

• Large pilots are harder to support: in Rowan Salisbury it was difficult to gather data and support all teachers, and implementation varied significantly between schools, making district-wide conclusions challenging.
• Districts should discuss product cost at the beginning of a pilot; one district found the cost of Mathspace likely prohibitive only after agreeing to pilot it.

Target Learners¶

• K-12 middle school students in district ed-tech pilots

Target Learning Goals¶

• Middle school mathematics skills, including ratios and proportions

Claims¶

Related Strategies¶

• Build formal feedback mechanisms and an evaluation plan into ed-tech pilots
• Maintain professional development and educator collaboration throughout an edtech pilot
• Ensure communication and co-development with school leaders throughout edtech process rollout
• Ed-tech companies should use multi-method, multi-stakeholder research at every product stage and share results publicly
• Distinguish product users from purchasers and recruit representative users through school-like community settings
• Evaluate & Reflect then Sustain & Scale: analyze pilot data against targeted goals and share results with critical partners to plan expanded use
• Start simple: relationships-first, low-tech student success systems can suffice in small, tight-knit communities
• Mobilize & Implement: provide professional learning throughout the pilot and use real-time data dashboards to keep usage aligned with dosage requirements
• Provide professional learning opportunities and free tools to support educators in conducting edtech pilot studies
• Five-step iterative co-design process for developing edtech evaluation criteria with learners and vendors
• Five recommendations for higher education systems elevating student and faculty voice in edtech evaluation
• Adopt AI incrementally in waves from simple to complex tasks, on free-tier tools, with mandatory multi-step human review before any document is issued

Examples¶

-

Key Sources¶

• Luke, Christina C., Ph.D. & Francisco, Aubrey T., Ph.D. (2016). Rapid Cycle Pilots: Lessons Learned from Math Trials in Six Districts. Digital Promise. https://digitalpromise.dspacedirect.org/items/3d9bb6ec-6ef2-4b64-a130-ce9c19574443