Build formal feedback mechanisms and an evaluation plan into ed-tech pilots¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The report recommends that districts treat student and teacher feedback as a formal component of pilot evaluation rather than an informal afterthought, and that they plan evaluation designs in advance when assessment outcomes are the success benchmark. It states that "Pilots should include formal mechanisms for collecting student and teacher feedback" and that when assessment outcomes benchmark success, "districts should develop an evaluation plan and research design to validly measure the impact of product use on student outcomes." Administrators should integrate this feedback into pilot evaluation.

Design Implications¶

Context¶

Requirements¶

• Districts must create formal collection mechanisms for student and teacher feedback rather than relying on informal conversation.

Constraints¶

• The recommendations are offered as items to consider, not a prescribed pilot process; valid measurement of product impact requires an evaluation plan and research design developed in advance.

Target Learners¶

• K-12 students in districts piloting education technology products

Target Learning Goals¶

• Informed procurement decisions about education technology products based on evidence of product effectiveness

Related Strategies¶

• Distinguish learner research from user research and choose success metrics tied to educational outcomes rather than engagement
• Establish a national Ed-tech Product Information Exchange website
• Conduct ed-tech pilots by articulating the instructional need, setting clear product-review criteria, involving educators in planning, supporting educators throughout, gathering data from all users, deciding from data, negotiating purchasing options, and sharing results
• Provide professional learning opportunities and free tools to support educators in conducting edtech pilot studies

Examples¶

-

Key Sources¶

• Adams-Bass, V., Atchison, D., & Moore, L. (2015). Piloting Ed-tech Products in K-12 Public Schools: Pilot-to-Purchase Project. A Report from the University of California Davis School of Education to Digital Promise. https://digitalpromise.dspacedirect.org/items/38af8409-82b3-4f0e-baff-2a093963bada