No readily accessible sources of rigorous evidence exist for most ed-tech products¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2019: baselines and conditions change
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on an ERIC document (a report, conference paper or thesis). It is kept because a design page cites it; treat it as a lead to check, not an established finding.

Subclaims¶

q2 i? There are no readily accessible sources of rigorous evidence on the effectiveness of the vast majority of ed-tech products, so districts largely depend on recommendations from peers and their own teachers and principals. → Morrison 2014 q2 i? Local evidence frequently comes from informal pilot studies whose conclusions about product quality may be questionable. → Morrison 2014

Evidence¶

Morrison 2014¶

Morrison, J. R., Ross, S. M., Corcoran, R. P., & Reid, A. J. (2014). Fostering Market Efficiency in K-12 Ed-tech Procurement. Center for Research and Reform in Education, Johns Hopkins University. https://eric.ed.gov/?id=ED563768

q2 · i? · associational · r1

Survey and interview findings from district participants. The report states there are "no readily accessible sources of 'rigorous' evidence" and that districts "largely depend on recommendations from peers and from their own teachers and principals."

"There are no readily accessible sources of "rigorous" evidence on the effectiveness of the vast majority of ed-tech products. As a result, school districts largely depend on recommendations from peers and from their own teachers and principals who have familiarity with the products."

Studies bearing on this claim¶

When a school or district chooses an ed-tech product, is there rigorous evidence to choose by, and do purchasers use it? This table gathers the studies behind this claim and the 3 narrower claims that bear on it: 5 distinct studies (4 associational, 1 theoretical), q2, 0 of 5 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• Discovery of ed-tech products is a serious challenge for both districts and providers — related
• School districts and ed-tech developers hold different perspectives on the role and value of school-based technology pilots in procurement — related
• District stakeholders are mostly satisfied with procurement while providers are largely dissatisfied — related
• Teachers' Grow use is strongly influenced by technology professional development, their own use of technology, and principal support for Grow — related
• Districts rely more on peer recommendations and pilots than on rigorous evidence when choosing ed-tech products — related
• Students, teachers, and principals are perceived to have limited involvement in ed-tech procurement decisions, and providers are far less satisfied with end-user involvement than district stakeholders — related
• Technology directors are the least likely to trust evidence from providers, with only 29 percent satisfied with the credibility of provider evidence — related
• Existing school-based behavioral health TA in California is siloed, and equitable access depends on local leaders' individual knowledge and networks — related
• Most surveyed ed tech developers report their products are informed by research, but purchasers struggle to evaluate that research — related
• More than 70 percent of the 180 most popular literacy applications referenced no research in their design — related
• Districts engage with edtech products primarily through three scenarios: RFPs, vendor pitches, and peer recommendations — related
• Districts defined pilot success differently, making it difficult to set a standard for evaluating product effects on student improvement — related
• District teams struggled to produce rigorous, reliable evidence for assessing their improvement work — related
• Teachers tended to adopt district-provided tools, and single-application step-by-step training supported uptake of Forms — related