Use a two-step quality-indicator and decision-rule procedure that considers both high-quality and acceptable-quality research when establishing level of evidence¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The review used the NSTTAC two-step procedure: first applying a 20-item quality indicator checklist to rate studies high or acceptable quality, then applying decision rules on number of studies, research teams, functional relationship, and contradictory evidence to classify evidence as strong, moderate, or potential. The authors argue that "a procedure like the one developed by Test et al. (2009) that considers both high-quality and acceptable quality research when establishing the level of evidence for an instructional practice may be useful" when the review's purpose is helping educators make informed decisions.

Design Implications¶

Context¶

Requirements¶

• A 20-item quality indicator checklist and decision rules for determining levels of evidence

Constraints¶

• Requiring all 21 Horner et al. indicators for high quality may prohibit quality research from contributing to an evidence base; other reviews found only one of 12 and one of 11 studies met all 21

Target Learners¶

• students with disabilities

Target Learning Goals¶

• identifying evidence-based instructional practices

Related Strategies¶

• Route items flagged for misfit through content and psychometric review before deciding whether to deactivate them

Examples¶

-

Key Sources¶

• Melissa E. Hudson and David W. Test. (2011). Evaluating the Evidence Base of Shared Story Reading to Promote Literacy for Students With Extensive Support Needs. Research & Practice for Persons with Severe Disabilities, 36(1-2), 34-45. http://www.education.uncc.edu/access/