Use two researchers as independent instruments for interpretation instead of one researcher confirming the other's reliability¶

Strategy · All strategies
Evidence · no claims cited

Description¶

This strategy replaces conventional single-coder reliability checking with collaborative interpretation. The study uses "two researchers as independent instruments for interpretation in education policy evaluation", and rather than "using one researcher to confirm the reliability of the other", it draws on each researcher's positional reflexivity to cogenerate themes and theory. It is implementable in qualitative evaluation projects analyzing interview or other qualitative data.

Design Implications¶

Context¶

Requirements¶

• Two researchers with distinct perspectives willing to engage in positional reflexivity during analysis

Constraints¶

• Described for qualitative data analysis and theory generation in developmental evaluation of education policy

Target Learners¶

• education evaluators
• qualitative research teams

Target Learning Goals¶

• cogenerating themes and theory in evaluation of complex policy or programs

Affordances¶

• Collaborative Grounded Theory Developmental Evaluation

Related Strategies¶

• Assess social and group flow in real time through non-intrusive observation instead of interrupting learners with self-report surveys
• Report intercoder agreement at each level of a hierarchical codebook using set-valued measures such as Jaccard similarity

Examples¶

-

Key Sources¶

• Anderson, R., Guerreiro, M., & Smith, J. (2016). Are all biases bad? Collaborative grounded theory in developmental evaluation of education policy. Journal of Multidisciplinary Evaluation 12(27), 44-57. https://www.nwea.org/research/publication/are-all-biases-bad-collaborative-grounded-theory-in-developmental-evaluation-of-education-policy/. doi:10.56645/jmde.v12i27.449