Validation evidence across the 15 RCR measures is partial and heterogeneous, with no test–retest, criterion validity or factor-analytic evidence reported¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Twelve measures reported reliability evidence, mostly limited to internal consistency via Cronbach's alpha; no measure reported test–retest reliability. → M. van Loon 2026 q2 i? Nine measures provided some validity evidence, but no measure provided criterion validity evidence and none reported factor-analytic examinations of dimensional structure. → M. van Loon 2026 (2)

Evidence¶

M. van Loon 2026¶

M. van Loon, D. Crean, M. van den Hoven, J. Katsarov, N. Evans. (2026). How is the Impact of Responsible Conduct of Research Education Evaluated? A Review of Measures Used and a Content Analysis of Outcome Measures with Reported Validation Evidence. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10187-8

q2 · i? · review · r2

Review synthesis of Table 5 validation data across the 15 measures. Eleven measures relied on Cronbach's alpha, "often reporting values ≥ 0.80"; only PDR reported parallel-forms reliability and the Saudi Arabia instrument split-half reliability; P2I and ACP reported none.

"Notably, no measure reported test–retest reliability, meaning that stability over time—a key psychometric criterion—remains unknown for all measures."

M. van Loon 2026 (2)¶

M. van Loon, D. Crean, M. van den Hoven, J. Katsarov, N. Evans. (2026). How is the Impact of Responsible Conduct of Research Education Evaluated? A Review of Measures Used and a Content Analysis of Outcome Measures with Reported Validation Evidence. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10187-8

q2 · i? · review · r2

Review synthesis of validity evidence: most validity evidence was content or face validity via expert review, often minimally described; only the RCR Knowledge Test and VSW supplemented it with cognitive interviews. The authors note validation was "partial, fragmented, or narrowly focused on internal consistency".

"Only two measures—Ethics QNN and LCRE—reported convergent construct validity, demonstrating expected correlations with existing validated scales. No measure provided criterion validity evidence, and none reported factor-analytic examinations of dimensional structure."

Discussion¶

Related Claims¶

• Instruments show great heterogeneity in measurement quality, with validity or reliability information lacking in many cases — a broader claim this one bears on
• Fifteen unique RCR outcome measures with supporting validation evidence were identified, all covering research integrity but only ten covering research ethics
• Validated RCR measures omit emerging and social-safety topics including EDI, whistleblower protections, AI use in research, predatory journals and open-science practices
• The majority of published RCR education evaluations use measures without reported validation evidence — related
• Developers of the principal practice instruments provided only partial information about the instruments' reliability and validity — related
• Most validated RCR measures assess a single learning outcome, with attitudes most common and only four measures testing knowledge — related
• Retest reliability may be more appropriate than alpha for knowledge tests, but requires a testing interval without learning — related