Outcome evaluation of research education programs concentrated at lower levels of the modified Kirkpatrick typology, with few objective or organisational outcomes¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2–r3 · q3
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Outcome measures largely reflected lower levels of the modified Kirkpatrick typology, such as satisfaction and improved research knowledge and confidence. → King 2022 q3 i? Few programs were evaluated using objective research milestones or organisational and practice outcomes. → King 2022 (2)

Evidence¶

King 2022¶

King, O., West, E., Lee, S., Glenister, K., Quilliam, C., Wong, S. A., & Beks, H. (2022). Research education and training for nurses and allied health professionals: a systematic scoping review. BMC Medical Education. https://doi.org/10.1186/s12909-022-03406-7

q3 · i? · review · r2

The review's synthesis of evaluation approaches across included programs, mapping reported outcomes to the modified Kirkpatrick typology. Outcomes were mostly satisfaction, knowledge and confidence, with "few evaluated objectively using traditional research milestones".

"Outcome measures largely reflected lower levels of Barr and colleagues' modified Kirkpatrick educational outcomes typology (e.g., satisfaction, improved research knowledge and confidence), with few evaluated objectively using traditional research milestones (e.g., protocol completion, manuscript preparation, poster, conference presentation)."

King 2022 (2)¶

King, O., West, E., Lee, S., Glenister, K., Quilliam, C., Wong, S. A., & Beks, H. (2022). Research education and training for nurses and allied health professionals: a systematic scoping review. BMC Medical Education. https://doi.org/10.1186/s12909-022-03406-7

q3 · i? · review · r3

The review reports that organisational and practice outcomes were rarely measured and that evaluation methods were "poorly described" across the included program evaluations.

"Few programs were evaluated using organisational and practice outcomes. Overall, evaluation methods were poorly described."

Discussion¶

Related Claims¶

• Korean simulation-based IPE studies report high satisfaction but concentrate on Kirkpatrick levels 1 and 2, with no study measuring level 4 training efficacy — a narrower finding that bears on this claim
• Reviewed peer-tutoring simulation studies mostly reached only Kirkpatrick Level 2, with no assessment of behavior change or clinical-practice results — a narrower finding that bears on this claim
• No included SDL studies measured Kirkpatrick Level 4 (patient/system) outcomes — a narrower finding that bears on this claim
• Evidence for technology-enhanced basic robotic skills training is concentrated at Kirkpatrick level 2, with no studies reporting transfer to operative behaviour or patient outcomes — a narrower finding that bears on this claim
• PBL in family medicine residency programs is associated with high learner satisfaction, though not always rated above lectures — related
• Peer tutoring in clinical simulation is associated with gains in non-technical skills including self-efficacy, self-confidence, self-perception of learning, and reflective abilities — related
• Research education programs in health settings often fail to account for adult learning principles and theories and are not always formally evaluated — related
• Educational interventions for FM residents targeting the informal curriculum rarely produced behavior change: most were evaluated only at Kirkpatrick level 2 and study quality was low
• No included systematic review reported outcomes on nurses' practice change (Kirkpatrick level 3), so the transfer of e-learning to practice remains unknown — related