Apply seventeen evidence-based design guidelines when developing formative assessment with analytic rubrics for complex skills¶

Strategy · All strategies
Evidence · 4 claims (1 for, 2 mixed, 1 against) · 2 studies (1 causal, 1 design), q1–q3 · 1 of 2 report an effect size · 4 claims rest on one study

Description¶

The article compiles seventeen guidelines for formative assessment with analytic rubrics, stating "Table 1 presents an overview of (seventeen) guidelines on analytic rubrics we encountered in literature. Guidelines 1–4 inform us how to use rubrics for formative assessment; Guidelines 5–17 inform us how to use rubrics for instruction, with Guidelines 5–9 on a rather generic, meso level and Guidelines 10–17 on a more specific, micro level." They cover preferring analytic over rating-scale rubrics, self- and peer-assessment, training teachers and students, authentic tasks of increasing complexity, modelling examples, rehearsal practice, high-quality feedback, and specific goal-setting. The study applied all seventeen in designing its classroom scenarios.

Design Implications¶

Context¶

Requirements¶

• Rubrics must be tailored to specific learning objectives and target groups; teachers and students need training before use

Constraints¶

• The guidelines were validated in secondary education for oral presentation; the authors note the design-based context limits exclusion of confounding factors

Target Learners¶

• Secondary school students acquiring complex skills

Target Learning Goals¶

• Complex skill acquisition supported by formative assessment with analytic rubrics

Claims¶

• Formative assessment with validated analytic rubrics in the Viewbrics tool produced higher oral presentation performance than conventional rating-scale rubrics in Dutch secondary education [+M]
• Oral presentation performance improved significantly over time, but only control-group students showed statistically significant growth between the two presentations [-M]
• Video-enhanced analytic rubrics did not significantly outperform text-based analytic rubrics on oral presentation performance, though a marginal trend favored video-enhanced rubrics [~M]
• Text-based rubrics alone provide insufficient contextual, dynamic and procedural information to support rich mental model development of complex skills [~W]

Related Strategies¶

• Six practical design guidelines for building a Video Enhanced Rubric
• Rubric Based Assessment
• Develop standardised validated content-assessment tools and theory-based guidelines for TST video creators

Examples¶

-

Key Sources¶

• Rob J. Nadolski, Hans G. K. Hummel, Ellen Rusman, Kevin Ackermans. (2021). Rubric formats for the formative assessment of oral presentation skills acquisition in secondary education. Education Tech Research Dev. https://doi.org/10.1007/s11423-021-10030-7