Future peer assessment tools should support unlimited reviewing, digital ink annotation, and more thorough evaluation studies¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 review), q3 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The review's conclusion sets an agenda for tool development and research. It argues that in systems without review limits, "students with higher grades tend to contribute more than weaker students, resulting in a greater amount of higher quality feedback being produced", so "future tools should support unlimited reviewing where possible." It also calls for usability and evaluation studies, comparative studies of weighting algorithms, evaluation of rubric forms, investigation of discussion at different stages, and support for digital ink annotation, which no tool then provided.

Design Implications¶

Context¶

Requirements¶

• Tools must handle late or missing submissions since one student's output is another's input
• Comparative and usability studies are needed before design choices can be grounded

Constraints¶

• The benefits of unlimited reviewing had not been formally investigated at the time of writing
• Only Praktomat supported direct annotation on the reviewed product and none supported digital ink

Target Learners¶

• higher education students
• computer science students

Target Learning Goals¶

• peer assessment skills
• providing high-quality formative feedback

Claims¶

• Multiple peer reviews produced greater improvement between draft and final paper than a single expert reviewer in a SWoRD evaluation [+W]

Related Strategies¶

Examples¶

-

Key Sources¶

• Luxton-Reilly, A. (2009). A systematic review of tools that support peer assessment. Computer Science Education, 19(4), 209-232. https://doi.org/10.1080/08993400903384844