Make platform quality standards interpretable by pairing automated scores with rationales and contextual cues¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 qualitative), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article's third design consideration states that automated feedback can support gig platforms' quality control by organizing evaluations around structured criteria rather than relying solely on learners' individual impressions. To make these standards actionable, "systems should pair scores with brief rationales, evidence, or contextual cues" explaining how tutors can improve within platform expectations, addressing tutors' requests for clearer explanations of discrepancies.

Design Implications¶

Context¶

Requirements¶

• Automated scores must be paired with brief rationales, evidence, or contextual cues that explain how tutors can improve.

Constraints¶

• Automated feedback in the study presented only numerical scores without context, which tutors found insufficient.

Target Learners¶

• Tutors on educational gig platforms subject to standardized evaluation

Target Learning Goals¶

• Understanding of platform instructional standards and actionable improvement

Claims¶

• Students report receiving scores with little individualized feedback and request more in-depth feedback that explains why and how they can improve [+W]

Related Strategies¶

• Distinguish the purposes and scales of coexisting feedback sources in educational gig platforms

Examples¶

-

Key Sources¶

• Yeon Su Park, Sieun Kim, Keighley Overbay, Seoyoung Kim, Sewook Wee, Daho Jung and Juho Kim. (2026). Supporting Tutors in the Gig Economy with Automated Feedback: A Case Study on Ringle. The 2026 Workshop on Pedagogical Evaluation of Automated Feedback (PEAF2026). https://osf.io/zsc3b/overview?view_only=dcf09d1a9fe64765b013df02af6bc382