The 4-category rating scale of the POET-O functions properly overall but its threshold structure could be improved¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Average measures advanced with rating categories and the scale functioned properly, though the threshold between categories 2 and 3 was disordered relative to the threshold between 1 and 2. → Jia 2025

Evidence¶

Jia 2025¶

Jia, Y., Spagnolo, A. B., Barrett, N., Murphy, A. A., Basto, P. M., Rothpletz-Puglia, P., & Luther, S. (2025). Validation of a peer observation and evaluation tool for online teaching in the U.S. Education Tech Research Dev. https://doi.org/10.1007/s11423-024-10428-z

q2 · i? · design · r2

Rating scale analysis of the four-category scale found average measures advanced as categories increased and outfit mean-square statistics were near expected values, but "The threshold between category 2 and 3 was smaller than the threshold between category 1 and 2, which was disordered."

"The threshold between category 2 and 3 was smaller than the threshold between category 1 and 2, which was disordered."

Discussion¶

Related Claims¶

• Asynchronous lessons were rated higher than synchronous lessons on the POET-O — related
• Rasch analysis indicates a five-point response format would improve the GenAI-RTS's category functioning — related
• The overall CT–PCK Survey functioned similarly across teacher characteristic groups, though some items showed raw differential item functioning above 0.64 logits — related
• The POET-O demonstrates adequate construct validity and reliability for peer evaluation of single online lessons — related
• ADA accessibility items were the most difficult and impacted the POET-O's construct validity — related