PsyScore¶
Product or Programme · All products and programmes
Evidence · 5 claims (4 for, 1 mixed) · 1 study (1 design),q2· 0 of 1 report an effect size · 5 claims rest on one study
Description¶
A psychometrically aware automated essay-scoring and formative-feedback framework developed by Wei Xia et al. that combines latent-ability scoring with ZPD-conditioned instructional feedback.
Components¶
Claims¶
- PsyScore-AES achieves a state-of-the-art average QWK of 0.747 on ASAP++, surpassing the strongest baseline SaMRL-large (0.722) and ranking first on 6 of 8 prompts [+M]
- In pairwise LLM-judge comparisons PsyScore-AEF wins over 90% in Actionability against GPT-4o and Llama, and removing the psychometric module reduces Adaptivity and Tone wins with PsyScore retaining above 75% [+M]
- Removing grid-search initialization of IRT discrimination and difficulty parameters drops scoring performance from QWK 0.747 to 0.705 [~W]
- At trait level PsyScore-AES ranks first in 10 of 11 dimensions (mean QWK 0.735), improving Voice scoring from 0.619 to 0.740 while trailing T-MES on Style [+W]
- Three education experts rate PsyScore feedback significantly higher than all LLM baselines on every dimension (p < .001), with the No-IRT ablation declining in Adaptivity and Accuracy (p < .05) [+W]
Related Products and Programmes¶
-
Key Sources¶
- Wei Xia, Jin Wu, Haoran Shi, Xiangyu Wang, Chanjin Zheng. (2026). PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback. arXiv preprint. https://arxiv.org/abs/2606.20287
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
psyscore |
title |
PsyScore |
description |
A psychometrically aware automated essay-scoring and formative-feedback framework developed by Wei Xia et al. |
status |
review |
product_kind |
framework |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.