Using performance on each individual question as features yields only a small improvement over a single-ability logistic regression model¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? The improvement from adding per-question features to logistic regression is quite small, supporting the conclusion that overall mathematical competence is the dominant factor in predicting performance. → Benjamin Mawdsley 2026
Evidence¶
Benjamin Mawdsley 2026¶
Benjamin Mawdsley, Tom Quilter, Richard Turner, Sarah Jackson, and Paul Edwards. (2026). Archetypes or ability? Clustering for modelling student mathematical competence. https://arxiv.org/abs/2607.26063 doi:10.48550/arxiv.2607.26063
q2 · i? · associational · r2
Comparison of LogRegs (single average-grade feature) and LogRegm (19 per-question features) in the predictive evaluation; Table 1 shows accuracy 0.769±0.005 versus 0.778±0.005 with overlapping error bars. The improvement is described as "quite small".
"Despite adding many more features, the performance improvement between a single feature logistic regression and a multi question performance is quite small, supporting the conclusion from [20] that the dominant factor in predicting performance is overall mathematical competence."
Discussion¶
Related Claims¶
- Cluster profiles of student mathematics performance are dominated by overall ability, with most clusters being linear rescalings of average exam performance — related
- Explainable models including logistic regression, kNN and a Bernoulli Mixture Model all reach around 78% accuracy in predicting question-level pass/fail, with well-calibrated probabilities — related
- Ablation of feature-vector models: time-window features add no predictive power to logistic regression but boost a feedforward network, and total count features substantially boost performance on all datasets — related
- Supplementary nonlinear models yield only modest AUC gains over the calibrated logistic baseline — related
- Prior mathematical accuracy is the dominant propensity factor: over half of January accuracy variance and under one-fifth of fluency variance predicted June performance — related
- Bernoulli Mixture Model fits select around 10-12 clusters per question, with no runs finding high cluster counts optimal, indicating specialised skill sets do not generally exist — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
small-gain-from-question-level-features |
title |
Using performance on each individual question as features yields only a small improvement over a single-ability logistic regression model |
description |
Using performance on each individual question as features yields only a small improvement over a single-ability logistic regression model |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — benjamin-mawdsley-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.