Four-task architecture of difficulty prediction models

Research Method · All research methods
Evidence · 2 claims (2 for) · 1 study (1 review), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description

The review characterises difficulty prediction models as sharing four architectural components representing four fundamental tasks: "1) Ground truth labelling, 2) pre-processing, 3) feature extraction and 4) prediction." Ground truth is labelled via psychometric theories (CTT, IRT) or expert rating; pre-processing applies standard NLP techniques such as stemming, POS tagging and lemmatisation; feature extraction operates at syntactic (surface) or semantic levels; and prediction is overwhelmingly supervised. The review uses this structure to organise its findings sections.

Accounts

• Four-task architecture of difficulty prediction models: ground truth labelling, pre-processing, feature extraction, prediction: The review characterises difficulty prediction models as sharing four architectural components representing four fundamental tasks: "1) Ground truth labelling, 2) pre-processing, 3) feature extraction and 4) prediction." Ground truth is labelled via psychometric theories (CTT, IRT) or expert rating; pre-processing applies standard NLP techniques such as stemming, POS tagging and lemmatisation; feature extraction operates at syntactic (surface) or semantic levels; and prediction is overwhelmingly supervised. The review uses this structure to organise its findings sections. (Samah AlKhuzaey et al. (2023))

Claims

• Ground-truth difficulty labels in reviewed studies come roughly equally from expert/annotator labelling (34%), Classical Test Theory (34%), and Item Response Theory (26%) [+W]
• All surveyed automatic question difficulty prediction studies frame the task as supervised learning, mostly as classification of pre-defined difficulty categories [+W]

Related Research Methods

-

Key Sources

• Samah AlKhuzaey, Floriana Grasso, Terry R. Payne, Valentina Tamma. (2023). Text-based Question Difficulty Prediction: A Systematic Review of Automatic Approaches. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00362-1

Page metadata

Field	Value
type	research-method
id	four-task-architecture-of-difficulty-prediction-models
title	Four-task architecture of difficulty prediction models
description	The review characterises difficulty prediction models as sharing four architectural components representing four fundamental tasks: "1) Ground truth labelling, 2) pre-processing, 3) feature extraction and 4) prediction." Ground truth is labelled via psychometric theories (CTT, IRT) or expert rating;
status	review
generated	by process:settle-candidates on 2026-10-28
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.