Software engineering open-ended answer dataset¶
Product or Programme · All products and programmes
Evidence · 2 claims (2 for) · 1 study (1 design),q1· 0 of 1 report an effect size · 2 claims rest on one study
Description¶
An anonymized dataset created by Pecuchova et al. containing 1,885 Moodle open-ended responses from 110 university software-engineering students, with reference answers and human-rater grades for evaluating automated grading systems.
Components¶
- Software engineering open-ended answer dataset (1,885 responses, 110 students, 24 questions): An anonymized dataset of open-ended answers collected via Moodle in a blended Software Engineering course at Constantine the Philosopher University in Nitra. "In total, the dataset includes 1,885 responses from 110 unique students" answering 24 questions on software requirements, UML diagrams, development models and agile methodologies. Each record holds the student ID, question, student answer, reference answer, and grades from two human raters. The dataset was created by the authors under a research project on automated grading with generative AI support. (Pecuchova et al. (2025))
Claims¶
- NLI analysis reveals reference-based grading is limited: many semantically correct responses contradict reference answers [+W]
- Sentence embedding models reach only moderate agreement and need human supervision for open-ended grading [+W]
Related Products and Programmes¶
-
Key Sources¶
- Pecuchova, J., Benko, Ľ., & Drlik, M. (2025). Automated Grading of Open-Ended Questions in Higher Education Using GenAI Models. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00517-2
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
software-engineering-open-ended-answer-dataset |
title |
Software engineering open-ended answer dataset |
description |
An anonymized dataset created by Pecuchova et al. |
status |
review |
product_kind |
dataset |
generated |
by process:settle-candidates on 2026-10-12 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.