IntroEduAI¶
Product or Programme · All products and programmes
Evidence · 4 claims (2 for, 2 mixed) · 3 studies (1 causal, 1 review, 1 associational),q2–q3· 0 of 3 report an effect size · 4 claims rest on one study
Description¶
IntroEduAI is an educational platform developed to deliver programming assignments, automatically evaluate student responses with large language models against instructor-authored rubrics, and generate response-level feedback and assignment-level reports.
Components¶
- IntroEduAI two-layer LLM feedback pipeline (rubric-based GPT-4.1 evaluation plus consolidated report generation): The IntroEduAI platform manages assignment delivery, automated evaluation, and feedback generation in programming courses. Each response is scored against an instructor-authored criterion-referenced rubric in a few-shot prompt, evaluated three times with GPT-4.1-2025-04-14, with text responses enhanced by retrieval-augmented generation over course materials. The three-stage pipeline produces "a single consolidated feedback per response, and an assignment-level report", the latter generated by claude-sonnet-4-20250514, containing a balance, strengths, areas to develop, and suggested next steps. (Mendonça PC et al. (2026))
- IntroEduAI platform for assessment collection and parallel human/LLM evaluation: IntroEduAI is a web-based platform, originally developed for prior research on automated formative assessment, that served as the technical infrastructure for assessment collection and automated evaluation in this study. It implements a Model-View-Controller architecture using Flask with REST API endpoints for LLM integration, and supports rubric upload and configuration, student response collection, parallel human and LLM-based evaluation, and data aggregation for export. Automated evaluation uses few-shot prompting with rubric integration; for text questions it adds Retrieval-Augmented Generation via LangChain and ChromaDB over course materials (1000-character chunks with 200-character overlap, top-5 retrieval), while code questions are evaluated without external retrieval. (Pedro C. Mendonça et al. (2026))
Claims¶
- Students rate LLM-generated response-level feedback favourably, with clarity and perceived accuracy highest and actionability and usefulness lowest [+M]
- Students rate LLM-generated consolidated reports favourably, with cognitive load rated lowest [+M]
- Prompt quality and detailed scoring rubrics significantly influence the accuracy and consistency of LLM-generated grades [~W]
- Under LLM evaluators, structured rubrics produced larger negative deviations from the human baseline than free-form rubrics, the opposite direction of the human-evaluator pattern [~W]
Related Products and Programmes¶
-
Key Sources¶
- Mendonça PC, Quintal F, Figueiredo M and Mendonça F (2026) Perceived usefulness and intention to use large language model-generated feedback across three educational levels: a user-centred study in programming. Front. Educ. 11:1934069. https://doi.org/10.3389/feduc.2026.1934069
- Pedro C. Mendonça, Filipe Quintal, Mário Figueiredo, Karolina Baras, Fábio Mendonça. (2026). A hybrid reasoning framework for artificial intelligence assessment rubric generation in human and automated contexts: Evidence from an undergraduate programming course. Computers and Education: Artificial Intelligence 11 (2026) 100663. https://doi.org/10.1016/j.caeai.2026.100663
Page metadata
| Field | Value |
|---|---|
type |
product |
id |
introeduai |
title |
IntroEduAI |
description |
IntroEduAI is an educational platform developed to deliver programming assignments, automatically evaluate student responses with large language models against instructor-authored rubrics, and generate response-level feedback and assignment-level reports. |
status |
review |
product_kind |
software |
generated |
by process:settle-candidates on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.