Assignments at the first five levels of Bloom's taxonomy are suited to LLM grading, while Create-level assignments are more challenging for LLMs to assess¶

Claim · All claims
Evidence · 1 study · causal r1 · q1
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q1 i? The authors argue LLM grading suits assignments requiring Remember, Understand, Apply, Analyze, and Evaluate, but Create-level and metacognitive writing is more challenging for LLMs to assess. → Chris Impey 2024

Evidence¶

Chris Impey 2024¶

Chris Impey, Matthew Wenger, Nikhil Garuda, Shahriar Golchin, Sarah Stamer. (2024). Using Large Language Models for Automated Grading of Student Writing about Science. Preprint. https://doi.org/10.21203/rs.3.rs-3962175/v1

q1 · i? · causal · r1

Authors' interpretation in the discussion, linking the observed grading results to Bloom's taxonomy levels; no new data are reported for this statement.

"Any assignments that fall in the sixth level (Create) will be more challenging for the LLM to assess, as was seen in the results from the Astrobiology class assignments. Current LLMs fall short of being able to evaluate creative assignments and any writing that requires metacognitive thinking."

Discussion¶

Related Claims¶

• Grading time varied systematically with Bloom's taxonomy category: procedural knowledge was fastest to grade and conceptual knowledge slowest — related
• Both models target Higher Bloom levels far more accurately than Lower levels, with the general model outperforming the coder model upward — related
• Most student–AI interactions occur at the understand and apply Bloom levels, with analyze appearing rarely — related