Instructors value design problems for assessing higher-order thinking in PjBL but cite creation effort and scalability as adoption barriers¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? All 31 surveyed instructors supported the potential of DPs to assess HOT in PjBL, with 71.0% indicating clear agreement and 29.0% agreeing with caveats. → Ahmad D. Suleiman 2026 q2 i? Only 6.5% of instructors explicitly used scenario-based transfer questions, suggesting a gap between stated intent to assess HOT and use of transfer-oriented assessment. → Ahmad D. Suleiman 2026 (2)

Evidence¶

Ahmad D. Suleiman 2026¶

Ahmad D. Suleiman, Daqing Hou, and Maliha Noushin Raida. (2026). LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning. arXiv preprint arXiv:2607.11032. https://arxiv.org/abs/2607.11032 doi:10.48550/arxiv.2607.11032

q2 · i? · design · r2

Reflexive thematic analysis of a mostly open-ended survey of 31 PjBL instructors (~10% response rate) found unanimous support for DPs, with "22/31; 71.0%" indicating clear agreement and caveats centering on validity under generative AI use.

"First, all 31 instructors supported the potential of DPs to assess HOT in PjBL. Most indicated a clear agreement that DPs can be effective (22/31; 71.0%), while the remainder agreed but with caveats (9/31; 29.0%)."

Ahmad D. Suleiman 2026 (2)¶

Ahmad D. Suleiman, Daqing Hou, and Maliha Noushin Raida. (2026). LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning. arXiv preprint arXiv:2607.11032. https://arxiv.org/abs/2607.11032

q2 · i? · design · r2

The same survey found that although 71.0% said they explicitly assess HOT, only "2 instructors (6.5%)" used scenario-based transfer questions, with most relying on project artifacts and oral defenses instead.

"Only 2 instructors (6.5%) explicitly mentioned scenario-based questions that require students to analyze a new situation and propose solutions."

Discussion¶

Related Claims¶

• Student performance on design problems correlates negligibly with traditional project grades, suggesting DPs capture distinct transfer-oriented HOT dimensions — related
• Instructors reported that FORAP packaging reduced the initial effort to adopt and adapt PjBL activities, based on anecdotal feedback — related
• Students rated DPs positively, did not distinguish the two LLMs, and disagreed with experts on the same DPs, rating feasibility significantly lower — related
• Student DP performance differed significantly across the three course deployments, with the paper-based DT3 exam scoring lowest, likely due to administration mode — related
• Half of LLM-generated design problems received perfect expert ratings, though common failure modes included scenarios too similar to the original project and ambiguous scope — related