ChatGPT use improves the quality, elaboration, and originality of university students' solutions to a complex ill-defined problem-solving task¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r3 · q3 · i2 medium
Currency · newest study 2024 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q3 i2 Students solving the Product Improvement Task with ChatGPT produced solutions of higher quality than controls, F(1,142) = 19.07, η2p = .12. → Urban 2024 q3 i2 ChatGPT-assisted solutions were better elaborated, F(1,142) = 14.87, η2p = .10, and more original, F(1,142) = 12.56, η2p = .08, than control solutions. → Urban 2024 q3 i2 A significant group-by-task interaction on originality indicated a moderate improvement when ChatGPT was used, with no pre-test difference between groups. → Urban 2024 (2)

Evidence¶

Urban 2024¶

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT Improves Creative Problem-Solving Performance in University Students: An Experimental Study. Computers & Education. https://doi.org/10.1016/j.compedu.2024.105031

q3 · i2 · causal · r3

ANCOVAs (with prior ChatGPT experience as covariate) in a randomized experiment with 68 control and 77 experimental university students. The article reports a "strong impact on the quality of the solutions" with η2p = .12, elaboration η2p = .10, originality η2p = .08.

"Use of ChatGPT had a strong impact on the quality of the solutions, F(1,142) = 19.07, p <.001, η2p = .12, a moderate impact on elaboration, F(1,142) = 14.87, p = <.001, η 2p = .10, and a moderate impact on originality, F(1,142) = 12.56, p < .001, η2p = .08."

Urban 2024 (2)¶

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT Improves Creative Problem-Solving Performance in University Students: An Experimental Study. Computers & Education. https://doi.org/10.1016/j.compedu.2024.105031

q3 · i2 · causal · r3

Mixed within-between ANCOVA comparing pre-test Unusual Uses Task (solved without ChatGPT in both conditions) with the experimental task. Pre-test groups did not differ, F(1,142) = 0.01, and the interaction term reached η 2p = .05.

"However, the significant interaction term, F(1,142) = 7.79, p = .006, η 2p = .05, indicated a moderate improvement in originality scores when ChatGPT was used."

Discussion¶

Related Claims¶

• ChatGPT use boosts university students' self-efficacy for task resolution, but self-efficacy barely tracks actual performance — reports the opposite
• Solving a creative problem-solving task with ChatGPT is perceived as easier and requires less mental effort, but is not more interesting — related
• ChatGPT use does not change absolute self-evaluation accuracy, but self-evaluation judgments fail to correlate with performance under ChatGPT assistance — related
• AI use in training carries risks of uncritical acceptance and overreliance, including undetected conceptual errors and increased delegation of problem solving — related
• Free ChatGPT use boosts originality on the assisted creative task but the advantage disappears when assistance is removed — related
• The regulated 'think first, ChatGPT later' group outperformed both other groups in independent originality on the unassisted task despite no assisted-task advantage — related
• Nasty task environments increased negative affect and improved complex problem solving performance relative to nice environments