ChatGPT use boosts university students' self-efficacy for task resolution, but self-efficacy barely tracks actual performance¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r1 · q3 · i2 medium
Currency · newest study 2024 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

In the same experiment ChatGPT did improve actual performance (solution quality, elaboration and originality); the finding here is that the self-efficacy boost was larger than performance warranted, since self-efficacy correlated negligibly with performance and was associated with overestimation.

Subclaims¶

q3 i2 Self-efficacy for task resolution with ChatGPT was moderately higher than control self-efficacy, F(1,142) = 16.28, η2p = .10. → Urban 2024 q3 i2 Within-subject self-efficacy was much higher with ChatGPT than without, F(1,75) = 17.57, η2p = .19. → Urban 2024 q3 i? Self-efficacy correlated only negligibly with actual problem-solving performance (control median r = .11; experimental medians .01 and .04), while higher self-efficacy was associated with greater overestimation (control median r = .49). → Urban 2024 (2)

Evidence¶

Urban 2024¶

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT Improves Creative Problem-Solving Performance in University Students: An Experimental Study. Computers & Education. https://doi.org/10.1016/j.compedu.2024.105031

q3 · i2 · causal · r3

ANCOVA on self-efficacy ratings (100-point scale) from the randomized two-condition experiment (N = 145). Between-group self-efficacy without ChatGPT did not differ; the with-ChatGPT comparison showed η2p = .10, and the within-subject contrast η2p = .19.

"However, participant assessments of self-efficacy for task resolution with ChatGPT showed moderately higher self-efficacy in comparison to the control group, which did not use ChatGPT, F(1,142) = 16.28, p < .001, η2p = .10."

Urban 2024 (2)¶

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT Improves Creative Problem-Solving Performance in University Students: An Experimental Study. Computers & Education. https://doi.org/10.1016/j.compedu.2024.105031

q3 · i? · associational · r1

Exploratory correlation analysis of Table 3 from the same experiment. The article reports very small control-group and negligible experimental-group correlations, while higher self-efficacy was associated with higher bias indices (control median r = .49).

"self-efficacy and the quality, elaboration, and originality of the solutions on PIT (hereafter labeled as actual problem-solving performance; see variables 5, 6 and 7 in Table 3) had only a very small correlation in the control group (median r = .11) and a negligible one in the experimental group (median rno-ChatGPT = .01, median rwith-ChatGPT = .04)."

Discussion¶

Related Claims¶

• ChatGPT use improves the quality, elaboration, and originality of university students' solutions to a complex ill-defined problem-solving task — same study: ChatGPT did improve performance
• Solving a creative problem-solving task with ChatGPT is perceived as easier and requires less mental effort, but is not more interesting — related
• ChatGPT use does not change absolute self-evaluation accuracy, but self-evaluation judgments fail to correlate with performance under ChatGPT assistance — related
• Perceived usefulness of ChatGPT, an invalid cue, is associated with overestimation of performance — related
• Subjective effort ratings are affected by invalid cues and only weakly valid as cues to judged learning outcomes — related
• Primary studies report possible benefits of GenAI for children's divergent thinking, narrative creativity and creative self-efficacy, without confirmed efficacy — related
• Learning-process and psychological constructs correlate only weakly with aggregate final artistic performance — related