Second-submission accuracy on the electrical grid problem was significantly lower than first-submission accuracy, the reverse of expectations¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Subjects averaged 18.8% accuracy on first submission and 15.4% on second attempt for the electrical power grid problem, a statistically significant decrease (t(129) = 2.919, p < .01). → Foster 2016 q2 i? For the heat exchanger problem, first (11.7%) and second (12.6%) submission accuracy did not differ significantly (t(128) = -1.17, p > .05). → Foster 2016 (2)

Evidence¶

Foster 2016¶

Foster, W. T., Shahhosseini, A. M., & Maughan, G. (2016). Advancing Diagnostic Skills for Technology and Engineering Undergraduates: A Summary of the Validation Data. Journal of Technology Education, 28(1). https://scholar.lib.vt.edu/ejournals/JTE/v28n1/

q2 · i? · causal · r2

Paired-samples comparison of first versus second map submissions in the quasi-experiment; percent accuracy was based on a perfect match with the expert map. The significant difference was a decrease, which the article calls the reverse of expectations.

"the subjects who completed the second problem (i.e., electrical power grid problem) scored an average of 18.8% (Sn-1 = .042) on their first submission and 15.4% (Sn-1 = .043) on their second attempt. A paired samples t-test revealed that the difference was statistically significant (t(129) = 2.919, p < .01), although the reverse of expectations."

Foster 2016 (2)¶

Foster, W. T., Shahhosseini, A. M., & Maughan, G. (2016). Advancing Diagnostic Skills for Technology and Engineering Undergraduates: A Summary of the Validation Data. Journal of Technology Education, 28(1). https://scholar.lib.vt.edu/ejournals/JTE/v28n1/

q2 · i? · causal · r2

Paired-samples comparison for the heat exchanger problem in the same quasi-experiment; the small descriptive increase from 11.7% to 12.6% was not statistically significant.

"Subjects who completed the third problem (i.e., heat exchanger problem) scored on average 11.7% (Sn-1 = .021) for their first submission and 12.6% (Sn-1 = .03) for their second submission. This was not found to be a statistically significant difference (t(128) = -1.17, p > .05)."

Discussion¶

Related Claims¶

• Feedback type (metacognitive cues, expert map review, both, or scores only) produced no statistically significant difference in mean accuracy change on either technical problem — related
• A subset of top-performing subjects achieved substantially higher map accuracy, indicating some students understood the mapping technique well — related