Provide-Info, Request-Info, and Other are the dialogue act labels most commonly confused by both GPT-4 and T5 classifiers.¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Confusion matrix analysis shows that Provide-Info, Request-Info, and Other are the labels most commonly confused by both GPT-4 and T5 classifiers. → Spain 2024

Evidence¶

Spain 2024¶

Spain, R., Min, W., Kumaran, V., Pande, V., Saville, J., & Lester, J. (2024). Applying Large Language Models to Enhance Dialogue and Communication Analysis for Adaptive Team Training. https://doi.org/10.21203/rs.3.rs-4565500/v1

q2 · i? · design · r2

Error analysis using confusion matrices constructed from the classification results of the two highest-performing models (GPT-4 and T5) on the test dataset. The article reports that "the mislabeling patterns are similar for both classiers" and that "Provide-Info, Request-Info, and Other are the labels most commonly confused by both classiers".

"Notably, the mislabeling patterns are similar for both classiers. Provide-Info, Request-Info, and Other are the labels most commonly confused by both classiers."

Discussion¶

Related Claims¶

• Prompt refinements targeting inter-label confusion yield superior recall rates for challenging dialogue act labels, including Command (F1 raised to 0.75) and Other (GPT-4 recall 0.67 vs. T5 0.54). — related