The voice-based synthetic assistant interface showed very low false-positive rates in interaction with trainees, suggesting reliable rejection of unwanted commands¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2018 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across three training sessions (247 possible true-positive interactions per session from 19 participants × 13 steps), the number of false positives was very low (2, 3, 2). → Damacharla 2018

Evidence¶

Damacharla 2018¶

Damacharla, P., Dhakal, P., Stumbo, S., Javaid, A. Y., Ganapathy, S., Malek, D. A., Hodge, D. C., & Devabhaktuni, V. (2018). Effects of Voice-Based Synthetic Assistant on Performance of Emergency Care Provider in Training. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-018-0166-3

q2 · i? · design · r2

Descriptive analysis of VBSA interaction metrics (TP, TN, FP, FN) from 19 experiment-set participants each executing 13 steps in three training sessions, plotted in Fig. 9 (e.g., FP of 2, 3, 2 across Train 1–3). The authors observe "the value of FP is very low"; no statistical test is reported.

"As we can observe from the graph above, the value of FP is very low and suggests that it is highly unlikely that the VBSA will accept any unwanted command, making the developed system reliable."

Discussion¶

Related Claims¶

• Reporting errors were significantly reduced in every training session of the VBSA-trained group compared to the control group, and implementation errors were reduced from first to last training session — related
• Repetitive training with the synthetic assistant produced statistically significant performance growth across successive tests, whereas the control group's growth was largely insignificant — related
• Training with the voice-based synthetic assistant caused no statistically significant loss in task execution time and no significant time difference between groups across testing scenarios — related