FSM-based evaluation discriminates interactive quality across models, with GPT-5 Mini scoring highest and most tiers significantly separated¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? FSM metrics significantly separate models: GPT-5 Mini achieves the highest overall score (78.85%) and behavioral coherence (33.13%), significantly outperforming GPT-3.5-Turbo (Δµ = 8.54, p < 0.0001). → Xiaozao Wang 2026 q2 i? FSM evaluation reveals content-dependent interaction demands: structured topics yield more stable transitions, whereas procedural topics show greater variance. → Xiaozao Wang 2026 (2)

Evidence¶

Xiaozao Wang 2026¶

Xiaozao Wang, Zhewei Wang, Hongyi Wen. (2026). Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations. https://arxiv.org/abs/2606.31012 doi:10.48550/arxiv.2606.31012

q2 · i? · design · r2

Two-sample Student's t-tests over FSM scores for the six-model study of 2497 generated explanations, reported in Results. GPT-5 Mini scored highest, with a significant separation from GPT-3.5-Turbo; GPT-3.5-Turbo and DeepSeek-V3 showed no significant difference (Δµ = 0.18, p = 0.884).

"FSM metrics clearly separate models: GPT-5 Mini achieves the highest overall score (78.85%) and behavioral coherence (33.13%), significantly outperforming GPT-3.5-Turbo ( ∆µ= 8.54 , p <0.0001 )."

Xiaozao Wang 2026 (2)¶

Xiaozao Wang, Zhewei Wang, Hongyi Wen. (2026). Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations. https://arxiv.org/abs/2606.31012

q2 · i? · design · r2

Same discriminative-power analysis (RQ2, Fig. 4) reporting that interaction quality varies by content type: structured topics produced more stable transitions while procedural topics showed greater variance in FSM-based scores.

"FSM evaluation further reveals content-dependent interaction demands: structured topics yield more stable transitions, whereas procedural topics show greater variance."

Discussion¶

Related Claims¶

• FSM-based evaluation aligns with human judgments of interactivity, functional correctness, and visual quality more strongly than VLM and unit-test baselines on balanced data — related
• Under FSM metrics, GPT-3.5-Turbo outperforms GPT-4o-Mini, reversing that model's higher rank on coding-oriented leaderboards — related
• The OpenAI gpt-4o reference model outperformed all quantised models on human-rated coherence, fluency, relevance and cultural appropriateness, serving as an upper benchmark — related
• HAL's text-based human-likeness advantage did not transfer to embodied audiovisual interaction, and measured 3E performance was virtually identical between models — related
• ChatGPT sustains coherent, human-like multi-topic conversation and produces convincing creative content, as shown by the authors' own prompting test — related
• No significant differences emerged between GPT-3.5 Turbo and Claude 3 Opus on affective, collaborative, or performance measures — related
• Cross-model validation shows consistent degradation under multi-strategy attack across Llama-3.3-70B, GPT-4o-mini, and Claude-3.5-Haiku, with significant differences between models — related
• Prompting alone could not keep the gpt-3.5-turbo model on-topic, while the text-davinci-002 engine stayed on-topic in the deployed chatbot — related