TAM-grounded post-session survey instrument with five domains and 15 open-ended questions¶

Element · All elements
Evidence · 2 claims (2 for) · 1 study (1 associational), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description¶

A post-session Qualtrics survey completed by the same nine instructors was grounded in the Technology Acceptance Model and structured into five domains: perceived usefulness of advanced metrics and predictive scores, usefulness and structure of written feedback comments, interpretability of visual feedback displays, usefulness of instructional customization features, and behavioral intention and perceived feasibility for classroom integration. Closed items used 6-point Likert-type scales, with one reverse-coded time-burden item, plus "15 open -ended questions designed to elicit elaborated feedback on WAT-C's metrics, feedback structure, user interface, and instructional alignment" used to triangulate the qualitative themes.

Design Implications¶

Context¶

Requirements¶

• Administration immediately after the focus group sessions with the same participants who evaluated the prototype

Constraints¶

• The complete instrument is available only through the project's OSF repository link

Target Learners¶

• College writing instructors evaluating an automated writing evaluation tool

Target Learning Goals¶

• Assessing perceived usefulness, usability, interpretability, and adoption intention for instructional technology

Claims¶

• Adult ESL learners report above-neutral perceptions of ASR as a writing tool on usefulness, ease of use, and intention to use [+W]
• Among TAM constructs, result demonstrability, intention to use, and ease of use received the highest ratings while output quality was lowest [+W]

Related Elements¶

• WAT-C classroom writing analytics platform — the platform the survey evaluates

Examples¶

-

Key Sources¶

• Potter, A.; Serhan, Z.; Patne, N. A.; Öncel, P.; Ahmed, I.; Arner, T.; Islam, R.; Roscoe, R. D.; Allen, L. A.; Crossley, S. A.; McNamara, D. S. (2026). Human-AI Collaboration for Qualitative Analysis in Participatory Design: Refining the Writing Analytics Tool. Journal of Educational Data Mining, Volume 18, No 1. https://osf.io/8j9y6/?view_only=3a21ed900475418a8c258ea595cbe53b