User trial participants rated the C3-IoC system positively for usability, with the Job roles map and Job roles overview the highest-rated components¶

Claim · All claims
Evidence · 1 study (2 entries) · design r3 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Participants provided positive usability scores; 23 of 45 statement pairs in Q1 showed strong correlations (rs > 0). → Adán José-García 2023 q2 i? All 21 statement pairs across the seven components showed positive correlations (rs > 0.25), with the Job roles map and Job roles overview highest rated. → Adán José-García 2023 (2)

Evidence¶

Adán José-García 2023¶

Adán José-García, Alison Sneyd, Ana Melro, Anaïs Ollagnier, Georgina Tarling, Haiyang Zhang, Mark Stevenson, Richard Everson, Rudy Arthur. (2023). C3-IoC: A Career Guidance System for Assessing Student Skills using Machine Learning and Network Visualisation. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-022-00317-y

q2 · i? · design · r3

Online user trial survey of 64 valid responses (N = 64), October to December 2020, mostly aged 19–24 (72%) and in higher education (80%). The article reports 23 of 45 pairs with "correlations (rs > 0)"; no precise printed effect size value is given, so impact is null.

"Overall, the participants provided a positive score in terms of usability of the C3-IoC system. From the 45 statement pairs in Q1, illustrated in Fig. 11a, 23 pairs showed strong correlations (rs > 0), indicating that the tool was easy to use, the functions were well integrated, it showed consistency, and the user felt confident using the tool."

Adán José-García 2023 (2)¶

Adán José-García, Alison Sneyd, Ana Melro, Anaïs Ollagnier, Georgina Tarling, Haiyang Zhang, Mark Stevenson, Richard Everson, Rudy Arthur. (2023). C3-IoC: A Career Guidance System for Assessing Student Skills using Machine Learning and Network Visualisation. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-022-00317-y

q2 · i? · design · r3

Spearman correlation heatmaps of usability for the seven C3-IoC components from the same N = 64 trial. All 21 pairs showed "positive correlation value (rs > 0.25)" with the map and overview components rated highest; the article prints a threshold, not an exact effect size value, so impact is null.

"we observe a strong relationship between all the 21 different statement pairs having all of them a positive correlation value (rs > 0.25). Overall, the highest-rated components were the 'Job roles map' and the 'Job roles overview', which may suggest an interest in visualising or getting an overview of the job roles."

Discussion¶

Related Claims¶

• In the user trial, strong Spearman correlations (rs > 0.7) linked career exploration with identifying learning opportunities, and perceived career value with personal development value — related
• C3-IoC job role matching accuracy increases monotonically with the number of questionnaire questions answered, reaching about 80% accuracy with four questions and nearly 100% with 24 — related
• Exact-match (k=1) job role prediction requires approximately 24 questionnaire questions to achieve an average success rate of 90%, while at least six questions yield a top-10 success rate above 70% — related
• A pilot mixed-method trial with 71 participants found students valued the system along three dimensions: career exploration, personalisation, and skill awareness/self-reflection — related