KSA three-axis evaluation framework with a four-level Skills hierarchy (Center→Role→Scenario→Sub-scenario)

Research Method · All research methods
Evidence · 2 claims (2 for) · 1 study (1 design), q2 · 0 of 1 report an effect size · 2 claims rest on one study

Description

The framework operationalizes educational LLM capability using the KSA (Knowledge-Skills-Attitude) model from educational assessment. The article states: "The Skills domain comprises 122,425 items across 6 Centers, 11 Roles, 46 Scenarios, and 81 Sub-scenarios." Knowledge covers curriculum-based content and pedagogical knowledge; Skills measures situated capability through tasks emulating real educational decisions, with difficulty tiers from Bloom's and Krathwohl's taxonomies; Attitude examines epistemic, instructional, and normative-ethical stances plus deception resistance via Alignment Faking. The hierarchy contextualizes tasks by institutional function (Center), professional responsibility (Role), task type (Scenario), and task variant (Sub-scenario).

Accounts

• KSA three-axis evaluation framework with a four-level Skills hierarchy (Center→Role→Scenario→Sub-scenario): The framework operationalizes educational LLM capability using the KSA (Knowledge-Skills-Attitude) model from educational assessment. The article states: "The Skills domain comprises 122,425 items across 6 Centers, 11 Roles, 46 Scenarios, and 81 Sub-scenarios." Knowledge covers curriculum-based content and pedagogical knowledge; Skills measures situated capability through tasks emulating real educational decisions, with difficulty tiers from Bloom's and Krathwohl's taxonomies; Attitude examines epistemic, instructional, and normative-ethical stances plus deception resistance via Alignment Faking. The hierarchy contextualizes tasks by institutional function (Center), professional responsibility (Role), task type (Scenario), and task variant (Sub-scenario). (Unggi Lee et al. (2026))

Claims

• Content knowledge and pedagogical knowledge dissociate in frontier LLMs; Claude-Opus-4.5 shows a 19.8 percentage point gap (66.3% content vs. 86.1% pedagogical) [+M]
• Across frontier LLMs, Counseling is the easiest educational activity center (average 8.78) while Assessment and Research are hardest (8.42), and performance degrades from Easy to Hard difficulty [+M]

Related Research Methods

-

Key Sources

• Unggi Lee, Sookbun Lee, Heungsoo Choi, Jinseo Lee, Haeun Park, Younghoon Jeon, Sungmin Cho, Minju Kang, Junbo Koh, Jiyeong Bae, Minwoo Nam, Juyeon Eun, Yeonji Jung, Yeil Jeong. (2026). OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models. https://arxiv.org/abs/2601.13882

Page metadata

Field	Value
type	research-method
id	ksa-three-axis-evaluation-framework-with-a-four-level-skills-hierarchy-centerrolescenarios
title	KSA three-axis evaluation framework with a four-level Skills hierarchy (Center→Role→Scenario→Sub-scenario)
description	The framework operationalizes educational LLM capability using the KSA (Knowledge-Skills-Attitude) model from educational assessment.
status	review
generated	by process:settle-candidates on 2026-10-11
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.