Item-to-factor loading patterns qualitatively differ between human and LLM EFA solutions¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Under the Kaiser-retained solution, items loading on one LLM factor load across two or three different human factors on both instruments. → Alona Strugatski 2026

Evidence¶

Alona Strugatski 2026¶

Alona Strugatski, Licol Zeinfeld, Giora Alexandron. (2026). Do Assessment Instruments Measure the Same Thing for Humans and LLMs? A Latent Structure Analysis. https://arxiv.org/abs/2608.15630 doi:10.48550/arxiv.2608.15630

q2 · i? · associational · r2

Qualitative visual analysis of EFA loading structures (Figure 1) for both instruments under the Kaiser solution; RMSEA values are printed for each group (e.g., chemistry human 0.019, LLM 0.045). The article states parallel-analysis solutions also revealed dissimilar item-factor structure.

"For instance for Chemistry, the third LLM factor, Lk3 (L: LLMs, K: Kaiser, Factor: 3), has loadings larger than 0.5 on Q5, Q16, and Q19. On the human side, these items are loaded on two different factors: Hk1 and Hk2."

Discussion¶

Related Claims¶

• Parallel analysis yields different factor counts for humans and LLMs on both instruments — related
• LLM-human factor-structure similarity is consistently lower than human-human similarity on both assessments — related
• Exploratory factor analysis in the pilot stage supported a four-factor solution, with three items removed (a weak loading, a cross-loading, and a low item–total correlation with duplicated content) — related