Retrieval augmentation raises path Validity but not the joint Final Pass rate¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? RAG raises Qwen3-4B-2507 Zero-shot Validity from 39.7% to 73.8% and Adaptivity from 12.5% to 19.3%, but Efficiency falls from 25.8% to 18.4% and Final Pass changes from 3.2% to 2.9%. → Yu Liu 2026

Evidence¶

Yu Liu 2026¶

Yu Liu, Zeming Liu, Tianle Zhang, Zihao Cheng, Yuhang Guo, Kehai Chen, Min Zhang, Yunhong Wang, Haifeng Wang. (2026). PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning. arXiv. https://arxiv.org/abs/2609.18861 doi:10.48550/arxiv.2609.18861

q2 · i? · causal · r2

Controlled comparison of Qwen3-4B-2507 Zero-shot with and without retrieval augmentation (top-3 knowledge chunks via BAAI/bge-small-zh-v1.5 and FAISS) on 1,000 Basic Education personas, reported in Table 5. The article reports the printed percentage-point changes; no effect size is printed.

"RAG raises Validity from 39.7% to 73.8%, a gain of 34.1 percentage points, and Adaptivity from 12.5% to 19.3%. Efficiency decreases from 25.8% to 18.4%, while Final Pass changes from 3.2% to 2.9%."

Discussion¶

Related Claims¶

• Contextual noise from irrelevant textbooks broadly degrades KC planning, most in Validity and Efficiency — related
• One-shot path generation raises Validity but lowers Adaptivity and Efficiency relative to step-by-step interaction — related
• Chain-of-Thought prompting improves KC planning for Qwen models but its effect varies by model and educational stage — related
• Strongest LLMs reach only about 29.5% final pass rate on Basic Education KC planning and drop to 14.6% on Higher Education — related
• Adaptivity to individual learners is the primary bottleneck for LLMs in KC path planning — related