IKS-Instruct fine-tuning is necessary for IKS pedagogical capability: a compact 7B fine-tune reaches a median judge score of 6.39, within 0.15 of a general-purpose reference, while the base model scores near zero on IKS-specific dimensions¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Under a uniform five-judge external panel, all IKS-Instruct fine-tuned 7B versions score in the 6.1 to 6.4 range, whereas the base model scores near zero on the IKS-specific dimensions. → Shwetha Singaravelu 2026
Evidence¶
Shwetha Singaravelu 2026¶
Shwetha Singaravelu, Gayathri Muruganantham, Lakshmi Rajendran, Santhosh Sivasubramani. (2026). IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems. arXiv. https://arxiv.org/abs/2607.23322 doi:10.48550/arxiv.2607.23322
q2 · i? · design · r2
Model comparison in Section 5.3 using a uniform five-judge external panel over 1,201 stratified items, with the base model evaluated only in an earlier small-scale pilot. The fine-tuned models score "in the 6.1 to 6.4 range" while the base model scores near zero on IKS-specific dimensions; no effect size is printed.
"all fine-tuned versions score in the 6.1 to 6.4 range under the uniform external panel, whereas the base model (Airavata without IKS fine-tuning), evaluated in an earlier small-scale pilot, scores near zero on the IKS-specific dimensions"
Discussion¶
Related Claims¶
- Fine-tuning Llama2-7B with parameter-efficient methods yields unsatisfactory results for measuring subject-matter teaching practices, only marginally improving the majority baseline — related
- Model-level judge quality is non-monotonic in dataset size: the 13,882-pair v1.1 yields the best model (6.39), while both expansion to ~20,000 pairs (6.33) and aggressive curation to 7,130 pairs (6.12) reduce quality — related
- Education-oriented instruction tuning consistently improves curriculum grounding, K-12 problem solving, and pedagogical tutoring at every evaluated model scale — a broader claim this one bears on
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
iks-instruct-finetuning-necessary-for-iks-pedagogical-capability |
title |
IKS-Instruct fine-tuning is necessary for IKS pedagogical capability: a compact 7B fine-tune reaches a median judge score of 6.39, within 0.15 of a general-purpose reference, while the base model scores near zero on IKS-specific dimensions |
description |
IKS-Instruct fine-tuning is necessary for IKS pedagogical capability: a compact 7B fine-tune reaches a median judge score of 6.39, within 0.15 of a general-purpose reference, while the base model scores near zero on I... |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — shwetha-singaravelu-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.