LittleCurriculum

Product or Programme · All products and programmes
Evidence · 4 claims (4 for) · 1 study (1 causal), q2 · 0 of 1 report an effect size · 4 claims rest on one study

Description

LittleCurriculum is an 88B-token K–5-filtered pretraining corpus created by Fanfei Li et al. from FineWeb-Edu and released with the LittleLearner model for studying language-model knowledge acquisition under controlled exposure.

Components

Claims

• The multi-stage filtering pipeline reduces Beyond-K-5 retention to near zero while preserving a minority of K-5 content [+M]
• LittleLearner's capabilities tightly track its controlled exposure: strong on in-scope tasks, degrading beyond the K-5 boundary [+M]
• Across scaling, post-training, and in-context learning, the pretraining filter rather than the intervention sets LittleLearner's capability ceiling [+M]
• Scaling model size improves performance within K-5 and at the boundary but does not recover Beyond-K-5 performance [+M]

Related Products and Programmes

• LittleLearner

Key Sources

• Fanfei Li, Jana Zeller, Manuel Prada-Corral, Thaddäus Wiedemer, Prasanna Mayilvahanan, Ryan Cotterell, Wieland Brendel. (2026). LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure. Preprint. https://arxiv.org/abs/2608.13545

Page metadata

Field	Value
type	product
id	littlecurriculum
title	LittleCurriculum
description	LittleCurriculum is an 88B-token K–5-filtered pretraining corpus created by Fanfei Li et al.
status	review
product_kind	dataset
generated	by process:settle-candidates on 2026-10-11
trust	unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.