Skip to content

Criterion- and Norm-Referenced Testing

Principle · All principles
Evidence · 35 claims (10 for, 22 mixed, 3 against) · 23 studies (10 theoretical, 7 associational, 3 design, 2 causal, 1 review), q1–q3 · 0 of 23 report an effect size · 35 claims rest on one study

Conditional relationship

This page owns the choice of reference frame for a score: whether a result should be read against a defined standard (criterion-referenced), against the performance of a norm group (norm-referenced), or broken into components (diagnostic), given the decision the result will inform. The relationship is about assessment-accuracy, whether the decision made from a score is the one the learner's capability warrants, and only through that about learning.

For learners of any age and expertise (child to adult, novice to advanced) whose results will inform a decision, a decision is expected to be more accurate when the kind of score matches the question asked:

  • "Can this learner do X yet, and what should be taught next?" is answered by a criterion-referenced report: items that represent the skills of a domain defined before the test was written, a cut set in advance by a stated procedure, and a result reported per skill as mastered or not yet. A percentile cannot answer this question, because it says only where the learner stands among others.
  • "How does this learner compare with a wider population?" (selection for limited places, a national comparison, an unusual-score flag) is answered by a norm-referenced score, read against a norm group that resembles the learner and with its measurement error, and never as the only evidence for a gate.
  • "Why is this learner struggling?" is answered by neither alone, but by a diagnostic profile of component skills (for reading: word recognition, comprehension, fluency) combined with classroom observation and records.

The relationship is conditional on three things a page cannot settle in advance: whether the criterion-referenced items really sample the defined skills (rather than being chosen because they spread learners out), whether the cut was set defensibly and the learners near it are treated as uncertain, and whether the norm group fits the learner being compared.

No claim in the wiki tests this relationship. None compares instructional or selection decisions made from criterion-referenced reports with decisions made from norm-referenced scores, or measures whether matching the frame to the decision improves anything. The claims that bear on it are of four kinds: definitional and design arguments from the early criterion-referenced literature (Pikulski 1973 on reading measurement; Nitko 1970; Gentile 1971 on grading; Hunt 1973 on mental-age and IQ metrics), none with data; a simulation and operational-data comparison of decision-consistency estimates around pass/fail cuts (Wolkowitz & Smith 2024); a single-state standard-setting report for an alternate portfolio assessment (Olson et al. 2002); and congressional testimony and reviews on norm-referenced selection tests (SAT, gifted identification). Everything below is a default design built from the earlier page's guidance, each step labelled with its evidence status.

How the converted siblings sit beside this: Competency-Based Assessment owns the "demonstrated / not yet" threshold judgment, assessor agreement and threshold setting for a criterion-referenced decision, and this page does not repeat its model; Mastery Learning owns the gate that such a judgment controls (correct, recheck, advance); Summative Assessment owns the end-point judgment's design (sampling, timing, stakes) and already sets its cut against the capability; Validity, Reliability and Bias in Classroom Assessment owns whether any score supports the decision drawn from it; Standardized Test Fairness and Bias owns whether a standardized score predicts and decides equally well across groups. This page sits before all of them: it decides which kind of score to build, buy or consult for a given decision, and how to read it.

Default design, while the relationship is untested

A procedure for a teacher, programme lead or assessment designer who must decide which scores to use and how to read them, run over a term rather than one lesson. Each step's evidence status is beside it.

  1. Write down the decision before choosing or reading a score. Name what the result will decide (what to teach next, who advances, who gets a limited place, whether to refer for assessment of a disability or giftedness, what to report to a school board) and who will act on it. Then classify it: mastery or next teaching (criterion-referenced), relative standing or ranked selection (norm-referenced), or explaining a difficulty (diagnostic). Untested; from the earlier page's requirement that the test type match the decision.
  2. For a criterion-referenced check, define the skills first and write items to represent them. List the skills of the domain, in the order they are taught, and write several items per skill that sample it; do not split the domain into so many tiny skills that the check stops representing the capability. Basis: Pikulski's (1973) recommendation that items represent skills essential to learning to read and be arranged in the hierarchy used for teaching (items should represent essential skills in a teaching hierarchy [+W]), an author's recommendation for reading tests, not a tested design.
  3. Do not drop items because they fail to spread learners out. On a criterion-referenced check, an item that nearly everyone gets right after teaching is doing its job; removing it to raise discrimination changes what the test measures. Basis: Nitko's (1970) argument that screening items for desirable statistical characteristics changes the definitions of the behavior categories a criterion-referenced test represents (statistical item selection undermines criterion-referenced definitions [+W]), an argument with no data.
  4. Report per skill, as mastered or not yet, with what the learner did. Anchor the scale at mastery of the defined skills at the top and their absence at the bottom, not at other learners' scores. Basis: Pikulski's (1973) definitional account (criterion-referenced tests indicate whether a skill has been achieved [+W]); Gentile's (1971) argument that rank-ordering does not serve the teacher's task of identifying who has and has not reached the objectives (rank-ordering irrelevant to teaching [+W]).
  5. Set the cut by a stated procedure before scoring, and treat learners near it as uncertain. If a panel sets it, have each panelist classify real work samples independently and compute the cut from each work sample's mean classification, which is less swayed by one deviant rating; record how often panelists agree. Then, for every learner within a band of the cut (as a proposal, one item per skill either side, or the test's reported standard error of measurement), gather one more observation before acting. Basis: in one state's alternate portfolio standard setting, a mean-classification method was more robust to outliers than a category-mean method (mean-classification cut-score method more robust [~M]) and panelists' classification agreement ranged from 71% to 97% across grades and subjects (body-of-work agreement 71% to 97% [~M]); decision-consistency estimates are lowest when the cut sits at the peak of the score distribution, where inconsistent decisions are most likely (DC lowest at the distribution peak [~M]). None of these tests the band itself.
  6. Check that the classification means what it says. Once a term, take a small sample of learners classified "mastered" and "not yet" on a skill and watch each do a real task that uses it; if the classification and the task disagree often, revise the items or the cut. Basis: Nitko's (1970) argument that criterion-referenced tests need empirical evidence that their task classes reflect the competence of interest (criterion-referenced tests need construct validation [+W]); the sample check is an untested proposal.
  7. For a norm-referenced score, find out who the norm group is before reading a percentile. Look up in the technical manual where, when and from whom the norms were drawn (state or national, year, ages, language background), and compare that with your learner. Report the percentile as a band, not a point, and do not use it to plan teaching. Untested; from the earlier page. A related argument: Hunt (1973) holds that mental-age and IQ metrics hide the structure of developing abilities (norm-referenced metrics hide structural development [-W]), an argument with no data.
  8. Never let one norm-referenced score decide entry. Combine it with at least one other piece of evidence the decision rests on (grades, a work sample, a classroom-learning task), and look again at every learner near the cut. Basis: 1987 congressional testimony that girls averaged lower SAT scores than boys despite higher high-school and college grades (SAT underpredicts girls' performance [~W]) and that MIT found women admitted with lower SAT mathematics scores did as well in freshman mathematics (MIT lower SAT math, equal performance [~W]), both testimony with no data printed; a review's statement that widely used identification tests cannot fully determine a student's learning potential (identification tests cannot fully determine learning potential [~W]). The fairness side of this step is on Standardized Test Fairness and Bias.
  9. Grade classroom work against criteria, not on a curve, and still assess every learner. Fix the standard for each grade in advance; do not reserve a share of top grades. Basis: Gentile's (1971) arguments that competitive grading sets students against one another and makes cooperation risky (competitive grading pits students against each other [-W]) and that abandoning assessment is not an acceptable alternative (non-assessment unacceptable [+W]); the same essay argues that adjacent scores either side of a grade cutoff are not significantly different (grade cutoffs arbitrary [~W]), which is a further reason for step 5's band. All three are arguments, not results.
  10. For a diagnostic question, build a profile, not a single score. Use an individually administered diagnostic test of component skills, together with classroom observation, records and an interview with the learner (and family for a child), before any referral. Where a disability determination is at stake, the earlier page notes that it usually requires both an aptitude measure of general cognitive functioning and an achievement test of specific content. Untested here; from the earlier page.
  11. When a classroom grade and a standardized score disagree, treat the gap as a question. List the explanations (grading-criteria mismatch, curriculum not aligned with the test, unfamiliar item formats, anxiety, an off day) and gather one observation that would tell them apart before revising either the grade or the view of the learner. Untested; from the earlier page.
  12. Where an audience needs both, report both. A skill-level mastery result for the teacher and learner, and, where a school or family asks how the learner compares, the percentage of a stated group reaching that level. Untested; from the earlier page's note that some newer tests blend the two.

Across a term (dose and progression; all quantities are untested proposals).

  • Week 1: a criterion-referenced baseline on the term's skills, 20 to 30 minutes, three to five items per skill, scored per skill. If a standardized test is required, note its date and the decision it feeds, and look up its norm group now.
  • Weeks 2 to 10: a 10- to 15-minute criterion-referenced check every two to three weeks on the skills just taught plus one earlier skill. Early in the term the teacher scores and explains each item against the criteria with an exemplar; by mid-term learners score their own checks against the same criteria and the teacher re-scores a sample of about one in five to keep both calibrated.
  • After a "not yet": correction on that skill, then a re-check on a parallel set of items within a week. A learner moves on from a skill when they reach the cut on two separate checks (as a proposal, four of five items each time), which is the gate Mastery Learning models.
  • End of term: a criterion-referenced report per skill for teaching decisions; a norm-referenced score only where a decision needs relative standing, at most once a year, read with its norm group and band. Once a term, run step 6's sample check of what "mastered" means.

When a condition is not met: if you cannot change the test (a state or national examination), the steps still open to you are 1, 5's band (second evidence near the cut), 7, 8, 11 and 12; if no technical manual is available for a norm-referenced test, report the percentile as unverified and do not use it for a gate.

Fitting the design to a situation

The three facts that most change the decision:

  • What decision the result will inform, and who acts on it. A teacher choosing what to teach next needs a skill-level criterion-referenced report; an admissions panel with limited places needs relative standing plus second evidence; a referral team needs a diagnostic profile. If unstated, ask: what will be decided from this result, by whom, and how soon?
  • Whether you can change the test, or only how its scores are read. A teacher writing their own checks can define skills and set the cut; a user of a state, national or commercial test can change only what is combined with the score and how near-cut learners are handled. If unstated, ask: who owns the test, and can you see its technical manual (norm group, standard error, how the cut was set)?
  • Who the learners are relative to the norm group and the defined skills. A percentile is only as meaningful as the norm group is like the learner; a criterion-referenced check is only as useful as its skills match what this learner is being taught. If unstated, ask: what are the learners' ages, language backgrounds and prior schooling, and what curriculum or job tasks define the skills?
If the brief says… Then change… Basis
The decision is what to teach next for novice learners early in a domain Use short criterion-referenced checks on the first skills in the teaching order, three to five items per skill; report mastered / not yet per skill; no percentiles Items should represent essential skills in a teaching hierarchy [+W] (an argument about reading tests, carried to other domains)
Learners are advanced or near the top of the range Add items at the upper skills of the hierarchy, or a broader task, so the check is not at ceiling; if relative standing matters (an enrichment place), use a norm-referenced score from an above-level test and a work sample untested proposal
Learners are young children (child) in screening or readiness testing Do not use one sitting for a placement; use an individually administered criterion-referenced check with pictures and spoken instructions over two or three occasions; re-check within the year untested proposal
The decision is selection for limited places (a programme, a scholarship, admission) Use the norm-referenced score with at least one other piece of evidence; look again at everyone near the cut; check the norm group fits the applicants SAT underpredicts girls' performance [~W]; MIT lower SAT math, equal performance [~W] (1987 testimony, no data printed)
The decision is gifted or special-education identification Build a diagnostic profile: a norm-referenced aptitude or ability measure plus achievement, classroom observation and a classroom-learning task; screen all learners rather than relying on referral; compare identification rates by group with enrolment Identification tests cannot fully determine learning potential [~W] (a review section); diverse students under-identified nationwide [~W] (an issue brief); the dual-measure requirement is the earlier page's
You are writing your own criterion-referenced test Define skills before items; keep items that nearly everyone passes after teaching; set the cut by a stated panel or rule before scoring; check a sample of classifications against real tasks once a term Statistical item selection undermines criterion-referenced definitions [+W]; criterion-referenced tests need construct validation [+W] (both Nitko 1970, arguments)
Most learners score close to the cut (a pass/fail examination where the cut sits at the most common score) Expect the least consistent decisions there: give near-cut learners a second, parallel check before deciding, or report "borderline" as its own category DC lowest at the distribution peak [~M] (simulated data, a measurement property, not a test of the remedy)
A panel will set the cut (a programme standard, a licensing threshold) Use independent classification of real work samples, compute the cut from each sample's mean classification, and report panelist agreement with the cut Mean-classification method more robust to outliers [~M]; agreement 71% to 97% [~M] (one state's alternate portfolio assessment for students with significant disabilities, carried to other assessments)
Learners take the test in a second language, or have low literacy Do not read a low percentile as low subject knowledge; check the norm group included learners with this language background; add a low-language criterion-referenced check (spoken, pictorial, or practical) on the same skills untested proposal
Learners have low confidence or are anxious about tests Report criterion-referenced results privately and per skill ("not yet" with the next step), never as a rank; keep norm-referenced results out of routine feedback untested; the reason not to rank is Gentile's argument (competitive grading pits students against each other [-W])
Results are private or sensitive (a diagnostic profile, a disability assessment) The learner (or family, for a child) decides who sees the full profile; share with teachers only the skills to teach and the supports agreed; never post or rank untested proposal
Adults in work settings: licensing, certification or job entry Use a criterion-referenced judgment on tasks sampled from the job, with the cut set before scoring; use a norm-referenced score only if places are limited, and then with a work sample untested proposal; the threshold model is Competency-Based Assessment's
The brief wants a whole-task performance kept whole (a recital, a clinical scenario, a product for a real audience) Judge the whole performance against criteria the audience would recognize; do not split it into paper items to get skill scores; record per-criterion notes from the one performance instead untested proposal
The brief names its own rubric, categories or reporting levels Keep them as the criteria; change only what the evidence here says matters: set the levels before scoring, report per criterion rather than as a rank, and take a second look at learners on a level boundary untested; the three changes follow steps 4, 5 and 9
A fixed short time budget (one 15-minute slot per check) Check two or three skills with three items each rather than every skill thinly; rotate skills across checks untested proposal
Only a single session is available (a one-off placement) Use a criterion-referenced placement check on the entry skills, 20 to 30 minutes; treat near-cut results as provisional and re-check in the first week of teaching untested proposal
Mixed modes: some learners in the room, some remote, some on phones only Give the same items, time and aids in every mode, or record the difference beside each result; keep items short and single-screen for phones; compare results by mode before pooling untested proposal
One teacher for many learners, or facilitator time is scarce (volunteers, a short weekly session) Learners self-score short checks against an answer key with worked reasons; the teacher spends contact time only on near-cut learners and the skill most learners have not mastered untested proposal
Results are reported for accountability (school, district or programme reports) Report the percentage reaching each level with the number tested, and the level definitions; do not compare schools on a norm-referenced percentile alone; see High-Stakes Testing and Accountability Effects untested proposal
A parent or learner asks why a classroom grade and a test score disagree List the candidate explanations and gather one observation that separates them (a task in the test's format, a look at what the test covers) before changing either untested; from the earlier page
Self-paced mobile study for school-age learners with no live teacher Teach the difference with short worked examples and a tap-to-sort activity (statements sorted into "tells you what I can do" and "tells you how I rank"), and assess with auto-scored items against a stated criterion; do not ask learners to write test items on a phone untested proposal

Nine of the twenty rows cite a claim, and none of those claims tests the move in its row: they are arguments from the 1970s criterion-referenced literature, one state's standard-setting report, a measurement simulation, 1987 congressional testimony and a review section, each carried to the decision in its row. The other eleven rows are untested proposals.

Observation, state and explanation

What can be observed is a learner's response to items, a skill-level classification, a percentile, a classroom grade and, later, performance on a real task. "Has mastered fractions" or "is below average" are inferences from those, with error.

Response Plausible explanations Observation that could change the interpretation
A learner is classified "not yet" on a skill by one or two items Has not learned the skill; a slip or misread item; the items sample a narrower version of the skill than was taught A parallel set of items on the same skill a few days later; ask the learner to explain one item aloud.
A learner scores just above or just below the cut True capability near the standard; ordinary measurement error, which is largest where scores cluster A second, parallel check before acting; compare with a real task.
Most learners are "mastered" on a skill and the item statistics look poor (little spread) Teaching worked and the items are fine; the items are too easy for the skill Watch a sample of "mastered" learners use the skill in a task; if they cannot, the items are too easy.
A low percentile on a norm-referenced test for a learner who does well in class The norm group differs from the learner (language, age, schooling); the test samples content the class has not covered; format unfamiliarity or anxiety; classroom grading is lenient Check the norm group in the manual; compare the test's content with the curriculum; give a criterion-referenced check on the same content in a familiar format.
A high percentile but "not yet" on several classroom skills The norm group is lower-performing than the learner's peers; the test samples general ability more than the taught skills; the classroom checks are harder than the standard Compare item content; ask what the learner does on the skills the test covers.
Two panelists or teachers place the same work in different levels The level descriptions are vague; one rater is more lenient; the work is genuinely borderline Score common samples together and compare reasons; check whether disagreements cluster on one level boundary.

These rows are proposals, untested as diagnostics. Each probe is a further measurement with its own error, so record it and keep the explanations that remain open.

Evidence and qualifications

No claim in the wiki compares decisions made from criterion-referenced and norm-referenced scores, or tests any step of the default design. The claims below bear on parts of it.

The claims differ in kind (definitional arguments, a simulation, a standard-setting report, testimony), in era (1970 to 2024) and in setting (clinical reading evaluation, university grading, alternate assessment for students with significant disabilities, college admission), so do not rank them. None measures whether a decision became more accurate.

The evidence by question

Further evidence, not yet read against this model

The earlier page cited no claim pages. These bear on neighbouring questions:

Objective and learner-valued goal

The designer's objective here is an accurate decision: the right next teaching, a fair selection, a referral that finds the learners who need it. Learners often value something else: knowing what they can already do and what to do next, not being ranked in front of peers, gaining a place or a credential, or avoiding a label. Ask the learner (or the family, for a child) what the result is for, from their side. Where they want to know what to do next, a criterion-referenced report per skill serves both purposes; where they want a place and the place is rationed, a norm-referenced score is unavoidable, and the learner is owed the second look near the cut. Where a diagnostic label is at stake, the learner's or family's goal (support, or avoiding a label) decides who sees the profile, and divergence between that goal and the school's should be recorded, not resolved by assumption.

What would revise this model?

The model should weaken if teachers given skill-level criterion-referenced reports chose next teaching no better, or learners learned no more, than with norm-referenced percentiles on the same content; if criterion-referenced "mastered" classifications agreed no better with later performance on real tasks than percentiles did; or if a second check near the cut changed few decisions. It should strengthen if a comparison found that matching the score's frame to the decision improved later performance on the skills taught or the accuracy of selection by group. A study comparing decision consistency for different cut placements on classroom checks, or one testing whether learners' self-scoring against criteria stays calibrated across a term, would settle two of the proposals above.

Claims

Examples

Key Sources

  • Linn, R. L., & Miller, M. D. (2005). Measurement and assessment in teaching (9th ed.). Pearson.
  • Popham, W. J. (2004). America's "failing" schools: How parents and teachers can cope with No Child Left Behind. RoutledgeFalmer.
  • Haertel, E. H., & Herman, J. L. (2005). A historical perspective on validity arguments for accountability testing. In J. L. Herman & E. H. Haertel (Eds.), Uses and misuses of data for educational accountability and improvement (pp. 1-34). National Society for the Study of Education. doi:10.1111/j.1744-7984.2005.00023.x
  • Arduini-Van Hoose, N. (2020). Standardized tests. In Educational psychology. Retrieved from https://edpsych.pressbooks.sunycreate.cloud. CC BY-NC-SA 4.0.
  • Nitko, Anthony J. (1970). Criterion-Referenced Testing in the Context of Instruction. https://eric.ed.gov/?id=ED047010
  • Katz, I. R., Martinez, M. E., Sheehan, K. M., & Tatsuoka, K. K. (1993). Extending the Rule Space Model to a Semantically-Rich Domain: Diagnostic Assessment in Architecture. Educational Testing Service. https://eric.ed.gov/?id=ED366652. doi:10.21236/ada273789
Page metadata
Field Value
type principle
id criterion-and-norm-referenced-testing
aliases base-attributes-on-cognitive-model
title Criterion- and Norm-Referenced Testing
description A decision about a learner is expected to be more accurate when the score matches the question asked: a criterion-referenced report against skills defined in advance for what a learner can do and should be taught next, a norm-referenced score read against a fitting norm group (and never alone) for relative standing and selection, and a diagnostic profile for why a learner struggles; no claim here tests the relationship.
status review
moved_from pattern/four-characteristics-criterion-referenced-tests
canonical True
generated by claude/unspecified on 2026-10-05
trust unverified — no one has confirmed this page's content
sources 2 — nitko-1970, katz-1993

Frontmatter is the page's machine-readable record — see the Schema & Guide.