Skip to content

Modality Principle

Principle · All principles
Evidence · 37 claims (10 for, 23 mixed, 4 against) · 32 studies (12 causal, 8 review, 7 theoretical, 4 quant-synthesis, 1 associational), q1–q3 · 8 of 32 report an effect size · 32 claims rest on one study

Conditional relationship

This page owns one decision inside Multimedia Learning: once a lesson pairs words with a picture, should those words be spoken or printed? When the words explain a diagram, animation or scene that the learner has to look at while taking the words in, and the learner can follow the spoken language at the pace it is delivered, presenting the words as narration rather than as on-screen text is expected to produce better learning, typically measured on a transfer or comprehension test soon after the lesson. The usual explanation is that printed words and the picture compete for the eyes, while speech leaves the eyes free for the picture. That mechanism is a hypothesis; the evidence below measures outcomes and, in a few studies, reported load.

The relationship is tested, and the claims report six conditions on it:

  • Who controls the pace. In Ginns's meta-analysis the effect was large for system-paced material (d = 0.93) and absent, slightly negative, for self-paced material (d = −0.14). When learners can stop and reread, printed words beside the picture do as well. Mayer's (2010) summary of the literature says the same, and adds speed: the effect "tends to be strongest when the material is fast-paced … and under system control" (modality boundary conditions [~M], a narrative statement with no effect size).
  • How complex the material is. Ginns found the benefit moderated by element interactivity, and Noetel et al. found the multimedia design principles as a group mattered much more in complex media (g = 0.70) than in simple media (g = 0.20).
  • What field the content is in. Science (d = 1.20), engineering and mathematics lessons showed the effect; social-science lessons did not (d = −0.13).
  • How much the learner already knows. In the experiments reviewed by Kalyuga et al. (2003), diagram-plus-narration helped inexperienced trainees, and after further training a diagram-only version did better: the narration had become redundant. The same reversal is reported across worked examples, integrated text and isolated elements (guidance that helps novices can become counterproductive as expertise grows [~M]; for example, three experiments in which electrical trainees moved from needing text on a circuit diagram to doing better without it).
  • How big the effect is once the whole literature is pooled. Mayer's programme reports 17 of 17 experiments with a median d = 1.02; Ginns pooled d = 0.72 across 39 experiments; the largest review, as summarised by Noetel et al., found g = 0.38, and g = 0.20 after correcting for publication bias. Plan with the smaller figures. In Noetel et al.'s comparison of principles, where the average across all multimedia principles was g = 0.38, modality sat with captioning, contiguity and signalling among the moderate-to-large ones (multimedia design principles improve learning on average [+M]).
  • What medium carries the lesson. One meta-analytic contrast found the effect larger with head-mounted VR displays, but a later VR experiment found it reversed: printed text beat narration on recall and comprehension (see the VR row below). The direction in immersive VR is unsettled in the wiki.

How this sits beside its neighbours. The umbrella Multimedia Learning principle decides whether to add a picture at all, and its pattern carries the full lesson policy; this page assumes the picture is already there and is worth looking at. The redundancy decision (whether the spoken words should also appear as on-screen text) is the next question after this one, and the evidence on it is cited here only where it changes the modality choice. The contiguity decision (putting printed words next to the part of the picture they name) is what to do when the words stay printed. Cognitive Load Management is the general account that all of these draw on.

Default design, while the dose is untested

The claims establish the direction and the conditions; none sets a segment length, a narration rate or a point at which to withdraw narration. The steps below follow the evidence where a claim is named and are labelled proposals elsewhere.

  1. Check the condition. Do the words explain something in a picture, animation or scene that must be watched at the same moment? Is the presentation system-paced (video, narrated animation, a lecture with slides), and fast for these learners? (Immersive VR is a separate case; see its row below.) Are the learners new to this content, and is the material complex (many parts acting on each other)? Can every learner hear and follow the spoken language? If the answers are yes, narrate. If the material is learner-paced pages, keep the words printed and placed beside the picture. Basis: pacing moderates the modality effect [~M].
  2. Narrate the explanation, and keep only labels or key terms on screen. Do not also print the narration word for word over the animation. Basis: Mayer's modality principle [+M]; Mayer's redundancy principle [+M] (5 of 5 experiments, median d = 0.72). The harm from verbatim duplicate text is less settled than this step makes it sound: Adesope and Nesbit's meta-analysis (57 effects) found spoken-plus-written words no worse than written alone on average, and better than speech alone for low-prior-knowledge learners, system-paced and picture-free lessons; showing key terms drawn from the narration did better than verbatim text (redundant on-screen text can hurt, but spoken–written redundancy is not harmful on average [~S]). With middle-school learners and animation, adding the written text to narration made no significant difference (redundancy not significant in middle school [~M]; not an equivalence test). So key terms are the default because they do at least as well, not because verbatim text is known to be costly.
  3. Time each sentence to the part of the picture it describes, so the learner hears about a part while it is on screen. Untested proposal on this page; the timing evidence sits with the umbrella's contiguity row.
  4. Segment length, 1–3 minutes of narration per segment, with a pause or a "continue" control at each conceptual break. Untested proposal: the narration studies recorded here are short lessons (a 15-minute immunology animation in Guttormsen Schär & Zimmermann), and no claim sets a length.
  5. Keep in print what learners must consult or copy: formulas, code, definitions, step lists they will follow later. Spoken words disappear. Untested proposal.
  6. Offer a transcript or captions on demand, off by default, for anyone who cannot or prefers not to listen. Untested proposal for content learning; captions do help second-language listening comprehension (captioning [~M]). Where a policy or platform turns captions on for everyone, keep them: the redundancy evidence in step 2 suggests a small cost at most; use condensed captions where the policy allows.
  7. Across sessions, withdraw the narration as expertise grows. Early sessions: narrated, system-paced segments. Once a learner can explain the diagram unaided on a short check (proposal: two consecutive correct explanations of a new "what happens if" question), offer the diagram alone with narration available on request. Basis for the direction: the modality effect reverses with expertise [~M] and the general expertise-reversal claim [~M]; the switching criterion is an untested proposal, and none of these studies says how many sessions the withdrawal takes.
  8. Judge it on the brief's own test, ideally a transfer or explanation question rather than recall of the words, since that is what most of the modality experiments measured.

Fitting the design to a situation

Three facts change the decision most. Ask for them when the brief omits them: who controls the pace (does the video play on, or can learners pause, scroll and reread?); can every learner listen (hearing, language of instruction, a quiet place and headphones?); and how much do the learners already know about this content, and how complex is it?

If the brief says… Then change… Basis
System-paced video, narrated animation or lecture, novice learners, words explain the picture Narrate the words; keep only labels or key terms on screen Spoken narration rather than on-screen text alongside graphics improves learning [+S] (Ginns d = 0.72; Noetel g = 0.38, 0.20 after bias correction); pacing [+M] (d = 0.93 system-paced); strongest when the pace is fast for the learner (boundary conditions [~M])
Learner-paced slides, web pages or an e-book learners scroll through themselves Keep the words printed, beside the part of the picture they describe; do not convert to narration Pacing moderates the modality effect [~M] (d = −0.14 self-paced)
Learners already know this content well, or have used the device or diagram before (advanced) Give the diagram with little or no accompanying text or narration; make narration optional The modality effect reverses with expertise [-M]
Mixed prior knowledge in one group Narrate by default and let more experienced learners mute it or skip ahead Design principle effects were not moderated by prior knowledge across reviews [~M]; the mute option is an untested proposal
Complex material: a process with many interacting parts (circuits, physiology, machinery) Narration is the priority; apply it before lower-value refinements Design principles mattered more in complex media (g = 0.70) than simple (g = 0.20) [+M]
Simple material, few interacting parts Either channel is acceptable; choose by access and cost same [~M]
Content is social science (argument, policy, history) rather than science, engineering or mathematics Do not expect a benefit from narrating; printed words the learner can reread are as good Modality effects differ by field: science d = 1.20, social science d = −0.13 [~M]
Middle-school children (k-12) learning a science unit from animation Narrate; expect a smaller effect than in the adult lab studies. Adding the written words under the narration is not needed, but did no measurable harm in the same study The modality effect held for middle-school students learning electricity, d = .35 [+M] (quasi-experiment); adding written text to narration made no significant difference [~M]
Younger children, still learning to read Speech is the default for any accompanying explanation; untested as a modality comparison at this age. In an e-book whose aim is exposure to print, pair the narration with highlighting or a tracking pointer on the words being read, which draws children's eyes to the text Modality comparison untested; narration with finger-tracking animation directed bilingual preschoolers' looking to the print being read [~M] (attention measured, not learning; the effect stopped when the animation ended)
Adults (postsecondary, workplace training) Narrate system-paced explanations Mayer: 17 of 17 experiments with college learners, median d = 1.02 [+M]; animation or picture with voice beat picture with text for 36 adults [+M]
VR headset or immersive scene Do not assume narration wins. Pilot both channels on the brief's own test; if printed text is used, keep it short and anchored to the object it names rather than floating over the scene Conflicting: the modality effect was larger with head-mounted displays (d = 1.34) than computer screens (d = 0.55) [~M] (a moderator contrast, few studies); in a later VR experiment (61 learners, a unit on HIV) printed text beat narration on recall [-M] and comprehension [-M], with Bayesian tests favouring the reversal (reverse modality effect in VR [-M])
Phone-only, commuting or noisy settings where learners may not listen Make the narrated version the default but ensure the lesson works with captions on and learner pause control untested
Print only (handout, workbook) Modality does not apply; put each sentence next to the part of the diagram it explains untested on this page; see the placement rows of Multimedia Learning
Live teacher with slides The teacher's speech is the narration: show the picture with key terms, and do not read out a slide of full sentences Redundancy principle: animation plus narration beat animation, narration and on-screen text [+M]; carrying it from computer lessons to a live teacher is untested
Learners study in a second language Keep narration for system-paced lessons with pictures, slow it or segment it, and offer captions; for picture-free audio, written text alongside helps captioning L2 video aided comprehension (g = 0.99) [~M]; spoken–written beat spoken-only only when system-paced [~M]; no spoken–written benefit once pictures were present [~M]; where listening itself is a goal, captioned video is reported to improve L2 listening (bimodal captioned input [~W], reported second-hand)
Captions or on-screen text must stay on for everyone (accessibility policy, platform default) Keep them, condensed to key terms where the policy allows; narration stays the main channel for the explanation Spoken–written redundancy is not harmful on average; key terms beat verbatim text [~S]; Mayer: verbatim on-screen text over narrated animation hurt transfer [-M]; middle-school null [~M]
Low literacy Narrate; printed explanation alongside a picture adds a reading load these learners cannot spare untested; the evidence was gathered with readers
Deaf or hard-of-hearing learners Printed words (captions or text) are required; place them next to what they describe and give learners pause control, which removes most of the pacing cost untested; the modality evidence assumes the learner hears the words
Blind or low-vision learners The picture is not the channel; narration must describe what the picture shows, and the modality comparison does not apply untested
A learner says they are an "auditory" or "visual" learner Choose the channel by the material and pacing, not the stated style Matching instruction to learning style does not improve learning [-S]
One teacher for many, or online-self-paced with no teacher Narrated short segments with pause and replay; a short "what happens if" question after each Letting learners pace helped on complex material, while learner control in general showed almost no effect [~M]; the check question is an untested proposal
A single session, or a fixed short time budget Keep the lesson as long as it is; narrating does not lengthen a system-paced lesson. Do not add a test the brief did not ask for untested
The brief's own materials, routine or rubric Keep them as given; change only the channel of the explanatory words, and run the brief's own assessment as given untested
Private or sensitive material, or learners in shared spaces Provide a captioned or transcript route so no learner must play audio aloud untested
A whole task kept whole (a full procedure shown end to end) Narrate within the whole demonstration, segmenting at natural breaks rather than splitting the task untested
High stakes (certification, safety) Use narration for system-paced explanations but plan for the smaller bias-corrected effect (g = 0.20) and pilot on the real assessment before relying on it Spoken narration rather than on-screen text improves learning [+S]

Observation, state and explanation

These rows are proposals, untested with learners as diagnostics.

Response Plausible explanations Observation that could change the interpretation
Learners with the narrated version recall the words but cannot explain the picture The narration ran ahead of the picture; segments were too long; the learner is new to the vocabulary Replay one segment with pauses; ask the learner to point to the part being described; check the technical terms first.
Learners with narration do worse than with printed text The lesson was in fact learner-paced; learners could not follow the spoken language; the learners already knew the content Ask how they used pause; test listening comprehension of the narration alone; check prior knowledge.
Learners repeatedly pause and rewind the narration The narration is too fast or dense; second-language listening; the material is complex Offer captions on demand or a slower segment and compare.
Advanced learners report the narration as tedious The narration has become redundant for them (expertise reversal) Offer the diagram alone and compare their explanation of a new case.

Evidence and qualifications

Taken together: for system-paced lessons in which words explain a picture, narration beats printed text by a moderate amount, plausibly about 0.2 to 0.4 SD once publication bias is allowed for, and larger in science content and complex material. It is not expected for learner-paced material or social-science content, it fades or reverses as learners become expert, and in immersive VR the recorded studies point in opposite directions. Most studies are short computer lessons with adult or older-school learners tested soon after; the wiki holds no claim on delayed retention, on narration in long courses, or on learners who cannot hear or follow the spoken words.

The evidence by question

Further evidence, not yet read against this model

Read against the model on 2026-10-19; the eight bullets that changed it now sit in the conditions, the default design, the situation rows and the evidence list above. These three are relevant but change nothing:

Objective and learner-valued goal

The designer's objective is that learners understand what a picture shows well enough to explain it and use it on a new case. Ask separately what the learner wants and how they will study: a learner who wants a reference to return to and copy from is served by print they can reread, and a learner who studies on a phone on a bus may not be able to listen. Agreement between the two makes narration an easy choice; divergence means offering both routes, with narration the default only where the lesson is system-paced.

What would revise this model?

The expectation should weaken if new system-paced comparisons, with comparable learners and the same words and pictures, find no advantage for narration on transfer, or if the advantage disappears on delayed tests. It should narrow if the field-of-study difference turns out to reflect the kind of picture (a process diagram rather than a photograph) rather than the discipline. It should be extended, not reversed, by studies in long online courses, with second-language learners and with learners who use captions as a matter of course.

Claims

Examples

Key Sources

  • Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15(4), 313–331. doi:10.1016/j.learninstruc.2005.07.001
  • Mayer, R. E. (2008). Applying the Science of Learning: Evidence-Based Principles for the Design of Multimedia Instruction. American Psychologist. doi:10.1037/0003-066x.63.8.760
  • Noetel, M., Griffith, S., Delaney, O., Harris, N. R., Sanders, T., Parker, P., del Pozo Cruz, B., & Lonsdale, C. (2022). Multimedia design for learning: An overview of reviews with meta-meta-analysis. Review of Educational Research. doi:10.3102/00346543211052329
  • Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312–320. doi:10.1037/0022-0663.90.2.312
  • Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive Architecture and Instructional Design: 20 Years Later. Educational Psychology Review, 31, 261–292. doi:10.1007/s10648-019-09465-5
  • Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The Expertise Reversal Effect. Educational Psychologist, 38(1), 23–31. doi:10.1207/S15326985EP3801_4
  • Adesope, O. O., & Nesbit, J. C. (2012). Verbal redundancy in multimedia learning environments: A meta-analysis. Journal of Educational Psychology, 104(1), 250–263. doi:10.1037/a0026147
  • Kutbay, E., & Akpınar, Y. (2020). Investigating modality, redundancy and signaling principles with abstract and concrete representation. International Journal of Education in Mathematics, Science and Technology, 8(2), 131–145. ERIC EJ1255527
  • Guttormsen Schär, S., & Zimmermann, P. G. (2007). Investigating Means to Reduce Cognitive Load from Animations: Applying Differentiated Measures of Knowledge Representation. Journal of Research on Technology in Education, 40(1), 64–78. ERIC EJ826063
  • Albus, P., & Seufert, T. (2023). The modality effect reverses in a virtual reality learning environment and influences cognitive load. Instructional Science. doi:10.1007/s11251-022-09611-7
  • Mayer, R. E. (2010). Seeking a science of instruction. Instructional Science, 38, 143–145. doi:10.1007/s11251-009-9113-x
  • Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors, 40(1), 1–17. doi:10.1518/001872098779480587
  • Yow, W. Q., & Priyashri, S. (2019). Computerized Electronic Features Direct Children's Attention to Print in Single- and Dual-Language e-Books. AERA Open. doi:10.1177/2332858419878126
Page metadata
Field Value
type principle
id modality
title Modality Principle
description When words explain a picture or animation that learners must look at as they hear or read them, presenting the words as speech rather than printed text is expected to improve learning, most of all in system-paced, complex lessons for novices; claims test this (two meta-analyses and a review of 17 experiments), and they report it vanishing when learners set the pace, in social-science content, and as learners gain expertise.
status review
canonical True
generated by claude/unspecified on 2026-10-19
trust unverified — no one has confirmed this page's content
sources 13 — ginns-2005, mayer-2008, noetel-2022, mayer-moreno-1998, sweller-2019, kalyuga-2003, adesope-nesbit-2012, kutbay-2020, guttormsen-schar-2007, albus-seufert-2023, mayer-2010, kalyuga-1998 … and 1 more

Frontmatter is the page's machine-readable record — see the Schema & Guide.