Composition Projects Multimodal¶

Strategy · All strategies
Evidence · 10 claims (8 for, 2 against) · 28 studies (11 quant-synthesis, 10 causal, 4 review, 2 theoretical, 1 associational), q1–q4 · 10 of 28 report an effect size · 1 claim rests on one study

Description¶

Multimodal composition projects ask learners to produce original artifacts that deliberately combine modes of meaning — writing, visuals, audio, video, gesture, and spatial layout — rather than text alone. Learners plan, draft, and revise across modes, making rhetorical decisions about which mode best serves their purpose and audience. The composition process, not just the artifact, is the site of learning.

Design Implications¶

Multimodal composition leverages the productive overlap of verbal and visual representation: presenting and producing content in complementary modes supports encoding and recall Combining words and visuals improves learning over words alone. [+S]. Because learners choose modes, media, and framing, the work builds ownership and intrinsic motivation Supporting learner autonomy increases intrinsic motivation and engagement. [+S]. However, production quality is not learning quality — decorative or seductive visuals chosen for polish rather than meaning can add load without adding understanding Decorative illustrations do not improve learning. [+M]. Design decisions about mode should be tied to rhetorical purpose, and the cognitive load of learning new tools should be managed separately from the load of learning the content Cognitive Load Management — see Cognitive Load Management.

Context¶

Requirements¶

• An authentic purpose and audience; multimodal work gains coherence when it addresses real readers or viewers (Authentic Audiences & Purposes)
• Explicit instruction in mode affordances — what video does that prose cannot, when a diagram outperforms a paragraph
• Time and scaffolding for drafting and revision across modes, with Assessment criteria that evaluate rhetorical choices, not technical polish
• Tool instruction delivered just-in-time so software learning does not crowd out content learning
• Access to creation tools (audio recorders, video editors, web authoring platforms) and time to learn them
• Explicit instruction in design principles — coherence, signaling, redundancy, spatial contiguity — so learners can justify mode choices (Clark & Mayer, 2016)
• Structured critique and revision cycles with Feedback on both content and design decisions
• Access to computers, multimedia software, and internet connectivity; training in composition techniques for each mode
• Explicit design criteria so learners understand that mode choice is assessed, not decorative
• Time for iteration — multimodal work is substantially more revision-intensive than prose

Constraints¶

• Tool complexity can overwhelm working memory; when learners must simultaneously learn unfamiliar software and unfamiliar content, both suffer Cognitive overload degrades learning outcomes. [-S]
• Assessment is harder to standardize; rubrics that privilege polish over reasoning reward technical skill rather than understanding
• Learners with limited access to devices, bandwidth, or quiet production space are systematically disadvantaged
• Open-ended multimodal tasks can flounder without prior knowledge or models; unguided discovery during production is inefficient for novices [~M]
• Technology access and technical skill gaps can shift cognitive effort from composing to tool troubleshooting, effectively imposing extraneous load Cognitive Load Management [-M]
• Seductive but irrelevant visuals, music, or animation split attention and reduce retention rather than support it Media combinations affect recall and retention. [-M]
• Assessment is difficult: rubrics that privilege polish over rhetorical reasoning reward technical fluency rather than learning
• Open-ended mode choice can overwhelm novices; without constraints, learners default to decoration rather than design [~W]
• Production demands (software learning curves, rendering, editing) can consume working memory and time that should go to rhetorical learning, effectively adding extraneous load Cognitive Load Theory [-M]
• Unequal access to devices, software, and bandwidth can turn the assignment into a measure of household resources rather than learning
• Assessment is difficult: rubrics that privilege polish over rhetorical reasoning reward technical skill rather than learning
• Learners with weak prior knowledge of a topic may produce visually impressive artifacts with shallow content; mode choice does not substitute for substantive understanding

Implementation Variability¶

• Remix tasks — learners re-represent existing content in a new mode (turn an essay into a podcast), isolating the translation skill
• Scaffolded progression — start with single-mode-plus-annotation, then two modes, then full multimodal projects
• Collaborative production — team projects distribute technical roles and add peer feedback; collaborative writing improves text quality and transfers to multimodal drafting Collaborative writing improves text quality. [+M]
• Process portfolios — collect drafts, storyboards, and reflection alongside the final artifact to make thinking assessable
• Constrained formats (e.g., 60-second audio story, single-image essay) reduce tool overhead for novices
• Low-tech variants (poster + oral presentation, zines) preserve the design reasoning without digital barriers
• Interactive fiction tools (branching narratives) add structural complexity suited to advanced learners
• Individual vs. collaborative composition changes the workload distribution; teams can divide mode expertise
• Low-tech variants: poster sessions, zines, storyboards, annotated slides — the pedagogy survives without expensive tools
• Scaffolded variants: provide templates or exemplar analyses before free composition (Annotating professional examples builds the evaluative vocabulary learners need)
• Individual vs. collaborative production; collaborative variants add coordination load but build shared critique skills

Target Learners¶

• Adolescents through adults; younger learners benefit from heavily scaffolded versions with limited toolsets
• Learners whose strengths are non-verbal — multimodal options give them a legitimate channel for demonstrating understanding [~M]
• Advanced learners ready for rhetorical decision-making; novices need more constrained task structures first
• Secondary and post-secondary learners developing disciplinary and digital literacies
• Multilingual learners, who can leverage visual and audio modes to express understanding beyond English prose [+W]
• Learners with high task value [+W] for creative work show stronger engagement; learners motivated purely by grades may engage superficially [~W]
• Higher education and professional development learners, who can manage the production demands while focusing on rhetorical decisions

Target Learning Goals¶

• Deep understanding: re-representing content in a new mode forces generative processing
• Communication and rhetorical skill: audience analysis, mode selection, design reasoning
• Self-regulation: managing a long, multi-stage project builds planning and revision habits Self-Regulated Learning
• Rhetorical and design literacy: understanding how mode choices shape meaning for an audience
• Deep processing of content: transforming source material into a new representational format requires generative encoding Self-explanation improves conceptual understanding. [+M]
• Genre awareness across media forms
• Collaboration and project management in team-based productions
• Digital and media literacy: understanding what platforms and modes allow, disallow, and require
• Rhetorical reasoning: matching mode, medium, and audience to purpose
• Deep processing of content: transforming source material into a new representational format forces generative encoding Active learning improves exam performance. [+S]

Instructions¶

1. Define an authentic task with a real or realistic audience and purpose (Authentic Audiences & Purposes).
2. Analyze mentor texts — study 2–3 published multimodal artifacts, asking what each mode contributes (Case Studies can structure this comparison).
3. Plan in storyboard or wireframe form before production, mapping content to modes.
4. Draft with just-in-time tool instruction, keeping tool learning separate from content decisions (Cognitive Load Management).
5. Critique drafts against criteria focused on rhetorical effect, using structured peer feedback (Assessment for Learning — feedback is most effective at task and process levels Feedback most effective at task and process levels. [+S]).
6. Revise, publish to the audience, and reflect on mode choices in a written or recorded designer's statement.

Related Strategies¶

• Peer feedback and critique cycles — multimodal drafts need structured response before revision
• Process portfolio assessment — makes the composition process visible and gradable
• Remix and translation tasks — a lower-floor entry point into multimodal reasoning
• Digital storytelling — the most common single-mode-dominant variant, centering voiceover and imagery
• Video essay assignments — disciplinary analysis expressed through edited media
• Podcast production — audio-first composition emphasizing script and pacing
• Design composite STEAM tasks that jointly evidence technical fluency, ethical reasoning, collaborative practice and design iteration
• Annotating — analyzing others' multimodal texts builds the critical vocabulary learners need for their own compositions
• Chunking — segmenting multimodal content manages the high processing demands of combined modes

Examples¶

• National Writing Project — extensive practitioner work on digital and multimodal composition in K–12 and adult classrooms, including the Digital Is collection of classroom cases.
• Podcast units in secondary English — students research a topic, script, record, and publish an episode; scripting is revision of prose for the ear, making audience awareness concrete.
• Explainer video assignments in STEM courses — students produce a 3-minute video explaining a concept; the requirement to visualize mechanisms exposes shallow understanding that a written summary can hide.
• StoryMap projects (e.g., Knight Lab StoryMap) — learners combine maps, images, and narrative text, forcing spatial and verbal reasoning to align.
• Stony Brook University's digital storytelling program — showcases student multimodal compositions across courses as models of curricular integration.
• Inklewriter (inklestudios.com/inklewriter) — free tool for web-based interactive stories with branching endings; learners design narrative structure and reader choice.
• NWP Digital Is (National Writing Project) — curated collection of K–12 multimodal composition projects with teacher reflections.
• Soundtrap for Education — collaborative browser-based audio production used for student podcast assignments.
• A first-year composition course where students produce a video essay remixing a research paper, with a written rationale defending each mode choice
• NWP Digital Is — the National Writing Project's collection of multimodal composition classroom practices
• Pre-service teacher programs requiring an audio documentary on a community issue, assessed on rhetorical effectiveness rather than production polish

Key Sources¶

• Mayer, R. E. (2021). Multimedia learning (3rd ed.). Cambridge University Press. doi:10.1017/9781316941355
• Paivio, A. (1990). Mental representations: A dual coding approach. Oxford University Press. doi:10.1093/acprof:oso/9780195066661.001.0001
• Kress, G., & van Leeuwen, T. (2021). Reading images: The grammar of visual design (3rd ed.). Routledge.
• Selfe, C. L. (2009). The movement of air, the breath of meaning: Aurality and multimodal composing. College Composition and Communication, 60(4), 616–663.
• New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92. doi:10.17763/haer.66.1.17370n67v22j160u
• New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–93. doi:10.17763/haer.66.1.17370n67v22j160u
• Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge University Press. doi:10.1017/CBO9780511811678
• Clark, R. C., & Mayer, R. E. (2016). E-Learning and the science of instruction (4th ed.). Wiley. doi:10.1002/9781119239086
• Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Routledge.
• Miller, S. M., & McVee, M. B. (Eds.). (2012). Multimodal composing in classrooms: Learning and teaching for the digital world. Routledge.
• Bezemer, J., & Kress, G. (2008). Writing in multimodal texts: A social semiotic account of designs for learning. Written Communication, 25(2), 166–195. doi:10.1177/0741088307313177
• Takayoshi, P., & Selfe, C. L. (Eds.). (2007). Multimodal composition: A critical sourcebook. Hampton Press.
• Shipka, J. (2011). Toward a composition made whole. University of Pittsburgh Press.