Developing a Critical Lens for Multimodal Analysis¶

Strategy · All strategies
Evidence · 6 claims (4 for, 1 mixed, 1 against) · 20 studies (11 causal, 4 quant-synthesis, 4 review, 1 theoretical), q1–q4 · 4 of 20 report an effect size · 1 claim rests on one study

Description¶

Students develop a critical lens for multimodal analysis by examining how different modes — image, written text, layout, color, sound — carry meaning individually and in combination. Instruction typically begins with simple media (e.g., a picture book) and structured discussion of how images and words convey, complement, or contradict each other's meanings, then expands across genres and media. The goal is to build a repertoire of exemplar structures that students can draw on in their own multimodal composition.

Design Implications¶

This strategy applies social-semiotic analysis to composition instruction: learners cannot deliberately deploy multimodal resources they cannot perceive or name. Analyzing multiple contrasting cases across genres builds the flexible, condition-sensitive knowledge needed for transfer to new media Analyzing multiple contrasting cases supports flexible knowledge transfer. [+M]. Because meaning arises from the interaction of modes, analysis should focus on combinations rather than isolated elements — how placement, salience, and framing direct attention and imply relationships Mayer, R. E. (2009). Multimedia Learning (2nd ed.) [+S].

Context¶

Requirements¶

• A curated set of multimodal texts (picture books, informational graphics, websites, news media) chosen to illustrate distinct design effects
• Analysis prompts that name specific semiotic resources (gaze, framing, color, typography, spatial layout) rather than asking for general impressions
• Demonstration of expert analysis — the teacher modeling how to "read" a page — before students analyze independently
• Structured class discussion and time for annotating texts to make observations concrete

Constraints¶

• Students with limited prior knowledge of multimodal composition may produce only surface-level observations without scaffolded vocabulary and modeling [~M]
• Poorly chosen examples can backfire: decorative or irrelevant imagery teaches students that visuals are ornamental, undermining the analytic goal Decorative illustrations do not improve learning and can distract from essential content. [-M]
• Analysis without a follow-on composition task yields recognition knowledge that does not transfer to production; the exemplar collection must be connected to practice [-M]
• Highly open-ended analysis can overload novices; constraining the first analyses to one or two modes reduces unproductive search Example-problem sequences reduce cognitive load for novices. [+M]

Implementation Variability¶

• Progression by complexity: start with word–image relations in picture books, then move to layout and typography in informational texts, then to interactive and moving media
• Comparative analysis: juxtapose two treatments of the same content (e.g., a tabloid and a broadsheet front page) to make design choices visible through contrast
• Reverse engineering: analyze a professional text, then have students redesign it for a different audience and justify the changes
• Annotation schemes: provide a shared vocabulary (e.g., Kress & van Leeuwen's representational, interactive, and compositional metafunctions) or let classes co-construct their own terms

Target Learners¶

• K–12 and undergraduate students preparing for multimodal composition tasks
• Novices benefit most from modeled analysis and constrained early cases; as expertise grows, open-ended analysis of unfamiliar media becomes more productive and highly scaffolded analysis becomes redundant Guidance effectiveness reverses as learner expertise increases. [~M]
• English learners and students from diverse media backgrounds benefit when examples draw on familiar cultural texts, activating prior knowledge Activating prior knowledge improves learning. [+M]

Target Learning Goals¶

• Critical media literacy: recognizing how design choices position viewers and construct authority
• Semiotic vocabulary: naming the affordances and constraints of individual modes
• Composition transfer: collecting exemplar structures to inform deliberate design decisions in students' own multimodal texts
• Dual-channel processing: understanding how verbal and visual information combine Dual coding of verbal and visual information improves recall. [+S]

Instructions¶

1. Select a simple multimodal text and demonstrate an expert think-aloud analysis, naming specific design choices and their effects on meaning.
2. Guide whole-class analysis of a second text using shared prompts (What does the image show that the words do not? Where do they conflict? What does layout imply about importance?).
3. Have students work in pairs or small groups to annotate new texts, recording effective and ineffective mode combinations in a class exemplar chart.
4. Use class discussion to compare across texts and genres, generalizing patterns (e.g., "gaze at the viewer builds intimacy; gaze away invites observation").
5. Connect analysis to practice: students compose their own multimodal text, citing exemplar structures from their collection, and receive feedback on their design decisions.
6. Revisit the exemplar collection as new genres are analyzed, extending it cumulatively across the term.

Related Strategies¶

• Activating Prior Knowledge — grounding analysis in media students already know makes semiotic patterns more noticeable
• Case-based discussion — the analyzed texts function as cases whose features generalize to new composition situations

Related Elements¶

• Demonstration — the modeled think-aloud that makes expert analysis visible
• Annotating — the recording mechanism that turns observation into a reusable exemplar collection
• Class Discussion — the social comparison that generalizes across cases
• Practice — the composition task that converts analytic knowledge into design capability

Tools¶

• Annotation tools (e.g., Perusall, Hypothesis) for collaborative markup of digital texts
• Digital composition platforms (e.g., Book Creator, Canva, Adobe Express) for the follow-on composition cycle

Examples¶

• Elementary picture-book study: Students compare how The Three Pigs (Wiesner) and a traditional telling use frame-breaking images to shift narrative authority, then chart word–image relationship types before illustrating their own retellings.
• Secondary media analysis: Students analyze two news websites' coverage of the same event, annotating how image selection, headline typography, and layout construct different credibility, then redesign a neutral story for each outlet's style.
• Undergraduate multimodal composition: In first-year writing courses using Serafini's Reading the Visual framework, students build a semester-long "design notebook" of analyzed exemplars cited in the rationale for their final multimodal project.

Key Sources¶

• Kress, G., & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design (2nd ed.). Routledge.
• Bezemer, J., & Kress, G. (2008). Writing in multimodal texts: A social semiotic account of designs for learning. Written Communication, 25(2), 166–195. doi:10.1177/0741088307313177
• Serafini, F. (2014). Reading the Visual: An Introduction to Teaching Multimodal Literacy. Teachers College Press.
• Mayer, R. E. (2009). Multimedia Learning (2nd ed.). Cambridge University Press. doi:10.1017/cbo9780511811678
• Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading Visual Narratives: Image Analysis of Children's Picture Books. Equinox.