Understanding Language and Culture
through Multimodal Text Analysis
Neil Johnson
Kanda University of International Studies, Japan
CERCLL IC, January 22nd 2016
Overview
—  Context
—  The problems in Japanese Higher education
—  Two Part Solution: Process and Literacy-based Pedagogy
—  A classroom example of re-designed activity
—  Conclusions
Context
—  Academic Reading and Writing program small private
university in suburban Tokyo
—  Curriculum redevelopment project: “Grobaru
Jinzai” (2011)
—  1989-2011 Skills-based approach:
Native-speaker model
Spoken proficiency
“Four-skills”
Culture: “Food, festivals and famous men…”
(Coehlo, 1998)
Japanese Higher Education
—  Falling standards generally – demographic issues
—  Wide range of proficiency levels – all taking the same
courses
—  Materials-based skills courses unsuitable
—  ‘Pressure’ from government to globalize and study
abroad
—  Study abroad students in particular need intercultural
competence and range of text-based literacies
Two Part Solution:
Process and Literacy-Based Pedagogy
Process Framework (Breen, 1984)
•  Self analysis
•  Audio-visual analysis
•  Data analysis
•  Text Analysis
•  Communication strategies
•  Problem solving
Literacy-based Pedagogy
—  Kern, R. (2000). Literacy and Language Teaching. OUP.
—  Kern, R (2015). Language, Literacy, and Technology. CUP.
—  Swafer, J. & Arens, K. (2005) Remapping the Foreign Language
Curriculum: An Approach Through Multiple Literacies. New York:
MLA
—  Paesani, K., Allen, H. W., & Dupuy, B. (2015). A
Multiliteracies Framework for Collegiate Foreign Language Teaching.
Prentice Hall.
Literacy-based Pedagogy
•  To account for the context of our culturally and linguistically
diverse and increasingly globalized societies.
•  The burgeoning variety of text forms associated with
information and multimedia technologies…
(Kalantzis and Cope, 2000, p. 9)
Modes (Social) and Media (Material)
—  Modes - Abstract, non-material resources of meaning-
making: Linguistic, Aural, Visual, Gestural, and Spatial.
—  Media - specific material forms in which modes are
realised (e.g. a photograph, computer screen, audio
recording)
Kress, G.and Van Leeuwen, T. (2001). Multi-modal Discourse. London: Arnold.
Multimodality
“The adaptive capacity to recognize, imagine, and
effectively exploit the meaning-making potential of a
diverse array of resources in view of one’s own purposes
for communication.”
Nelson (2010)
Semiotic agility (Prior, 2010)
Capacity vs. competence ( Widdowson, 1983)
Affordances and Constraints of Mode
(Kress, 2003)
Explain a plant cell linguistically:
The plant cell has a nucleus…
Explain a Plant Cell Visually:
Classroom Example
—  Discussion-Reading-Writing-Reading-Transforming-
Reflecting
—  Process: Text analysis
—  Explore different intercultural takes on cultural anxiety
about ‘Consumerism’ through array of texts
—  Multiliteracies/Process framework:
Situated practice
Critical framing
Overt instruction
Transformed practice
Situated Practice: Introducing Theme
Journal Responses
—  “Boring family…they look tired” (Junko)*
—  “I noticed Japanese food is fresh and healthy…” (Masa)
—  “My family also watches TV a lot together…”(Hana)
—  Wow! Japanese houses are so small LOL (Kenta)
*All names have been changed
Overt Instruction
—  How to interpret visual texts?
—  We need to first identify the different elements that make up
the text.
—  These include: Resources of Visual Mode:
—  Images
—  Colour
—  Words – includes title, headlines, captions
—  Typographical features - type of font, font size
—  Layout – spatial arrangement of different elements in a text
—  Icon, index and symbol
Vocabulary – Available Designs of
Multimodal Texts: Photo
1.Angle
2. Frame
3. Depth
4. Image
5. Spatial
6. Layout
7. Contrast
8. Background knowledge
9. Ideology
10. Metaphor
11. Icon
12. Symbol
13. Index
Critical Framing
Critical Framing
Critical Framing
Critical Framing
Journal Response
—  I’m surprised now about the Japanese family…what is all
that stuff? Do we really need it? The framing of the shot
is very skillful – there is no room, the shot is full. This is
a visual metaphor I think. (Junko)
—  The photographer does a great job of showing how we
are sinking in all the things we buy…a metaphor through
the idea of a sea of wrapping. I’m not so sure about what
fresh food means now. The texture of the shot is very
impressive. Author is able to communicate detail so well
but the detail takes on a different meaning…etc.
(Masa)
Process – Text Analysis
—  Prepare a short presentation for your group based on a
text that you have found of interest.
—  The text must relate to our theme of consumerism.
—  Be prepared to describe your text and explain what it
means to you.
—  Highlight the ‘available designs’ used by the author
—  How have these been used – how do they create
meaning?
—  What do YOU think about the text?
Text Response Worksheet
—  Questions related to different text types and genres
—  Repetition reveals interesting patterns and authorial
choices
—  Thinking “multimodally” - about design - through all
kinds of texts
Group 1: Gursky 99 Cents
Group 1: Landfill Photo
Group 1: Illustration
Group 1: Website on Ecology
Group 1: Website Editorial
Class Work: Text Analysis - Gursky
Available designs for
Meaning
How is it used in this text?
What does this mean to you?
Frame
Texture, repetition
Points of view: nature, consumer
culture
Colour
Bright, artificial, no space
Nature has been removed?
Point of view
Where are we in relation to the
We are them?
author and the people in the shot?
Metaphor
Repetition
Modern life is cheap, artificial
Detail
Overwhelming
So much detail but no meaning
Confused
Depth and anxiety
It goes on forever?
Use of people, light.
Composition - shot
Class Work: Text Analysis - Editorial
Available designs for
Meaning
What does it mean?
How does it mean?
Layout – headline and text
Points of view: nature, consumer
culture
Easy to read
Academic language
People are ‘lost’ - isolation
Serious opinion
Point of view
Cheapness of modern life?
Personal experience
Metaphor
Modern life
Maze = lost
Links to other sites
Support for ideas
Others agree
Detail
Confused
Depth of thinking
Composition
Short paragraphs
We can stop reading so the form
is important
Butterflies - Narrative
“She said butterflies are beautiful creatures…they visit
the pretty flowers, she said.You don’t kill butterflies, that’s
what she said.”
The grandmother and grandfather were quiet for a long
time, and their granddaughter, holding the book stood still in
the warm garden.
“Because you see,” the grandfather said, “your teacher,
she buy all her cabbages from the supermarket and that’s why.”
Patricia Grace
Class Work: Text Analysis - Narrative
Available designs for
Meaning( Short Story)
How is it used in this text?
How does it mean?
Plot – structure - action
Points of view: nature, consumer
culture
Characters: Teacher, student,
grandparents
Setting
different perspective – school irony
School should accept different
ideas
Brings in different cultural
perspectives – we see different
cultures working
Plot: Maori voice VS Western
voice
structure
Voice
Language: description
Metaphors: Butterflies, cabbages,
supermarkets, farms
Conversation
Western Culture:
Love, beauty, excitement
Butterfly – Moth?
Maori: Pest. Japan: Spring,
elegance
Presents different points of view.
Nothing is so easy.
Transformed Practice
Consider the main meaning of the short story
‘Butterflies’. Try to re-write the story from the point of
view of the teacher. How does this change the story?
Can you maintain some of the same meaning? What
changes do you have to make? What aspects remain the
same?
Class Journal: Megumi
…Now I see that buying things is not just shopping. We are doing
something bigger. We should think about how is nature? What is the
effect of our action? The butterfly becomes a symbol for different
ideas and points of view. The author uses the narrative to give
us different ideas. The waiting and then the ending from
the grandfather gives us big impact and shows the teacher
never considered another point of view. The grandfather
represents age and wisdom. A photo couldn’t work this
way. In considering a different perspective it becomes not so easy to
say exactly what is right and wrong. But now I feel personally the
world is in danger because of our waste and our moving away from
natural way of life…
Preliminary Conclusions
—  A process-based approach allows for a dynamic and
ecological pedagogy that is co-constructed and reflects
the needs and interest of learners
—  Focus on the “affordances and constraints” of mode
(Kress, 2003; Stein, 2009) and the design of texts across a
variety of text types and cultural contexts, rather than on
comprehension of the written word.
—  Account for creative and experience-based responses to
texts and the context and multimodal dependent nature
of meaning in contemporary communication landscape
Thanks!