The Effects of Animated Feedback
on Locating Verbs in a Dynamic
Contextual Scene Display on an
Augmentative and Alternative
Communication Device
Jacquelyn Kearns
The Cleveland Clinic Children’s Hospital for Rehabilitation, Cleveland, OH
John McCarthy
Ohio University, Athens
M
ore than 3.5 million Americans have physical, cognitive, and developmental impairments so severe that they cannot rely on
their own speech to meet their daily communication needs
(Beukelman & Mirenda, 2005). Augmentative and alternative communication (AAC) offers the potential to enhance
these individuals’ communicative skills. AAC can include
using a voice output communication aid to talk to a friend,
ABSTRACT: Purpose: Technological advances in augmentative and alternative communication (AAC) offer the potential to enhance the communication needs of individuals
with severe communication disorders. However, using AAC
to express concepts that cannot be easily represented in
static line drawings is a significant challenge. Animation of
line-drawn action verbs represents a means to enhance the
transparency of previous static representations and potentially make the representations easier for children to use.
Method: A mixed group design with repeated measures
was used to test the effect of animation used as feedback
in locating and selecting action verbs in a visual scene on
a dynamic AAC display. Twenty 3-year-old children without
disabilities (10 boys and 10 girls) were quasirandomly assigned to an animation or no-animation group. Participants’
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
pointing to pictures in a communication book to order at
a restaurant, or using a computer to type and send e-mail.
Despite advances in technology, AAC systems may impose
significant learning demands on users (Light & Drager,
2007). Children requiring AAC face the larger task of
acquiring language heard around them while simultaneously
needing to express themselves via a different output modality (Light, 1997).
accuracy in locating and selecting representations of 15
action verbs in a play activity repeated across 3 sessions
was recorded.
Results: Results of the study revealed that animation did not
lead to significant performance differences between the 2
groups. Adequate representation of concepts allowed children to perform near mastery level after 2 sessions.
Conclusion: Action verb representations can be quickly
learned within visual scenes. Ceiling effects may have
contributed to the results. The role of animation in learning
should be further explored and separated from navigational
demands in future studies.
KEY WORDS: augmentative and alternative communication,
assistive technology, children, learning, animation
Kearns
• McCarthy:
Animated
DISORDERS
• Volume
39 • 43–53 • Spring
2012 Feedback
© NSSLHAand Verb AAC Symbols
1092-5171/12/3901-0043
43
Individuals who can speak use words to represent the
message they are trying to convey to their partner.
Communication using aided AAC systems (i.e., those
requiring some kind of external equipment) often requires
the user to construct a message with visual symbols, which
may be transformed into spoken output. In some cases, the
visual representation can match an object very well. A line
drawing of a “dog,” for example, can resemble a dog. In
the case of more abstract items like actions such as “running,” “jumping,” or “building,” static line drawings may
not provide the necessary cues to aid in symbol recognition
or learning. In fact, children may have significant difficulty
locating currently available representations of abstract concepts within AAC devices (Drager, Light, Speltz, Fallon, &
Jeffries, 2003; Light et al., 2004).
Light and Drager (2007) suggested that in order for
AAC devices to be effective communication tools, they
must “minimize the cost of learning while at the same time
maximizing the power of communication” (p. 208). The
cost of learning is the amount of time and effort needed to
learn the basic skills to find, access, and produce desired
messages on an AAC system. The power is being able to
express exactly what is on the user’s mind. Consideration
of how best to improve AAC systems so they can convey
more abstract concepts like action verbs requires consideration of the nature of visual symbols in terms of how, why,
and under what conditions they work.
Symbolic Representation in Abstract Symbols
Often times, it is assumed that children understand symbol
representation before they are actually capable of using it.
Children using AAC must not only gain an understanding of the spoken word and its relation to a verb, but they
must also understand an external referent to express the
concept (Light, 2003).
In order to have symbolic understanding of pictures,
children progress through three stages of symbolic learning
(Mineo Mollica, 2003). In the first stage, representational
insight occurs when children understand basic similarities
between the symbol and its referent but do not fully grasp
the ability of the symbol to “stand in” for the referent (Uttal, Schreiber, & DeLoache, 1995). For example, a child at
this stage might become excited at seeing a box of cereal
with a photo of the cereal on the front.
In the next stage of understanding, children learn that
a symbol can dually represent an actual object as well as
be an object unto itself (DeLoache, 2000). To continue the
previous analogy, a child at this stage could use the cereal
box to request the cereal and could also put the box into a
toy shopping cart. The child understands both the referential value of the object as well as its immediate physical
function. For abstract concepts, children must recognize
two things: the individual items within a representation and
the new concept portrayed by the interaction of the individual items. For example, consider a boy riding a bicycle.
In an aided AAC display, this picture would be static. Children would see a boy and a bicycle (both concrete objects);
however, they may not immediately see that the picture can
also represent the concept of ride.
44
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
In the final stage of symbolic understanding, symbolic sensitivity, children look for the symbolic relationship in other
entities other than the ones they may have been previously
taught (DeLoache, 1995). For example, a child who can
point to a drawing of a milk carton can then start to make
other food item requests when eating (Mineo Mollica, 2003).
Rather than trying to disambiguate the meaning of a
single symbol in a box, it is possible that providing a
background or context for a visual symbol could aid in
identifying its intended meaning. Contextual scenes provide
a potential format for displaying abstract symbols.
Contextual Scene Displays
Young children learn language through interactions with
the environment. Therefore, it is appropriate for an AAC
device to approximate the medium through which the typical child learns language. Contextual scene AAC displays
embed symbolic representations into a cohesive scene
rather than as independent “boxes.” Contextual scenes can
be photographs of familiar scenes or line drawings. For
example, a contextual scene could contain a drawing of a
living room complete with the objects one might expect
to find in a living room (e.g., couch, lamp, table). Each
object would be represented according to the size and
location one might expect in an AAC display. Voice output
would be activated by touching an object in the scene. For
example, touching the couch could retrieve the message,
“couch.” Contextual scene displays have been found to be
an effective layout method for representing language concepts for young children (Drager et al., 2003, 2004; Light
et al., 2004).
Currently, contextual scenes are static in nature. Despite
contextual support, it is difficult to represent an abstract
concept like play or come within a static scene. Research
done by Drager et al. (2003, 2004) and Light et al. (2004)
concluded that children had significant difficulty locating
abstract concepts in AAC displays whether a scene or a
traditional grid of rows and columns was used. In these
studies, children 2½ to 5 years old were asked to locate
concrete and abstract concepts within various AAC displays
by helping “Bobby the bear” find and say words to participate in a birthday party. Concrete concepts were items
such as cake and present; abstract concepts included come
and play. Drager et al. (2003) found that 18 out of 30 of
the 2½-year-olds in their study could not locate any of the
six abstract concepts in any of the AAC systems used, even
after three learning sessions.
Similarly, Light et al. (2004) discovered that forty 4year-old and forty 5-year-old children located 12 or 15
abstract concepts, respectively, with significantly less
accuracy than equivalent numbers of concrete items over
three learning sessions. In looking at overall trends across
their studies, the authors found an advantage for organizing
items in scenes for 2- and 3-year-old children in particular,
but the disparity between abstract and concrete retrieval
could certainly be improved. An examination of the visual
cognition literature can help to determine more appropriate
symbol representations for abstract concepts.
DISORDERS • Volume 39 • 43–53 • Spring 2012
Visual Cognition and AAC Displays
Visual cognition is the study of visual, spatial functioning
and neurological foundations. It has been suggested that
considering what is known about visual cognition and how
different visual information impacts attention and memory
may aid in reducing the learning demands of AAC systems
for young children (Light & Drager, 2007). One of the current problems with AAC technologies is that visual stimuli
represented on the display can symbolize an object, an
event, an emotion, or even a concept (Wilkinson, Carlin, &
Jagaroo, 2006). The ability of young children to discriminate, identify, and/or recall information from memory may
depend highly on the degree to which symbol representations conform to visual processing principles (Wilkinson et
al., 2006).
Jagaroo and Wilkinson (2006) suggested that using movement may help increase the saliency of different concepts
on a schematic display by drawing attention to those
objects. Task learning, task performance, item discrimination, and memory improved when motion was given to the
stimulus (Jagaroo & Wilkinson, 2006). Motion can highlight things like causal or functional relationships between
objects to enhance the clarity of visual representations
(Jagaroo & Wilkinson, 2006).
Mineo Mollica, Peischl, and Pennington (2008) found
that simple animations with stick figures based on human
movement were more helpful than static drawings in helping 3-year-old children to identify action verbs in a grid
of four choices. The authors also noted a developmental
trend of improved performance across representation types
(static and animated) with 4- and 5-year-olds. The research
was not extended to include larger arrays of symbols or
embedding symbols within a contextual scene. Similarly,
Fujisawa, Inoue, Yamana, and Hayashi (2011) found that 16
children with intellectual disabilities ages 11–18 years were
more accurate in labeling symbols of 16 action words (e.g.,
throw, wipe, put) when the symbols were animated than
when they were not animated. Further, the authors found
that the participants with lower scores on a standardized
Japanese developmental inventory benefited more from the
animation than participants with higher scores.
Neither of the two previous studies involving animation
involved animation within a contextual scene; instead, the
animations could play continuously during presentation. In
a contextual scene, the presence of multiple moving images
is potentially distracting. To present images in a contextual
display, a static representation may need to be used initially
and then the animation could play on selection of an item.
This role of animation would be as feedback rather than
as an immediate aid to retrieval in finding an item within
a display. The consequence would potentially be an aid in
learning rather than in immediate recognition. Jagaroo and
Wilkinson (2006) stated that only simple, localized motion
was needed to represent a realistic concept, but they did
not speculate about the best representation of animation
within a contextual scene. Consequently, a simple animation (i.e., an animation made up of five sequential images
played in a repeated loop sequence) should be sufficient for
an animation task.
Although there is some empirical evidence on the positive effect of animation in small symbol arrays (Fujisawa
et al., 2011; Mineo Mollica et al., 2008), there is a lack of
research on how or if such animations would be helpful in
children who are retrieving or learning target concepts from
contextual scenes. Given the potential benefit for young
children in organizing AAC displays by contextual scenes,
the lack of success to date in facilitating the retrieval
of more abstract concepts within those scenes, and the
potential for animation to improve children’s identification
of representations of verbs, a study combining contextual
scenes with the decontextualized animation work to date
was needed.
Research Questions
The purpose of the current study, then, was to examine the
effect of simple animation as feedback on children’s ability
to locate verbs within a contextual scene display. The following questions were addressed:
• What is the accuracy of typically developing (TD) 3year-old children in locating verbs on a dynamic AAC
visual scene display with only verbal feedback versus
animated and verbal feedback?
• Do the children’s performances improve across learning sessions with these different forms of feedback?
METHOD
Research Design
A mixed group design with repeated measures was used
to determine the accuracy of children in the two feedback
conditions. The between-group factor was feedback condition (animated/verbal vs. verbal), and the within-group factor was time (Sessions 1, 2, and 3). In the one condition,
children saw animated feedback of a stick figure performing the action represented by a static figure in the display
and heard the label of the action. In the other condition,
children heard only a label of the action.
Participants
A total of 20 TD 3-year-old children recruited from a mix
of rural and urban area preschools participated in the study.
All children had no significant history of language, visual,
cognitive, or motor impairments, as indicated by parent
report. The use of TD children has been supported by research done by Drager, Light, and colleagues (Drager et al.,
2003, 2004; Light et al., 2004). Using TD children allowed
the researchers to examine the effects of feedback on learning without confounds of other impairments and helped to
ensure equivalency of groups.
At age 3, children are learning to take the perspective of
another person (Nelson et al., 2003), but they may not yet
have developed dual representation of graphic symbols (DeLoache, 2000). Children in the present study were required
to take the perspective of a doll that could not use her
Kearns • McCarthy: Animated Feedback and Verb AAC Symbols
45
speech to communicate. Parents of children who met the
above criteria for the present study filled out the MacArthur Communicative Development Inventory (CDI; Fenson
et al., 1994) to ensure that the children had knowledge of
the target concepts in the current study. Although not a
formal language test, this practice was consistent with other
published studies in this area (e.g., Drager et al., 2003,
2004; Light et al., 2004). The children recruited for the
study had a mean age of 40.9 months and ranged in age
from 37 to 45 months; 15% (3 of 20) were from diverse
ethnic and cultural backgrounds (all were African American
and all spoke English as their primary language at home
and at school).
Stimuli
The stimulus was a house that was created using Adobe
Photoshop (Adobe Systems) and BoardMaker (Dynavox
Technologies) software. Verbs were chosen based on verbs
from the CDI. According to the CDI, children should have
all of the verbs present in the current study by 30 months
of age (Fenson et al., 1994). Stimulus items for each verb
were found in Microsoft ClipArt online. The house had
four rooms and an outside icon as well. Children had five
contexts to choose from when locating verbs: outside,
bedroom, bathroom, living room, and kitchen. There was a
screen shot of each room on the main page for the house.
Each room in the house contained children acting on some
object in the room, for a total of 15 stimulus verbs and one
practice verb (see Table 1 for a list of the items). Three
verbs in each room reduced the opportunity for guessing
in the event that children could navigate to the correct
room but were unsure of the correct verb to choose. Verbs
were chosen to avoid words with overlapping meanings
and to include ones that had existing picture communication symbols (PCS) animations and that could logically be
performed within each room in the house. Upon selection
of a verb, the label of each item was announced automatically using the DECTalk Kit the Kid voice. To ensure that
the child understood the label, the experimenter repeated
each label after the synthesized speech played. A copy of
the home page is shown in Figure 1. The stimulus pages
were presented on a Panasonic Tough Book Model CF-18
with a 10.4" viewable LCD screen swiveled to be presented
as a tablet with no visible keyboard.
Procedure
After recruitment and consent, the convenience sample of
participants was matched for sex and was randomly placed
Table 1. Verb stimulus items.
Hide
Read
Dry
Build
Drink
Climb
Jump
Brush
Sit
Eat
Kick
Tickle
Wash
Call
Share
Note. The practice verb was Ride.
46
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
into two groups (Group 1, Group 2) of 10 participants with
5 girls and 5 boys in each group. The mean ages for each
group were identical, with a similar range and distribution
of ages between the groups. Group 1 received the animated
and verbal feedback. Group 2 received only verbal feedback. Each individual in each group participated in a play
activity for 20 min a session, for a total of three sessions
(average session length was ~15 min for the first session
and decreased with each subsequent session due to quicker
response time for each item). Sessions took place at the
day care center or the child’s home. All sessions were conducted by the first author.
The children were introduced to the AAC technology
in the context of a play activity involving a small doll
named Katie. Children were told that Katie could not talk,
and they had to use the computer to talk for her. The play
activity was explained to the children as follows:
Look up here. See the house? That is Katie’s house. Katie likes
to sleep a lot. She will wake up soon. When she wakes up, we
have to find things in and outside the house for Katie to do, so
she does not go to sleep. Katie can’t talk. We have to use the
computer to talk for her.
Children were given an action word to find and were
expected to use the computer to “say” things for Katie to
do. When children navigated to a room from the home page
and then clicked on a representation of a desired verb, they
received the verbal feedback of the verb name as well as a
five-frame animation of a stick figure acting out the verb.
Animations were used from PCS Animations (Dynavox
Technologies). The animations either directly or nearly
resembled the static action in the experimental house. The
animations were of stick figures performing the specific
actions. Although the software contained representations
with nonstick figures, the items did not integrate with the
background in size, shape, or orientation. Consequently, the
stick figures were used to emphasize the action that the
feedback was designed to provide. A copy of one of the
line-drawn animations is provided as Figure 2. All rooms,
verbs, and locations of verbs in the experimental condition
of the house remained the same as in the control condition.
The control condition contained only verbal feedback. No
animation was played on selection of an item.
After each successful trial or following experimenter
feedback, Katie was “woken up” by moving her around.
Toys (e.g., a stuffed animal, blocks, a towel, a cup, and a
ball) were provided for Katie to play with, although the
experimenter never directly chose a toy for Katie to use.
It was never noted that children used the toys with Katie,
but rather interacted with Katie directly by moving her
around to keep her awake. Play involving Katie was kept
to <30 s.
In order to promote learning and avoid memorization of
the task, sessions were scheduled at 3- to 5-day intervals.
Two participants had 6 days between sessions. The procedures were adapted from studies by Light, Drager, and
colleagues (Drager et al., 2003, 2004; Light et al., 2004).
During each session, children were required to locate 15
verbs in the context of the play activity. The order of the
verbs was randomly selected for each session. Each child
DISORDERS • Volume 39 • 43–53 • Spring 2012
Figure 1. Print screen capture of the home page for the experimental and control conditions.
also received an instructional model of the task before
being required to locate the 15 verbs. A scripted, instructional prompt was used for each target verb (e.g., “Katie is
hungry. Show her how to say ‘eat.’”). The last word of the
prompt served as the cue for the child to locate that verb.
Instructional prompts were only stated once. If children
were incorrect or did not respond within 20 s, they were
given an instructional response. That is, the researcher read
a prepared response explaining the rationale of where the
item is in the house (e.g., “‘Eat’ is in the kitchen because
that is where you eat food”) and the specific item (e.g.,
“This is ‘eat’ because the boy is eating a sandwich”).
Instructor responses for incorrect or no response were the
same in both conditions. The child’s first selection was
recorded, and children were prompted to continue to prevent revisions within sessions. To ensure that the children
understood the synthesized speech label from the computer,
the experimenter repeated each label after the synthesized
speech played. A portion of the script used for the sessions
is contained in the Appendix.
Reliability
To ensure correct administration of the task, procedural reliability was calculated for each condition. The first author
practiced the scripted procedures until she could administer
them consistently with at least 90% accuracy. An independent judge who was a graduate student in speech-language
pathology listened to audio-taped recordings of 20% of the
samples along with the same script used by the investigator to determine if the investigator followed the procedures
correctly. The judge practiced with two samples (once with
the investigator and once independently) and achieved
100% agreement with the investigator’s scores as training.
Procedural reliability was calculated to be 95% accurate
with deductions occurring for any deviation from the script
(resulting mostly from misspoken or skipped single words
that would most likely not have affected the child’s understanding of the trial).
To determine the reliability of scoring, at least 20% of
samples across children and sessions were chosen and were
coded by a separate trained observer. Scording reliability
Kearns • McCarthy: Animated Feedback and Verb AAC Symbols
47
Figure 2. Print screen capture of verb animation (e.g., build) in the experimental condition.
was judged by the same graduate student in speech-language pathology. The student practiced with two samples
(once with the investigator and once independently) and
achieved 100% agreement with the investigator’s scores as
training. Scoring was based on the label as spoken by the
investigator during trials (in no cases did the investigator
fail to repeat the same label as produced via synthesized
speech from the computer) and the synthesized speech
output from the child’s selection. Interrater agreement
was determined by calculating the number of agreements
divided by the number of agreements, disagreements, and
omissions. Scoring reliability was calculated to be 100%.
RESULTS
Verbal-Only (VO) Versus
Verbal+Animation (VA) Feedback
Data Analysis
The percentage of correct responses out of 15 was calculated for each child in each of the feedback conditions for
each of the three data collection sessions. Means and standard deviations were calculated for each session for the
control group and the feedback group. Planned comparisons using repeated measures analyses of variance (ANOVAs) were conducted to investigate each of the research
questions.
48
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
The first research question addressed whether the accuracy
of TD 3-year-old children locating verbs within a dynamic
AAC display would improve with the use of animation as
feedback. A repeated measures ANOVA was conducted with
feedback (VA vs. VO) as the between-subjects factor and
time (Sessions 1, 2, and 3) as the within-subjects factor.
The results of the ANOVA are displayed in Table 2.
The main effect for feedback was not significant, F(1,
18) = .242, p > .05 (p = .629), indicating no significant
effect for the use of animation as feedback. There was no
interaction between condition and time, p = .62. The mean
and standard deviation results for each condition across sessions are displayed in Table 3.
Performance Across Learning Sessions
The second research question asked whether 3-year-old
children’s performance for locating verbs improved across
DISORDERS • Volume 39 • 43–53 • Spring 2012
Table 2. Analysis of variance for the effects of feedback by
group and learning session.
Source
df
F
Between subjects
Group
Error
1
18
0.24
(13.52)
Within subjects
Session
Session x Group
Error (Session)
2
2
36
52.40**
.48
(5.24)
p
.63
.00
.62
**p < .01.
sessions with the different conditions (VO vs. VA). Results
revealed that time was a significant factor in learning to
locate verbs within a dynamic display, regardless of condition. The main effect for time was significant, F(1, 18)
= 52.40, p < .01. There was no interaction between time
and condition, p = .62. Post hoc testing was conducted to
investigate significant differences between sessions. The
Bonferonni correction was used to control for multiple
comparisons (α = 0.001). Posttesting revealed a significant
difference between scores in Sessions 1 and 2, p < .001,
with Session 2 having a significantly higher number of
correct answers. There was not a significant difference between scores in Sessions 2 and 3, p = .37. The results for
accuracy over time are displayed graphically in Figure 3.
In addition to the learning that occurred across sessions,
the majority of children (n = 14) performed at mastery
level (80%) by the second session of testing, regardless of
feedback condition. By the end of Session 3, 90% of the
children in the study had performed at mastery level on the
task. Further analysis showed that learning was demonstrated within the first session. Figure 4 provides a descriptive
representation of learning within Session 1.
Assessing the Impact of Age
The groups in the present study were matched by sex, and
equivalent mean age (40.9 months) was established for each
Table 3. Means and standard deviations for the number of
verbs located in each session by each group: control (verbal
only, VO) and experimental (verbal+animation, VA).
Group
M
Δ
SD
Session 1
VO
VA
Total
5.90
7.00
6.45
1.1
2.89
3.83
3.35
Session 2
VO
VA
Total
12.30
12.00
12.15
0.3
1.95
3.27
2.62
Session 3
VO
VA
Total
13.10
12.00
13.40
1.1
2.13
2.45
2.26
group. Children were not matched in each group by age.
In order to assess the impact of age, a repeated measures
ANOVA was conducted with age as the covariate. Statistical analysis with age as the covariate did not reveal significant results between groups, F(1, 17) = 2.24, p > .05 (p =
.15), nor within groups, F(2, 34) = .19, p > .05 (p = .83).
DISCUSSION
The Lack of Effect of Feedback
Contrary to predictions, animated feedback did not significantly improve children’s ability to accurately locate verbs
within a dynamic communication display. Children in the
experimental condition performed comparably to children
in the control condition. It is possible that animation in the
form of feedback did not aid in rapid learning. Children
were still required to understand the immediate symbol
representation in order to get an item correct. Animated
feedback may not have been necessary because all children
had knowledge of all of the verbs. There are several other
possible explanations for the lack of effect, having to do
with the age of participants, the demands of the task, and
the mismatch of the animated figures with the static representation.
Children who participated in the present study may have
been too old developmentally to benefit from the use of
animated feedback during the task. It was confirmed that
the children had receptive knowledge of all of the verbs in
the study, and so they should have had a firm representation of the verbs in their mental lexicon. The children may
have also fully developed dual representation of the stimuli
before completing the study, thereby reducing the learning
demands of the task considerably
The stimuli used in both the experimental and control
conditions may have given the children too many cues,
thus reducing the demands of the task. The only selectable items within each display were verb items. Children
were required to find a child who was “doing _______”.
The prompts for each verb did not vary across items or
sessions. Therefore, the tasks may have relied more on
the children’s ability to memorize the verb representations
rather than learn to develop a map for a given verb representation via animated feedback. Given how quickly most
of the children reached mastery, it is possible that inclusion
of more items (verb or nonverb items) would have led to
differences between the groups.
One concern about the animations that were used in the
study is that they were not similar to the symbol representations. The animations used in the experimental condition
were five-frame line drawings. The line drawings approximated the qualities of the symbol representation; they were
not actual animations of the symbol. It was thought that
an animated approximation of the symbol would serve as
enough of a referent for the children in the experimental
condition. Research has suggested that only a hint of animation (e.g., partial movement of an object) is enough for
an individual to determine the action taking place (Jagaroo
& Wilkinson, 2006); however, there is little guidance in the
Kearns • McCarthy: Animated Feedback and Verb AAC Symbols
49
Figure 3. Mean performance of 3-year-old children locating target verbs across three learning
sessions.
Note. VO = verbal-only feedback, VA = verbal+animation feedback.
literature to indicate how closely a static representation and
an accompanying animated representation need to resemble
each other. Despite difficulty with some representations, an
error analysis revealed no single item as being consistently
missed across sessions.
The Effect of Time
Results revealed that participants performed markedly better
after the first session of the task, regardless of condition.
Interestingly, learning occurred across both groups in the
first session. There are two possible explanations for the
learning: the instructional script and the stimulus items.
The instructional script provided a clear task for the children. For each verb, children were first given a semantic
cue (e.g., “Katie is hungry”) then the stimulus prompt (e.g.,
“Show her how to say ‘eat’”). The syntactic structure of
the prompts was the same for each stimulus item. This
reduced the sentence processing involved in the task while
simultaneously providing a task that was simple for the
children to follow. Furthermore, the task itself appeared to
be motivating for young children. Children enjoyed finding activities for the doll around the house, and they also
enjoyed time spent on the computer. Both features of the
script could have reduced the learning demands of the task
for the children.
It can be concluded that most of the symbols used in
the present study provided an appropriate representation
of the verbs. The stimulus symbols were a close match to
the children’s mental representations of the verbs. Few of
Figure 4. Learning within the first session for the experimental and control groups of
3-year-old-children locating target verbs.
50
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
DISORDERS • Volume 39 • 43–53 • Spring 2012
the symbol representations did not accurately reflect the
children’s mental lexicon. Those that were not as accurately
located between the two sessions may be considered more
difficult to represent in a static picture (e.g., share). For
those symbol representations, it may be that children did
not understand the picture used or confused it with another,
similar-looking concept (e.g., give).
Clinical Implications
There are two important interpretations to be taken from
the present study. First, results of the current study suggest
that animation may not be needed to access more transparent verbs provided that the scene and task promote links to
storage of representations as verbs. Animation may also not
be needed for children who already have a representation
of the verb in their mental lexicon. Previous research studies (e.g., Drager et al., 2003, 2004; Light et al., 2004) have
shown that young children with serious communication
needs understand language better when it is represented in
a scene. These scenes must be familiar to the child (e.g.,
a photograph of a living room) or must closely resemble
their understanding of the world (e.g., a cartoon picture of
a room). For nonphotographic scenes, it is important that
the symbols that are used in the picture accurately reflect a
child’s perception of the world.
A second implication is that the activity used for both
conditions may be an effective teaching tool for children
who are using AAC devices. Across two sessions (~25
min of interaction), children located 12 concepts within a
four-room scene on a dynamic display. This could provide
a child with a significant increase in AAC verb symbols for
two-word utterance construction. The activity could supply
children with action–object and agent–action word combinations not previously capable without the use of verb symbols. It would take little time for children to acquire this
vocabulary. The overall activity for embedding language
has been used effectively in other AAC studies as well
(Drager et al., 2003, 2004; Light et al., 2004).
Limitations
The present study had several limitations. Participants
in the current study were TD children. Results may not
generalize to children who require AAC. Children with
severe physical and communication impairments may not
have had the same experiences with the verbs in the current study. It is possible that children requiring AAC could
benefit from animation; however, the current results cannot
address this idea. All children had the concepts of the verbs
in their mental lexicon. It is unknown how children would
perform in the experimental task if they did not have prior
knowledge of the verbs. Given that children were required
to navigate to a page as well as select the correct item,
there were two demands in the current task. Future research
could separate the two before considering the combined
effects of each.
The number of verb items used in both conditions may
not have been enough to necessitate the use of feedback.
Fifteen verbs and one verb model were used in both displays. The low number of verbs in each room paired with
the room displayed on the home page may not have been
a challenge. The number of verbs deals similarly with the
overall issue of scene layout. It is not yet known how
many objects in a room is too many for a child to perform
successfully. Therefore, it is difficult to determine whether
the number of verb items had a significant impact on the
results of the study.
In terms of design, review of videotapes of sessions
would have helped to ensure that all animations played correctly. Further, use of multiple raters may have helped to
further establish procedural integrity and scoring reliability.
A larger number of participants would have given the study
more power in asserting conclusions.
Directions for Future Research
The present study examined the effects of animated feedback on TD 3-year-old children. Research conducted by
DeLoache (2000) has suggested that children develop dual
representation for symbols between 2½ and 3 years of age.
Taking into account the age at which dual representation
develops, it would be important to conduct the current
study with TD children between 2 and 2½ years of age.
Future research could determine the effects of dual representation development on children’s ability to locate verbs
within a contextual scene.
It is also important to consider the animations that were
used in the current study. Animations in the present study
did not resemble the static representation of the children on
the page. Instead, the children were shown a stick figure
performing an approximation of the action. Further research
may consider the use of direct animation (e.g., animating the stimulus items more realistically in the scene) or
of video animation. Video animations of actions being
completed could be another useful method of teaching
verb concepts to children with severe motor and cognitive
impairments.
Finally, it is important to consider isolating animation
from other confounding factors that may be present in hightech communication devices. Separating animation effects
from those demands placed on new user navigation issues
could provide interesting results. Separating the need for animation as a teaching tool from the purpose of gaining attention to a new item within a page may also be investigated.
Conclusion
Children who use AAC devices are in need of sufficient
symbolic representation for a variety of concepts. Visual
scenes have provided an appropriate context in which to
embed language concepts; however, current visual scenes
have not provided enough support for a dynamic verb.
Animation as a form of feedback showed potential for
decreasing the gap in symbolic representation of a verb in
a child’s lexicon to that on a static photo. Animations did
not help to decrease the learning demands of this particular visual scene task; however, children were still able to
Kearns • McCarthy: Animated Feedback and Verb AAC Symbols
51
learn, and master, the task within a short period of time.
Animation may still prove useful in teaching verb concepts
to young children with restricted environmental exposure.
For now, it is necessary to consider the representation of
the symbols used for various concepts when constructing a
visual scene.
REFERENCES
DeLoache, J. S. (1995). Early symbolic understanding and use. In
D. Medin (Ed.), The psychology of learning and motivation (Vol.
33, pp. 65–114). San Diego, CA: Academic Press.
DeLoache, J. S. (2000). Dual representation and young children’s
use of scale models. Child Development, 71, 329–338.
Drager, K. D. R., Light, J. C., Carlson, R., D’Silva, K., Larsson, B., Pitkin, L., & Stopper, G. (2004). Learning of dynamic
display AAC technologies by typically developing 3-year-olds:
Effect of different layouts and menu approaches. Journal of
Speech, Language, and Hearing Research, 47, 1133–1148.
Drager, K. D. R., Light, J. C., Speltz, J. C., Fallon, K. A., &
Jeffries, L. Z. (2003). The performance of typically developing 2½-year-olds on dynamic display AAC technologies with
different system layouts and language organizations. Journal of
Speech, Language, and Hearing Research, 46, 298–312.
Fenson, L., Dale, P., Reznick, J., Bates, E., Thal, D., & Pethick,
S. (1994). Variability in early communicative development.
Monographs of the Society for Research in Child Development,
59(5, Serial No. 242).
Fujisawa, K., Inoue, T., Yamana, Y., & Hayashi, H. (2011). The
effect of animation on learning action symbols by individuals
with intellectual disabilities. Augmentative and Alternative Communication, 27(1), 53–60.
Jagaroo, V., & Wilkinson, K. M. (2006). Further considerations
of visual cognitive neuroscience in AAC design: The potential
role of motion perception systems in maximizing symbol salience. Augmentative and Alternative Communication, 24, 29–42.
CONTEMPORARY ISSUES
IN
COMMUNICATION SCIENCE
AND
Light, J. (2003). “Shattering the silence”: The development of
communicative competence by individuals who require augmentative and alternative communication. In J. Light, D. Beukelman,
& J. Reichle (Eds.), Communicative competence for individuals
who use augmentative and alternative communication (pp. 3–38).
Baltimore, MD: Brookes.
Light, J., & Drager, K. (2007). AAC technologies for young
children with complex communication need: State of the science
and future research directions. Augmentative and Alternative
Communication, 23, 204–216.
Beukelman, D. R., & Mirenda, P. (2005). Augmentative and
alternative communication: Supporting children and adults with
complex communication needs. Baltimore, MD: Brookes.
52
Light, J. (1997). “Let’s go star fishing”: Reflections on the
contexts of language learning for children who use aided AAC.
Augmentative and Alternative Communication, 13(3), 158–171.
Light, J., Drager, K., McCarthy, J., Mellott, S., Millar, D.,
Parrish, C., … Welliver, M. (2004). Performance of typically
developing four-and five-year-old children with AAC systems
using different language organization techniques. Augmentative
and Alternative Communication, 20, 63–88.
Mineo, B. A., Peischl, D., & Pennington, C. (2008). Moving
targets: The effect of animation on identification of action word
representations. Augmentative and Alternative Communication,
24(2), 162–173.
Mineo Mollica, B. (2003). Representational competence. In J.
Light, D. Beukelman, & J. Reichle (Eds.), Communicative competence for individuals who use AAC: From research to effective
practice (pp. 107–145). Baltimore, MD: Brookes.
Nelson, K., Plesa Skwerer, D., Goldman, S., Henseler, S., Presler, N., & Fried Walkenfeld, F. (2003). Entering a community
of minds: An experiential approach to “theory of mind.” Human
Development, 46, 24–46.
Uttal, D., Schreiber, J. C., & DeLoache, J. S. (1995). Waiting to
use a symbol: The effects of delay on children’s use of models.
Child Development, 66, 1875–1891.
Wilkinson, K. M., Carlin, M., & Jagaroo, V. (2006). Preschoolers’ speed of locating a target symbol under different color
conditions. Augmentative and Alternative Communication, 22,
123–133.
Contact author: Jacquelyn Kearns, The Cleveland Clinic
Children’s Hospital for Rehabilitation, 2801 Martin Luther King Jr.
Dr., Cleveland, OH 44104, E-mail: [email protected].
DISORDERS • Volume 39 • 43–53 • Spring 2012
APPENDIX. SCRIPT USED IN THE STUDY
Look up here. See the house? That is Katie’s house. Katie likes to sleep a lot. She will wake up soon.
When she wakes up, we have to find things in and outside the house for Katie to do, so she does
not go to sleep. Katie can’t talk. We have to use the computer to talk for her.
Notes:
Model Completed?
Notes:
Say: Katie is ready to play. Show Katie how to say RIDE.
I look… (point to each room and outside as you look for RIDE).
YES ______
NO _______
I see a boy riding a bicycle. Let’s go Outside. (Push the
Outside Page)
This boy can ride a bike. (Push RIDE)
We wanted RIDE. We picked RIDE!
Now it’s your turn
Open Home Page
Model Completed?
Say: Katie is ready to play. Show Katie how to say RIDE.
Notes:
Let’s look for RIDE. RIDE is outside.
YES ______
NO_______
Look…..
You wanted RIDE. You picked RIDE!
Now you do it. Help Katie
Please Circle One
Say: Katie’s hands got wet. Show Katie how to say DRY.
(If right) You wanted DRY. You picked DRY!
(If wrong) You wanted DRY. Uh-oh you picked (actual)
(If no response) You wanted DRY. Uh-oh!
Model for wrong or NR: We need to find DRY. DRY is in the
Bathroom page because the bathroom is where you clean yourself.
This is the Bathroom page because children are washing
themselves. This is DRY because the boy is drying his hands
with a towel.
DRY
Open Home Page
No Response
Notes:
Right
Wrong (please write response given)
Please Circle One
Say: Katie likes to play a lot. Show Katie how to say CLIMB.
Notes:
CLIMB
(If right) You wanted CLIMB. You picked CLIMB!
(If wrong) You wanted CLIMB. Uh-oh you picked (actual)
(If no response) You wanted CLIMB. Uh-oh!
Model for wrong or NR: We need to find CLIMB. CLIMB is in
the Outside page because it is a place where you can climb. This
is the outside page because you can see people playing outside.
This is CLIMB because the boy is climbing the tree.
Right
Wrong (please write response given)
No Response
Open Home Page
Say: Katie is thirsty. Show her how to say DRINK.
Please Circle One
(If right) You wanted DRINK. You picked DRINK!
(If wrong) You wanted DRINK. Uh-oh you picked (actual)
(If no response) You wanted DRINK. Uh-oh!
Model for wrong or NR: We need to find DRINK. DRINK is in t
he Kitchen page because the kitchen is where you eat food. This
is the Kitchen page because it has a refrigerator and stove. This
is DRINK because the girl is drinking from a cup.
DRINK
Open Home Page
Notes:
Right
Wrong (please write response given)
No Response
Kearns • McCarthy: Animated Feedback and Verb AAC Symbols
53