Effects of Tactile Training on Visual Speechreading:
Performance Changes Related to Individual Differences
in Cognitive Skills
Ulf Andersson
Björn Lyxell
Jerker Rönnberg
Linköping University
Örebro University
Karl-Erik Spens
Royal Institute of Technology
We report on a follow-up study of the Rönnberg, Andersson,
Lyxell, & Spens (1998) speech tracking training study. The
purpose was to examine, initially and after training, the effects
of different tactile aids on tasks of visual speechreading. We
also examined cognitive prerequisites for initial baseline
speechreading and posttraining speechreading performance.
Compared with speechreading alone, tactile aids impaired
sentence-based speechreading at first, although performance
improved with training. No effects of vibrotactile aids or
training were obtained for visual word-decoding. Initial baseline speechreading performance and posttraining performance
correlated with cognitive skills, but the size of the correlations
changed. The size of the correlations also varied with the
different tactually mediated speechreading conditions.
Tactile aids are technical devices that convey sound and
speech information to hearing-impaired individuals by
transforming sound into tactile stimulation on the skin.
The idea is that parts of the speech information that
hearing-impaired individuals cannot obtain auditorily
or visually by means of speechreading could be
obtained using the sense of touch (Plant, 1988; Reed,
Durlach, Delhorne, Rabinowitz, & Grant, 1989;
Weisenberger, 1989). The two major categories of tactile aids, single-channel and multichannel, transmit
different types of speech information to the receiver.
Single-channel tactile aids primarily convey timeintensity information (i.e., prosodic information),
This research is supported by grants from the Swedish Council for Social
Research awarded to Björn Lyxell (97–0319) and grants awarded to Jerker
Rönnberg (30305108). We thank Ulla-Britt Persson for checking the language. Correspondence should be sent to UIf Andersson, Department of
Behavioral Sciences, Linköping University, S-581 83 Linköping, Sweden
(e-mail: [email protected]).
䉷 2001 Oxford University Press
whereas multichannel aids are designed to convey spectral information (i.e., phonemic information; see
Kishon-Rabin, Boothroyd, & Hanin, 1996; Plant, 1988;
Weisenberger, 1989).
This article reports on a follow-up study of the
Rönnberg, Andersson, Lyxell, and Spens (1998)
speech tracking training study. Speech tracking refers
to a procedure employed for measuring, training, and
evaluating reception of continuous speech. In tracking,
a talker reads a text, sentence by sentence, and the
speechreader’s task is to repeat each sentence verbatim
(De Filippo & Scott, 1978; De Filippo, Lansing, Elfenbein, Kallaus-Gay, & Woodworth, 1994). In the
Rönnberg, Andersson, Lyxell, & Spens (1998) study,
the differential effects of tactile aids in speech tracking
were investigated in a within-subjects design. Relatively low-intensity training during 10 weeks caused
substantial improvements in speech tracking rate.
However, no differential effects of tactile aid were obtained, and there was no interaction among training
trials and type of tactile aid. The main finding of the
study was that visual word-decoding skill, verbal information processing speed, and phonological processing
speed represented the cognitive abilities that predicted
individual differences in speech tracking performance.
This pattern of correlations remained invariant across
all four speech tracking conditions (i.e., visual only,
MiniVib 3, Tactilator, and Tact aid 7).
This follow-up study was conducted to further our
understanding of the impact of speech tracking training on speechreading of sentences and words. Previous
research has shown positive effects of tactile aids on
Vibrotactile Speechreading and Cognitive Skills
visual speech understanding (cf. Auer, Bernstein, &
Coulter, 1998; Bernstein, 1995; Bernstein, Eberhart, &
Demorest, 1989; Kishon-Rabin et al., 1996; Plant,
1998; Spens, Huss, Dahlqvist, & Agelfors, 1997;
Weisenberger & Russel, 1989). However, the benefit of
tactile aids is usually not obtained directly but only
after prolonged practice with the device (Lyxell, Rönnberg, Andersson, & Linderoth, 1993; Rönnberg, Andersson, Lyxell, & Spens, 1998; Weisenberger & Russel, 1989). Speechreading and speech tracking skills
also depend on a large range of other skills, and substantial differences in rates of improvement among individuals are typically obtained (e.g., Demorest &
Bernstein, 1992; Dodd & Campbell, 1987). Recent
studies of tactile aid benefit have shown that individual
differences in enhanced speechreading performance
prove to be the rule rather than the exception (Vergara,
Miskiel, Oller, & Eilers, 1998). Thus, the results for
tracking training are not straightforward.
Studies from our own laboratory corroborate the
fact that individual cognitive abilities can account for
substantial portions of the variation in visual and visual-tactile speechreading (see Rönnberg, 1995; Rönnberg, Andersson, Andersson, Johansson, Lyxell, &
Samuelson, 1998, for reviews) and success with cochlear implants (Lyxell et al., 1996; Lyxell et al., 1998).
In addition to the problem of accounting for individual
differences in tactile aid performance, effects of speechreading training depend primarily on characteristics of
the context and the talker (Öhngren, Rönnberg, &
Lyxell, 1992).
The first purpose of this follow-up study was thus
to examine, immediately as well as after 10 hours of
practice, the effects of different types of tactile aids on
visual sentence-based speechreading and visual worddecoding. With the same cognitive tests as in Rönnberg, Andersson, Lyxell, & Spens (1998), our second
purpose was to examine individual cognitive task correlations with sentence-based speechreading performance, before and after speech tracking training, with
visual tracking (i.e., the unaided speechreading condition), visual plus MiniVib 3 tracking, and visual plus
Tact aid 7 tracking. Different sets of correlates might
come into play during different phases of skill development, suggesting that different cognitive skills are related to initial baseline performance compared to the
posttraining prediction.
117
The reason for this assumption is that skill acquisition, in general, takes place in distinct phases and
different cognitive abilities may be more or less critical
for each phase (Ackerman, 1988, 1992; Runeson, Juslin, & Olsson, 2000). During the initial phase (i.e., the
cognitive phase), general cognitive skills determine task
performance, whereas perceptual skills typically determine performance during later phases (i.e., the perceptual phase; Runeson et al., 2000). However, if the task
has inconsistent information-processing skill demands,
(e.g., when the task continuously presents new information to the individual), skill acquisition may not proceed beyond the initial phase and thus task performance will continue to impose demand on general
cognitive skills (Ackerman, 1988, 1992).
Chronological age has been found to be related to
visual speechreading (cf. Dancer, Krain, Thompson,
Davis, et al., 1994; Lyxell & Rönnberg, 1991b; Rönnberg, 1990; Shoop & Binnie, 1979). Accordingly, we examine the possible associations between speechreading
and individual characteristics, such as chronological
age, age of onset of hearing loss, years with hearing
loss, years of hearing aid use, and dB loss. Furthermore, we compute partial correlations to control for the
possible contribution from each of these background
variables to the basic relation between the cognitive
tests and speechreading tests.
Method
Participants
Fourteen severely hearing-impaired participants (including one male participant), ages 21–76 years
(mean ⫽ 52 years, SD ⫽ 16) took part in this study.
Table 1 shows relevant background information. The
mean duration of the subjects’ hearing impairment was
32 years (SD ⫽ 14). Audiograms showed a pure tone
average hearing loss of 75 dB (SD ⫽ 16) for the best
ear according to the most recent available medical records. All participants were native speakers of Swedish
and preferred an oral communication mode.
Tactile Aids
This study employed two different types of vibrotactile
aid. The first, the MiniVib 3 (Ab Specialinstrument,
Stockholm), is a single-channel aid that extracts the in-
118
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
Table 1 Participant descriptions in four background
variables
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Mean
SD
Age
Age of
onset
Years with
hearing aid
Three frequency
average (500 Hz,
1, 2 KHz)
41
52
61
61
76
40
72
39
21
57
64
68
45
37
52
16
31
36
1
34
61
7
32
3
3
7
39
33
1
1
21
19
10
8
19
18
9
12
25
36
17
34
22
32
40
15
21
11
83
70
57
88
93
93
85
60
57
75
77
73
87
70
76
12
tensity contour between 500 and 1800 Hz, but
time/intensity variations are presented to the user at a
fixed 230 Hz frequency. The tactile signal is transmitted to the user via a bone conductor attached to the
user’s wrist by a wristband. Second is the Tact aid 7,
equipped with seven vibrators. Out of these seven
channels, four cover the first formant frequencies, and
four cover the second formant frequencies. Thus, one
channel is shared by the first and second formant analyzers. The seven vibrators were attached to a specially
designed hand prototype, and the participant held the
hand prototype in his or her right hand. Five of the
vibrators stimulated the fingertips, and the two remaining vibrators stimulated the palm (see Tact aid 7
users’ manual [1991] for technical details).
General Procedure
All cognitive and speechreading tests were administered before the participants took part in the Rönnberg,
Andersson, Lyxell, & Spens speech tracking study
(1998), and all testing was conducted individually. The
cognitive tests, except for the two measures of verbal
ability, were administered by a computer program
called TIPS, Text-Information-Processing-System
(Ausmeel, 1988). Thus, all test material was presented
visually (i.e., print).
Speechreading Tests
The speechreading tests were administrated on a Finlux 26⬙ color TV and a video cassette recorder (JVC
HR-7700EG). The tactile aids were connected directly
to the video recorder, which excluded all auditory information from the environment. In all conditions (visual and tactually supported) sound was presented with
the visual and the visual-tactile information, but all
participants performed the speechreading tests with
their hearing aids turned off.
Sentence-based speechreading test. The participants’ task
was to speechread sentences with and without tactile
support. The material included 36 sentences, subdivided into three separate scenarios of twelve sentences
each: a “train scenario,” a “shop scenario,” and a “restaurant scenario.” Based on these 36 sentences, four
different presentation blocks were constructed, first by
subdividing the 36 sentences into two separate balanced lists of 18 sentences each and then by reversing
the presentation order of the 18 sentences in the two
lists. Within each scenario, the participants speechread
two sentences in each one of the three test conditions
(i.e., visual only, MiniVib 3, and Tact aid 7). We counterbalanced the order of the test conditions (i.e., visual,
Tact aid 7, and MiniVib 3) and the four presentation
blocks, as well as the order of the test conditions over
the four presentation blocks. The sentences were presented by a female native speaker of Swedish. The
video recorder was stopped when the participants
wrote down what they had perceived on an answer
sheet. Performance was measured by the proportion of
words correctly perceived. After 10 sessions of speech
tracking training, the participants were tested once
again on the sentence-based speechreading test, using
the other list of sentences than that used in pretesting.
The design for the sentence-based speechreading test
was a 2 ⫻ 3 within-subject factorial design. The first
factor was the training factor (pre/posttraining), and
the second factor referred to the three test conditions
(visual only, MiniVib 3, and Tact aid 7).
Visual word-decoding test. Sixty common Swedish bisyllabic nouns were used. The participants’ task was to
decode these words, presented by the same female actor
as in the sentence-based speechreading task. The principles for construction of presentation blocks and the
Vibrotactile Speechreading and Cognitive Skills
procedure of presenting them were the same as in the
sentence-based speechreading test. The proportion of
words correctly perceived in each one of the three test
conditions was the dependent measure. As in the
sentence-based speechreading test, participants were
tested once again, but on the untested material, after
conducting 10 sessions of speech tracking.
Speech Tracking Test Procedure
In this test a computerized speech tracking procedure
(De Filippo & Scott, 1978; Gnosspelius & Spens, 1992)
was employed. The test material consisted of a simplified and easy-to-read book by Per-Anders Fogelström,
Mina drömmars stad (“The town of my dreams”). The
test procedure was as follows: the experimenter read a
text from a computer screen, sentence by sentence, and
the participant’s task was to repeat each sentence verbatim. Words the participant was unable to perceive
were repeated orally twice. If the speechreader still
could not perceive the word (i.e., after approximately
4–5 sec), it was presented orthographically on an electronic display. The participants speechread for 10 min
(2 ⫻ 5 min) in each of the test conditions, and the test
order was counterbalanced across participants. All participants performed the task with their hearing aids
turned off. Ten training sessions were administered, 1
week apart. The computer program automatically calculated a words-per-minute (wpm) rate by dividing the
total of words speechread during the test session by the
time elapsed.
Written Material Cognitive Tests
Lexical decision speed. The participants had to decide
whether a string of letters was a real word or not. The
test material included 100 items: 50 monosyllabic real
words (e.g., “snö,” SNOW) and 50 monosyllabic lures.
Of the 50 lures, 25 were pronounceable (e.g., “GAR”)
and 25 impossible to pronounce (e.g., “NCI”) The participants responded “yes” or “no” by button press. The
letter string was presented for 2 sec. Latency was measured from onset of the 2-sec interval, and the 2-sec
interval also served as the maximum response time.
After the response, another 2-sec interitem interval
commenced before presentation of the next word. Accuracy and speed of performance were measured.
119
Semantic decision speed. The participants had to decide
whether a word belonged to a predefined semantic category. The test material consisted of four categories
(colors, occupations, diseases, body parts), and each
category consisted of 24 items. Of the 96 (4 ⫻ 24)
items, half belonged to the category. The test procedure was set with a short pause between clusters, and
each category constituted a cluster. The procedure for
presentation, responses, and measure of performance
was identical to that of the lexical decision test.
Phonological Processing Tasks
Rhyme judgment. The stimulus material consisted of
four lists of word-pairs. The first list contained 50 pairs
of monosyllabic and bisyllabic Swedish words. The
word pairs included 13 nonrhyming, visually dissimilar
pairs (e.g., “cykel-päron,” BICYCLE-PEAR); 13 nonrhyming, visually similar words (e.g., “bil-bål,” CARPUNCH); 12 rhyming, visually dissimilar words (e.g.,
“kurs-dusch,” COURSE-SHOWER); and 12 rhyming,
visually similar words (e.g., “sal-bal,” HALL-BALL).
In the second list of 50 pairs of bisyllabic words, each
pair contained one “real” word and one nonword (e.g.,
“citron-mirol,” LEMON-MIROL). The third list contained 30 monosyllabic pairs of nonwords (e.g., PRETBLET). The fourth list contained 30 bisyllabic pairs of
non-words (e.g., KADIR-SPADIR). One pair of words
at a time was displayed on the computer screen, and
the participants’ task was to respond “yes” or “no” by
means of pressing predefined buttons if the two words
rhymed or did not rhyme. The response time was set
at 5 sec, and the word pair disappeared when the button was pressed. Accuracy and speed of performance
were measured.
Working Memory Capacity Tasks
Reading span test. The participants were presented with
sequences of sentences (3, 4, 5, or 6 sentences), each
sentence containing three words. Three different sequences were presented for each span size. The sentences were presented word by word, at a rate of one
word per 0.8 sec and with an interword interval of
0.075 sec. Half of the sentences were absurd (e.g.,
“Fisken körde bilen,” THE FISH DROVE THE
CAR), and half were normal sentences (e.g., “Kaninen
120
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
var snabb,” THE RABBIT WAS FAST). The participants had to read each sentence and answer “yes” (for
a normal sentence) or “no” (for an absurd sentence),
during the 1.75-sec interval before the next sentence
was presented. At the end of the sequence of sentences,
the participants had to recall orally the first or the final
word in each sentence. The participants were instructed that the recall should be in correct serial order.
The response interval was set at 80 sec. However, no
participant needed more than 30 sec to respond. The
experimenter started the next sequence of sentences by
pushing a button. The participants’ responses were
scored by the experimenter in terms of total number of
recalled words.
was divided by the maximum number of deleted words
in each sentence to constitute the basis for the scoring
used, and the average proportion based on the 28 sentences was used as the actual data for each participant.
Digit span. The test material consisted of digits (1–9).
The digits were presented at a rate of one digit per 0.8
sec with an interitem interval of 0.075 sec. Twelve sequences of digits (three sequences for each span size)
were used as test material. The span size ranged from
four to eight digits. The participants were asked to recall orally the digits in correct serial order. As no participant needed more than 30 sec to respond (maximized to 120 sec), presentation of the next series of
digits started after the experimenter pushed a button.
The participants’ responses were scored in terms of total number of digits recalled in correct serial order.
Analogy test. This test was also a paper-and-pencil test,
and the participants’ task was to decide which two
words out of five alternatives that were related to each
other, in a similar (analogous) way as two target words
(e.g., SUGAR-SWEET: SUN, DAY, WHITE,
NIGHT, DARK). The test consisted of 27 five-word
strings and was performed under time pressure (maximum 5 minutes and 30 sec). The scoring procedure
was the same as for the antonym test.
Verbal Inference-Making Tasks
Sentence completion test. The participants were shown 28
sentences that had some words missing. Out of these
28 sentences, 14 were related to a shop scenario, and
the other 14 were related to a restaurant scenario. The
task was to fill in the missing words (e.g., “Kan
jag . . . ett par byxor?,” MAY I . . . A PAIR OF
TROUSERS?). The sentences contained 4 to 13 words
each. All words were familiar common Swedish words
(Allén, 1970). From each sentence two to four words
were omitted. Each incomplete sentence was presented
on the computer screen for 7 sec. Immediately after the
presentation of the sentence, the response interval
started, which was set at 30 sec. The participants had
to complete the sentence orally, and the experimenter
wrote down all answers on an answer sheet. The responses were scored according to their semantic and
syntactic appropriateness. The number of correct words
Verbal Ability Tasks
Antonym test. This test was a paper-and-pencil test in
which the participants were required to judge which
two words out of five were antonyms (e.g., BEAUTIFUL, OLD, SAD, FAST, YOUNG). The test consisted of 29 five-word strings and was performed under
time pressure (maximum 5 minutes). The proportion
of correct responses was measured.
Results
The results are presented in two separate parts. In the
first, we examine the results of the effect of tactile aids
on speechreading performance for pre- and posttraining. In the second, we present a correlational analysis
between the cognitive tests and pre- and posttraining
speechreading performance.
Effects of Tactile Aids on Visual Sentence-Based
Speechreading and Word-Decoding
As the sentence-based speechreading tests and the
word-decoding tests were scored as proportions correct, a root arcsin transformation was applied to normalize the data for parametric analysis. The transformed values were subsequently used in all analyses
of variance (ANOVAs). Figures 1 and 2 display mean
performance for each condition of pre- and posttraining testing (i.e., the non-transformed values). To evaluate the effect of tactile aid before and after tracking training, we computed a 2 ⫻ 3 repeated measures
Vibrotactile Speechreading and Cognitive Skills
Mean proportion speech reading performance
0.45
Minivib3
0.425
0.4
Tact aid VII
Visual
0.375
0.35
0.325
0.3
0.275
Initial baseline performance
Post-training performance
Figure 1 Initial baseline and post-training sentence-based speechreading performance in each condition.
Mean proportion speech reading performance
0.45
Minivib3
0.425
Tact aid VII
Visual
0.4
0.375
0.35
0.325
Initial baseline performance
Post-training performance
Figure 2 Initial baseline and post-training word-decoding performance in each condition.
121
122
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
Table 2 Means, standard deviations, and correlations among initial baseline speechreading
conditions, age, and cognitive tests in the battery, expressed as proportions and speed
MiniVib3
.28 (.24)
Tact aid 7
.29 (.30)
Visual
.37 (.31)
⫺.43
.76**
⫺.43
.60*
⫺.72**
.63*
.45 (.19)
.44 (.27)
.31
.40
.37
.48
.38
.59*
.72 (.11)
.41
.37
.37
.42 (.11)
.79 (.05)
.11
.21
.20
.32
.38
.32
.73 (.15)
.78 (.17)
⫺.49
⫺.66**
⫺.47
⫺.63*
⫺.68**
⫺.77**
1.44 (.44)
1.61 (.52)
1.69 (.54)
1.74 (.70)
1.60 (.52)
⫺.62*
⫺.70**
⫺.73**
⫺.74**
⫺.72**
⫺.53
⫺.58*
⫺.70**
⫺.70**
⫺.64*
⫺.67**
⫺.68**
⫺.77**
⫺.72**
⫺.73**
X (SD)
Age (yrs)
Word-decoding
Verbal ability
Antonym test
Analogy test
Inference making
Sentence-completion
Short-term/working memory
Reading span
Digit span
Verbal information-processing speed
Semantic decision speed
Lexical decision speed
Rhyme judgment
Word-pairs
Pairs of words and nonwords
Pairs of nonwords (monosyllabic)
Pairs of nonwords (bisyllabic)
Rhyme overall
52 (16)
.34 (.22)
*p ⬍ .05.
**p ⬍ .01.
ANOVA. The first factor was the pre/posttraining
variable, and the three speechreading conditions (visual, Tact aid 7, and MiniVib 3) constituted the second
factor. The analysis showed an interaction between the
two variables, F(2, 26) ⫽ 3.56, p ⬍ .05, MSe ⫽ .06).
Tests of simple effects revealed that this interaction
effect was due to pretraining performance of the visual
condition being significantly better than that of the
Tact aid 7 device and the MiniVib 3 device, F(2, 52) ⫽
3.43, p⬍ .05, MSe ⫽ .06), and to a significant improvement in speechreading performance between pre- and
posttesting with the Tact aid 7 device, F(1, 39) ⫽ 10.20,
p⬍ .05, MSe ⫽ .07).
A corresponding 2 ⫻ 3 repeated measures ANOVA
was computed for the pre- and posttest measures of the
word-decoding test. This ANOVA did not reveal any
significant effects (see Figure 2).
Cognitive Correlates of Sentence-Based
Speechreading Performance
To examine the patterns of correlations for visual and
visual-tactile speechreading performance and whether
different cognitive skills are related to initial baseline
performance compared to the posttraining prediction,
we performed correlational analyses. The results of
these analyses and descriptive statistics are displayed in
Tables 2 and 3.
Initial baseline speechreading. The speed measures (Table
2) of the lexical decision test and the rhyme judgment
tests were all significantly correlated with sentencebased speechreading in all three conditions. In addition, the accuracy scores on the rhyme judgment test,
including a comparison of pairs of monosyllabic nonwords, showed a substantial association with visual
speechreading (r ⫽ .57, p ⬍ .05) and speechreading
mediated by the Tact aid 7 device (r ⫽ .59, p ⬍ .05).
Visual word-decoding proved also to be significantly
related to visual and visual-tactile speechreading. In
contrast, only one correlation (the analogy test) between the tests of verbal ability, verbal inference making, working memory, and speechreading performance
reached significance.
Chronological age was significantly correlated with
visual speechreading, but not with vibrotactile-
Vibrotactile Speechreading and Cognitive Skills
123
Table 3 Means and standard deviations for the posttraining speechreading conditions,
expressed as proportions, and correlations among speechreading conditions, age, and
cognitive tests in the battery
Age (yrs)
Word-decoding
Verbal ability
Antonym test
Analogy test
Inference making
Sentence-completion
Short-term/working memory
Reading span
Digit span
Verbal information-processing speed
Semantic decision speed
Lexical decision speed
Rhyme judgment
Word-pairs
Pairs of words and nonwords
Pairs of nonwords (monosyllabic)
Pairs of nonwords (bisyllabic)
Rhyme overall
MiniVib3
.33 (.28)
Tact aid 7
41 (.26)
Visual
.36 (.32)
⫺.54*
.66**
⫺.54*
.66*
⫺.43
.63*
.26
.41
.33
.49
.19
.29
.37
.50
.32
.24
.54*
.12
.38
.09
.34
⫺.44
⫺.59*
⫺.54*
⫺.71**
⫺.44
⫺.63*
⫺.53*
⫺.59*
⫺.62*
⫺.61*
⫺.60*
⫺.64*
⫺.68**
⫺.69**
⫺.72**
⫺.71**
⫺.54*
⫺.63*
⫺.68**
⫺.66*
⫺.64*
*p ⬍ .05.
**p ⬍ .01.
supported speechreading. None of the other individual
variables correlated with speechreading performance.
Posttraining speechreading performance. The correlations
between the word-decoding test, the verbal information-processing speed tasks (lexical decision, rhyme
judgment) and visual speechreading, and speechreading supported with MiniVib 3 were still significant.
The accuracy scores on the rhyme judgment test, including a comparison of pairs of monosyllabic nonwords, were not significantly correlated with posttraining speechreading performance in any condition.
In contrast, the associations between the worddecoding test and the information-processing speed
tasks and the Tact aid 7–mediated posttraining speechreading were similar to those for initial baseline speechreading performance. The semantic decision speed test
and the rhyme judgment of word-pairs also correlated
significantly with Tact aid 7–mediated posttraining
speechreading, in contrast to initial baseline speechreading performance.
Chronological age was again the only background
variable significantly related to speechreading (see
Table 3). However, the relationships were now with the
MiniVib 3 and Tact aid 7–mediated speechreading
conditions and not with the visual condition.
As a second general step in the analysis, partial correlations were calculated to examine whether our cognitive tests would still predict speechreading performance (initial baseline and posttraining) when the
effect of chronological age was partialled out. The outcomes of these analyses are displayed in Tables 4 and 5.
As can be seen in Table 4, significant correlations
remained only with the word-decoding test and the
speed measures of the rhyme judgment tests, except for
the test that included word-pairs. The correlation coefficients for the word-decoding test remained at the
same magnitude, whereas the relationships with rhyme
judgment speed were overall weaker, but still significant. Thus, neither lexical decision speed nor rhyme
judgment accuracy was a statistically significant correlate when age was statistically controlled for.
For posttraining speechreading performance, the
correlation coefficients for the word-decoding test did
124
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
Table 4 Partial correlations among initial baseline speechreading conditions, age, and
cognitive tests in the battery, controlling for age
Age (yrs)
Word-decoding
Verbal ability
Antonym test
Analogy test
Inference making
Sentence-completion
Short-term/working memory
Reading span
Digit span
Verbal information-processing speed
Semantic decision speed
Lexical decision speed
Rhyme judgment
Word-pairs
Pairs of words and nonwords
Pairs of nonwords (monosyllabic)
Pairs of nonwords (bisyllabic)
Rhyme overall
MiniVib3
Tact aid 7
Visual
⫺.43
.75**
⫺.43
.57*
⫺.72**
.69*
.24
.24
.30
.34
.31
.39
.35
.31
.31
.11
.22
.01
.34
.08
.43
⫺.26
⫺.55*
⫺.23
⫺.51
⫺.23
⫺.51
⫺.50
⫺.62*
⫺.66*
⫺.67**
⫺.64*
⫺.38
⫺.47
⫺.61*
⫺.64*
⫺.54*
⫺.43
⫺.54*
⫺.64*
⫺.65*
⫺.58*
*p ⬍ .05.
**p ⬍ .01.
Table 5 Partial correlations among posttraining speechreading conditions, age, and
cognitive tests in the battery, controlling for age
Age (yrs)
Word-decoding
Verbal ability
Antonym test
Analogy test
Inference making
Sentence-completion
Short-term/working memory
Reading span
Digit span
Verbal information-processing speed
Semantic decision speed
Lexical decision speed
Rhyme judgment
Word-pairs
Pairs of words and nonwords
Pairs of nonwords (monosyllabic)
Pairs of nonwords (bisyllabic)
Rhyme overall
*p ⬍ .05.
**p ⬍ .01.
MiniVib3
Tact aid 7
Visual
⫺.54*
.65*
⫺.54*
.65*
⫺.43
.60*
.16
.21
.25
.31
.10
.10
.30
.46
.26
⫺.01
.62*
⫺.15
.44
⫺.12
.37
⫺.03
⫺.34
⫺.20
⫺.55*
⫺.18
⫺.51
⫺.31
⫺.45
⫺.46
⫺.50
⫺.45
⫺.47
⫺.56*
⫺.56*
⫺.64*
⫺.59*
⫺.39
⫺.53*
⫺.59*
⫺.58*
⫺.54*
Vibrotactile Speechreading and Cognitive Skills
not change when the effect of age was partialled out
(Table 5). As in baseline speechreading, the speed measures of the rhyme judgment tests, except for the test
that included word-pairs, continued to be significantly
related to visual and Tact aid 7–supported speechreading, but not for MiniVib 3–mediated speechreading.
Furthermore, the lexical decision test (speed) was still
significantly correlated with Tact aid 7–supported
speechreading.
Although this sample may be regarded as small, the
pattern of absolute test performance levels replicates
previously reported empirical patterns with larger
samples. First, speed performance in the semantic decision test was significantly faster than for the lexical
decision test (t[13] ⫽ 3.17, p⬍ .05), which is consistent
with other studies using similar methodology (Lyxell &
Rönnberg, 1992; Lyxell et al., 1996), and the performance levels for these two tests were comparable to
what usually has been reported in the literature (Hunt,
1985; Lyxell & Rönnberg, 1992; Lyxell et al., 1996;
Rönnberg, 1990). Performances on the working memory tests (i.e., reading span, digit span) and the rhyme
judgment tests were also within range of those reported
in previous studies (cf. Lyxell et al., 1996; Rönnberg,
1990; Rönnberg, Arlinger, Lyxell, & Kinnefors, 1989).
Chronological age was, as expected, significantly related to the lexical and semantic decision speed tests, the
rhyme judgment tests, and speechreading performance
(cf. Birren & Fisher, 1995; Lyxell & Rönnberg, 1991b;
Rönnberg, 1990; Shoop & Binnie, 1979). Furthermore,
the correlations between the cognitive test and the tests
of visual speech understanding were consistent with
previous research (see Rönnberg, 1995, for a review).
Discussion
These results regarding the effects of different types of
tactile aid on visual sentence-based speechreading and
word-decoding can be summarized in four main points.
First, and consistent with some previous research
(Lyxell et al., 1993; Rönnberg, Andersson, Lyxell, &
Spens, 1998; Weisenberger & Russel, 1989), this study
provides further empirical support for the notion that
tactile aids do not enhance speechreading performance
immediately. If anything, providing additional tactile
information seems to initially interfere with the auto-
125
mation of speech processing and, consequently, cause
a decline in speechreading performance (cf. Lyxell et
al., 1993).
One possible explanation for the initial decline in
vibrotactile-mediated sentence-based speechreading
might be related to problems in the integration of visual and tactile information. If tactile aids have the potential of improving visual speechreading performance,
the two channels (i.e., visual and tactile), which both
separately provide poorly specified information, must
be effectively (i.e., automatically) integrated into one
percept (Öhngren et al., 1992; Plant, 1988; Rönnberg,
1993; Rönnberg, Andersson, Andersson, Johansson,
Lyxell, & Samuelson, 1998; Summerfield, 1987). As
the complementarities between the visual and tactile
modalities are less natural, or at least less common,
than the complementarities between the auditory and
visual modalities (see Summerfield, 1987), it seems
reasonable to assume that this integration process is
not functioning automatically at once. Until this integration process begins to function smoothly, it will interfere with the processing of visual speech information (cf. Ackerman, 1998, 1992). Thus, to obtain tactile
benefit, long-term practice is definitively required (cf.
Bernstein et al., 1989; Cowan, Galvin, Blamey, & Sarant, 1995; Kishon-Rabin et al., 1996; Plant, 1998;
Weisenberger & Russel, 1989).
Second, although there was no main effect of tactile
aid, the training ⫻ condition interaction showed that
only 10 ⫻ 10 minutes of vibrotactile tracking training
gave rise to a substantial increase in sentence-based
speechreading performance with the Tact aid 7 device.
No such increase was obtained for the visual condition,
which indicates that the improvement with the Tact aid
7 device is a genuine, aid-specific training effect. Thus,
although this study was not designed as a transfer-oftraining test, there appears to be transfer of training for
the Tact aid 7. However, the amount of practice used
in this study was not enough to obtain a significant improvement over unaided visual speechreading.
As previous tactile training studies vary widely
with respect to type and degree of training employed,
it is difficult to obtain an answer to the question of what
amount of training is sufficient to obtain a vibrotactile
benefit over visual speechreading alone. Most studies
obtaining a vibrotactile benefit have had their partici-
126
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
pants practice for at least 50–80 hours over a period
of 3–4 months (e.g., Bernstein, Demorest, Coulter, &
O’Connel, 1991; Lynch, Eilers, Oller, Urbano, & Pero,
1989; see also Kishon-Rabin et al., 1996, for a summary). A few studies have, on the other hand, showed
that tactile aids can enhance visual speechreading performance after only 10–20 hours of training (Boothroyd & Hnath-Chisolm, 1988; see Kishon-Rabin et al.,
1996). At any rate, it seems that at least three months
of training is required to obtain a tactile benefit, if in
fact there is one.
Although, the initial performance levels for the
Tact aid 7 and MiniVib 3 devices were equal, only the
Tact aid 7 device showed a substantial training effect.
It is possible that the single-channel MiniVib 3 device
does not provide enough or the right sort of information (i.e., suprasegmental) critical for an improvement
in sentence-based speechreading performance (cf.
Waldstein & Boothroyd, 1995). Alternatively, a sufficiently long period of training was not used.
Third, only sentence-based speechreading tests
showed any effects of aid use. The initial decline in performance when providing vibrotactile information was
not observed in the word-decoding tests. Ten sessions
of speech tracking training failed to improve the worddecoding level. Thus, the effects of vibrotactile aids
and training differed between the sentence-based
speechreading and word-decoding tests, providing further support for the conclusion that the training effect
for the Tact aid 7 is genuine. The lack of an initial interference effect in the word-decoding tests might depend on the fact that this test is linguistically less complex than the sentence-based speechreading test.
Consequently, the speechreader has to process only a
small amount of additional tactile information during
vibrotactile word-decoding, which imposes a minor
perceptual and cognitive interference. For more complex materials such as sentences, the tactile aid (i.e.,
Tact aid 7) also conveys linguistic information not present in speechreading of single words (e.g., sentence
prosodic information; Kishon-Rabin et al., 1996; Kjelgaard & Speer, 1999).
This can also serve as one explanation as to why
performance on the Tact aid 7 did not improve at the
word-decoding level. Specifically, the main potential of
tactile aids to improve visual sentence-based speech-
reading might be that they provide suprasegmental
information to the speechreader (see Waldstein &
Boothroyd, 1995). To improve speechreading of single
words, the individual has to rely on the phonemic information provided by the aid. Although this spectral
information is available and useful when the task is to
discriminate between phonemes (Oller, Payne, &
Gavin, 1980; Weisenberger, 1989; Sparks, Kuhl, Edmonds, & Gray, 1978), it is difficult to extract and
benefit from this information, due to co-articulation,
in speechreading at the word or sentence level (Lynch,
Oller, & Eilers, 1989; Sparks, Ardell, Bourgeois,
Wiedmer, & Kuhl, 1979).
Fourth, the failure to obtain any practice effect for
the word-decoding tests suggests that the tracking procedure employed may not be suitable to demonstrate
transfer to this type of visual speech task. This is not
surprising, assuming that this task primarily taps the
early perceptual stages of visual speech processing and
also seems relatively unaffected by contextual information available during speech tracking (see Lyxell &
Rönnberg, 1991a; Rönnberg, 1990). A more analytic
type of training strategy may be required to enhance
the individual’s decoding ability (Kishon-Rabin et al.,
1996; Plant, 1986).
Correlations with cognitive tasks replicated previous research; initial baseline sentence-based speechreading performance correlated with visual worddecoding skill, lexical decision speed, phonological
processing speed, and quality of phonological representations (see Rönnberg, 1995; Rönnberg, Andersson,
Andersson, et al., 1998, for a review). This pattern remained constant across all three conditions. The same
cognitive skills were associated with visual and visualtactile posttraining speechreading performance although the magnitude of correlations was slightly
smaller for the MiniVib 3 and visual condition.
Chronological age was again related to initial baseline visual speechreading performance (cf. Dancer et
al., 1994; Lyxell & Rönnberg, 1991b; Rönnberg, 1990;
Shoop & Binnie, 1979), but not to visual-tactile
speechreading. In contrast, age was related to the visual-tactile speechreading performance after 10 sessions
of speech tracking, but not to visual speechreading.
After we controlled for chronological age, only worddecoding and rhyme judgment speed remained sig-
Vibrotactile Speechreading and Cognitive Skills
nificant correlates of initial baseline visual and visualtactile speechreading performance.
The corresponding analysis for posttraining
speechreading performance also showed that worddecoding was a significant predictor of visual and
visual-tactile speechreading. Individual differences in
visual and Tact aid 7–mediated speechreading were
still related to rhyme judgment speed. Furthermore,
lexical decision speed accounted for a substantial portion of the variation in Tact aid 7–mediated speechreading performance.
Thus, these correlational patterns show that the
same cognitive skills are associated with initial baseline
performance and posttraining performance. The reduced correlation values in posttraining performance,
for the MiniVib 3 and visual condition, may suggest
that training reduces the demands on these cognitive
skills (cf. Ackerman, 1988, 1992; Runeson et al., 2000).
An important finding is that the Tact aid 7 created
a different pattern of correlations. We interpret the
combination of correlational patterns and the improvement in Tact aid 7–supported speechreading performance as follows: during vibrotactile speechreading,
two sources of information, visual and tactile, are provided to the speechreader. Although this may enhance
speechreading performance after some practice, a
larger amount of information must also be processed
by the speechreader. Thus, to take advantage of the
Tact aid 7, the speechreader must possess cognitive
processing skills that enable him or her to process the
additional linguistic information provided by the aid.
If not, the aid will only interfere with the automation
of processing speech.
The general explanation as to why the similar correlational patterns were not displayed for the MiniVib
3 might be that it represents a single-channel aid, providing only time and intensity information. The Tact
aid 7, on the other hand, conveys a broad spectrum of
speech information (e.g., fundamental frequency and
phonemic information). That is, speechreading with
single-channel aids is not as cognitively demanding as
with multichannel aids simply because they provide
less linguistic information to process.
In sum, the following conclusions can be made.
Tactile aids need not enhance sentence-based speechreading performance immediately. If anything, provid-
127
ing additional tactile information seems to initially
cause a decline in speechreading performance. To obtain a tactile benefit in sentence-based speechreading,
practice is required. Ten sessions of vibrotactile speech
tracking training resulted in a substantial improvement
in vibrotactile (i.e., Tact aid 7) sentence-based speechreading, but posttraining sentence-based speechreading performance did not exceed that of the visual only
condition. In contrast to sentence-based speechreading, no effects of vibrotactile aids and training were obtained for the ability to decode single words. The results of the cognitive analysis suggest that the same
cognitive skills are important when predicting initial
baseline performance and posttraining performance.
However, the relative strength of correlations differs
between initial baseline and posttraining performance,
and these differences vary among the speechreading
conditions. The explanation may be sought in the cognitive demands imposed by the particular level of information complexity provided by the aid. Finally, chronological age accounts for a significant proportion of
the individual differences in visual and visual-tactile
speechreading.
Received October 6, 1999; revision received April 13, 2000; accepted October 5, 2000
References
Ackerman, P. L. (1988). Determinants of individual differences
during skill acquisition: Cognitive abilities and information
processing. Journal of Experimental Psychology: General,
117, 288–318.
Ackerman, P. L. (1992). Predicting individual differences in
complex skill acquisition: Dynamics of ability determinants.
Journal of Applied Psychology, 77, 598–614.
Allén, S. (1970). Frequency dictionary of present-day Swedish. (In
Swedish: Nusvensk frekvensbok.) Stockholm: Almqvist &
Wiksell.
Auer, E. T. Jr., Bernstein, L. E., & Coulter, D.C. (1998). Temporal and spatio-temporal vibrotactile displays for voice fundamental frequency: An initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearingimpaired individuals. Journal of the Acoustical Society of
America, 104, 2477–2489.
Ausmeel, H. (1988). TIPS (Text-Information-ProcessingSystem): A user’s guide. Linköping: Department of Education and Psychology, Linköping University, Sweden.
Bernstein, L. E. (1995). Toward future tactile aids. In G. Plant &
K.-E. Spens (Eds.), Profound deafness and speech communication (pp. 147–162). London: Whurr.
128
Journal of Deaf Studies and Deaf Education 6:2 Spring 2001
Bernstein, L. E., Demorest, M. E., Coulter, D.C., & O’Connell,
M. P. (1991). Lipreading sentences with vibrotactile vocoders: Performance of normal-hearing and hearing-impaired
subjects. Journal of the Acoustical Society of America, 90,
2971–2984.
Bernstein, L. E., Eberhart, S. P., & Demorest, M. E. (1989).
Single-channel vibrotactile supplements to visual perception of intonation and stress. Journal of the Acoustical Society
of America, 85, 397–405.
Birren, J. E., & Fisher, L. M. (1995). Speed of behavior: Possible
consequences for psychological functioning. Annual Review
of Psychology, 46, 329–353.
Boothroyd, A., & Hnath-Chisolm, T. (1988). Spatial, tactile presentation of voice fundamental frequency as a supplement
to lipreading: Results of extended training with a single subject. Journal of Rehabilitation Research and Development, 25,
51–56.
Cowan, R. S. C., Galvin, K. L. Blamey, P. J., & Sarant, J. S.
(1995). Design fundamentals for electrotactile devices: The
Tickle talker case study. In G. Plant & K.-E. Spens (Eds.),
Profound deafness and speech communication (pp. 56–88). London: Whurr.
Dancer, J., Krain, M., Thompson, C., Davis, P. et al. (1994). A
cross-sectional investigation of speechreading in adults:
Effects of age, gender, practice, and education. Volta Review,
96, 31–40.
De Filippo, C. L., Lansing, C. R., Elfenbein, J. L., Kallaus-Gay,
A., & Woodworth, G. G. (1994). Deriving passage difficulties for a tracking study via the close technique. Journal of
the American Academy of Audiology, 5, 366–378.
De Filippo, C. L., & Scott, B. L. (1978). A method for training
and evaluating the reception of ongoing speech. Journal of
the Acoustical Society of America, 63. 1186–1192.
Demorest, M. E., & Bernstein, L. E. (1992). Sources of variability in speechreading sentences: A generalizability analysis.
Journal of Speech and Hearing Research, 35, 876–891.
Dodd, B., & Campbell, R. (1987). Hearing by eye: The psychology
of lipreading. London: Erlbaum.
Gnosspelius, J., & Spens, K.-E. (1992). A computer-based
speech tracking procedure. Speech Transmissions Laboratory
Quarterly Progress and Status Report, 2, 131–137.
Hunt, E. (1985). Verbal ability. In R. J. Sternberg (Ed.), Human
abilities: An information processing approach (pp. 31–58). New
York: Freeman.
Kishon-Rabin, L., Boothroyd, A., & Hanin, L. (1996). Speechreading enhancement: A comparison of spatial-tactile display of voice fundamental frequency (F0) with auditory F0.
Journal of the Acoustical Society of America, 100, 593–602.
Kjelgaard, M. M., & Speer, S. H. (1999). Prosodic facilitation
and interference in the resolution of temporary syntactic
closure ambiguity. Journal of Memory and Language, 40,
153–194.
Lynch, M. P., Eilers, R. E., Oller, D. K., Urbano, R. C., & Pero,
P. J. (1989). Multisensory narrative tracking by a profoundly
deaf subject using an electrocutaneous vocoder and a vibrotactile aid. Journal of Speech and Hearing Research, 32,
331–338.
Lynch, M. P., Oller, K. D., & Eilers, R. E. (1989). Portable tactile aids for speech perception. Volta Review, 91, 113–126.
Lyxell, B., Andersson, J., Andersson, U., Arlinger, S., Bredberg,
G., & Harder, H. (1998). Phonological representation and
speech understanding with cochlear implants in deafened
adults. Scandinavian Journal of Psychology, 39, 175–179.
Lyxell, B., Andersson, J., Arlinger, S., Bredberg, G., Harder,
H., & Rönnberg, J. (1996). Verbal information-processing
capabilities and cochlear implants: Implications for preoperative predictors of speech understanding. Journal of Deaf
Studies and Deaf Education, 1, 190–201.
Lyxell, B., & Rönnberg, J. (1991a). Visual speech processing:
Word-decoding and word-discrimination related to
sentence-based speechreading and hearing-impairment.
Scandinavian Journal of Psychology, 32, 9–17.
Lyxell, B., & Rönnberg, J. (1991b). Word discrimination and
chronological age related to sentence-based speech-reading
skill. British Journal of Audiology, 25, 3–10.
Lyxell, B., & Rönnberg, J. (1992). The relationship between verbal ability and sentence-based speechreading. Scandinavian
Audiology, 21, 67–72.
Lyxell, B., Rönnberg, J., Andersson, J., & Linderoth, E. (1993).
Vibrotactile support: Initial effects on visual speech perception. Scandinavian Audiology, 22, 179–183.
Oller, D. K., Payne, S. L., & Gavin, W. J. (1980). Tactual speech
perception by minimally trained deaf subjects. Journal of
Speech & Hearing Research, 23, 769–778.
Öhngren, G., Rönnberg, J., & Lyxell, B. (1992). Tactiling: A usable support system for speechreading? British Journal of
Audiology, 26, 167–173.
Plant, G. (1986). A single-transducer vibrotactile aid to lipreading. Speech Transmissions Laboratory Quarterly Progress and
Status Report, 1, 41–63.
Plant, G. (1988). Speechreading with tactile supplements. Volta
Review, 90, 149–160.
Plant, G. (1998). Training in the use of a tactile supplement to
lipreading: A long-term case study. Ear and Hearing, 19,
394–406.
Reed, C. M., Durlach, N. I., Delhorne, L. A., Rabinowitz,
W. M., & Grant, K. W. (1989). Research on tactual communication of speech: Ideas, issues, and findings. Volta Review,
91, 65–78.
Rönnberg, J. (1990). Cognitive and communicative function:
The effects of chronological age and “handicap age.” European Journal of Cognitive Psychology, 2, 253–273.
Rönnberg, J. (1993). Cognitive characteristics of skilled tactiling:
The case of GS. European Journal of Cognitive Psychology,
5, 19–33.
Rönnberg, J. (1995). What makes a skilled speechreader? In
G. Plant & K.-E. Spens (Eds.), Profound deafness and speech
communication (pp. 393–416). London: Whurr.
Rönnberg, J., Andersson, J., Andersson, U., Johansson, K., Lyxell, B., & Samuelsson, S. (1998). Cognition as a bridge between signal and dialogue: Communication in the hearing
impaired and deaf. Scandinavian Audiology, 27(suppl 49),
101–108.
Rönnberg, J., Andersson, U., Lyxell, B., & Spens, K-E. (1998).
Vibrotactile speech tracking support: Cognitive prerequisites. Journal of Deaf Studies and Deaf Education, 3, 143–156.
Rönnberg, J., Arlinger, S., Lyxell, B., & Kinnefors, C. (1989).
Visual evoked potentials: Relation to adult speechreading
Vibrotactile Speechreading and Cognitive Skills
and cognitive function. Journal of Speech and Hearing Research, 32, 725–735.
Runeson, S., Juslin, P., & Olsson, H. (2000). Visual perception of
dynamic properties: Cue heuristics versus direct-perceptual
competence. Psychological Review, 107, 525–555.
Shoop, C., & Binnie, C. A. (1979). The effect of age upon the
visual perception of speech. Scandinavian Audiology, 8, 3–8.
Sparks, D. W., Ardell, L. A., Bourgeois, M., Wiedmer, B., &
Kuhl, P. K. (1979). Investigating the MESA (multipoint
electrotactile speech aid): The transmission of connected
discourse. Journal of the Acoustical Society of America, 65,
810–815.
Sparks, D. W., Kuhl, P. K., Edmonds, A. E., & Gray, G. P.
(1978). Investigating the MESA (multipoint electrotactile
speech aid): The transmission of segmental features of
speech. Journal of the Acoustical Society of America, 63,
246–57.
Spens, K-E., Huss, C., Dahlqvist, M., & Agelfors, E. (1997). A
hand held two-channel vibro-tactile speech communication
aid for the deaf: Characteristics and results. Scandinavian
Audiology, 26(suppl 47), 7–13.
129
Summerfield, Q. (1987). Some preliminaries to a comprehensive
account of audio-visual speech perception. In B. Dodd &
R. Campbell (Eds.), Hearing by eye: The psychology of lipreading (pp. 3–51). London: Erlbaum.
Tactaid 7: User’s manual. (1991). Somerville, MA: Audiological
Engineering Corporation.
Vergara, K. C., Miskiel, L. W., Oller, D. K., & Eilers, R. E.
(1998). A longitudinal study of children’s speech perception skills
using hearing aids and tactile aids. Paper presented at ISAC
98, Portland, Maine.
Waldstein, R. S., & Boothroyd, A. (1995). Speechreading supplemented by single-channel and multichannel tactile displays of voice fundamental frequency. Journal of Speech
Hearing Research, 38, 690–705.
Weisenberger, J. M., (1989). Tactile aids for speech perception
and production by hearing-impaired people. Volta Review,
91, 79–100.
Weisenberger, J. M., & Russel, A. F. (1989) Comparison of two
single-channel vibrotactile aids for the hearing-impaired.
Journal of Speech Hearing Research, 32, 83–92.