Dept. for Speech, Music and Hearing
Quarterly Progress and
Status Report
Comparisons between
different measurements of
the auditory, visual, and
audiovisual speech
perception ability of hearing
impaired subjects; Some preli
Öhngren, G. and Dahlquist, M.
journal:
volume:
number:
year:
pages:
STL-QPSR
29
4
1988
023-030
http://www.speech.kth.se/qpsr
STL-QPSR 411988
C.
COMPARISONS BETWEEN DIFFERENT MEASUREMENTS OF THE AUDITORY, VISUAL, AND
AUDIOVISUAL SPEECH PERCEPTION ABILITY
OF HEARING IMPAIRED SUBJECTS; SOME
PRELIMINARY RESULTS
Gunilla Ohngren and Martin Dahlquist
Abstract
The aim of this study is to give a preliminary base for a communication
profile, which could be used in rehabilitative work when designing an individual training programme. A comparison is made between hearing impaired persons' results from auditory measurements (PTA, speech discrimination in quiet and noise) and their results from different speechreading
tests, both visual and audiovisual. The answers on a questionnaire, Nearing
Performance Inventory (HPI), - in which the hard of hearing person evaluates his difficulties in different situations - are correlated with the results
from tests, as well as with the duration of loss. A new method to measure
audiovisual speech perception in noise is described. Subjects were 13
prelingually hearing impaired persons. There was a significant correlation
(p<.05) between the results on the five tests measuring speechreading ability
and the duration of loss. One test of speech perception in noise correlated
(p<.05) with HPI-answers concerning auditory speech perception, the reaction to auditory failure, and the hearing impaired person's experiences in his
social situation. The results from the new audiovisual test did not give any
significant correlation with the HPI-measures.
Introduction
"Any restriction or lack of ability to perceive sounds in either quiet or noisy backgrounds within the range considered as normal," is the definition of hearing disability
made by World Health Organization (WHO) 1980. For a person with a hearing disability, it is important to get the most effective support system to the distorted auditory
system. For many hard of hearing persons, a well fitted hearing aid will be a satisfactory
solution, even if they will have more or less pronounced difficulties in the perception of
speech in noise. The evaluating procedure used at the audiologic clinic then gives - for
most of the hearing impaired persons - enough information to fit a hearing aid. The opthnum selection of gain, frequency response, use of amplitude compression, etc of the
hearing aid is, however, still under discussion.
There are some hard of hearing persons, who do not get enough support by the hearing aid, especially in noise (Plomp, 1978).They need a rehabilitation period to get used
to the hearing aid and to learn to make use of other cues important for the speech perception process as speechreading and the possibility to combine auditory and visual information.
When designing a rehabilitation programme for a hearing impaired, it would be very
useful, if he could go through a battery of tests that measure different aspects of his
communication difficulties. The test results then form a communication profile. Based
on this profile, the hearing impaired person can be placed into an individualised training
programme. The ultimate goal of our work is the design of such a communication profile.
At the Hearing Institute in Uppsala, this strategy has been used for some years. The
test battery consists of a number of standard audiological tests as well as speecbreadhg
tests and questionnaires. The aim of the present study is to get a preliminary evaluation
of the different tests used and some results from a newly developed method of measuring auditory and audiovisual speech-perception ability in noise. Although this last test
situation is the situation that corresponds best with a person's real-life situation, there is
- at present - no standard procedure for a test of this type. There is, however, a growing
interest for this test procedure and a number of different tests have beem wied
(Middlewerd & Plomp, 1987; Hawkins, Montgomery, Meuller, & Sedge, 1988).
Subjects
In the study, 13 hearing impaired persons, ten women and three men, participated.
They were enrolled in a rehabilitation programme at the Hearing Institute in Uppsala.
All had a sensorineural hearing impairment. They wore hearing aids and communicated
orally, but they all expressed problems in many communication situations. The mean
age was 41 years, range 21-70. The mean hearing loss (500, 1 000,2 000 Hz) was 64
dB, with the range 33-93 dB. They were tested with the below described test-battery at
the onset of the rehabilitation course.
Description of tests
In the following, the different tests administered are described. For each test, the
designation used in the table and the correlation matrix is given within parentheses. The
tests are divided into four subgroups:
A. Auditory tests to get information concerning the impaired auditory system.
The auditory tests used were i) Pure Tone Audiometry (PTA), and ii) Speech Discrimination (Disc) measured by standard Swedish phonetically balanced (PB) word lists
presented in quiet over headphones. Results are given for the hearing aided ear and, if
the subject uses two hearing aids, for the best ear. iii) Speech Discrimination in noise
(BH SIN) measured by Hagerman's test (Hageman, 1984). This test consists of fivewords sentences with the construction: fust name, verb, number, adjective, and noun. A
sentence is constructed based on a choice between ten alternatives in each position. The
test is presented in a speech-shaped noise and the signal-to-noise-ratio is established
where 50% of the words in the sentences are correctly identified. The measurement was
made with the fitted hearing aid in a free field. iv) In the last test on this level, the signal-to-noise-ratio was established, at which the subject estimated that he could just urtderstand continuous discourse (Subj.SIN;aud), which is a method based on methods
described by Egan (1948) and Hawkins, & al. (1988) . A short story was recorded on
videotape. The subject was first asked to adjust the level of the speech to a comfortable
level in quiet without visual information. Noise was then added to the speech signal,
and the subject was asked to adjust the noise level so that he could just follow the
speaker. He was also told that the tape could be stopped, and he had to answer some
questions about the content of the story. The adjustment of the noise level was made
eight times and the median value of the adjusted signal-to-noise-ratios was calculated.
The signal was presented over headphones (TDH39) on the hearing aided ear. The noise
was pink-shaped.
STL-QPSR 411988
B. Visual speechreading test to measure the subject's possibility to speechread.
The speechreading tests used are the tests described by Risberg & Agelfors (1979).
The test consists of three parts assumed to measure three factors important for the
speechreading process: the ability to identify the information-carrying element of
speech (1-word), short-term memory, and information processing ability (3-words), and
the ability to extract the meaning of an utterance (Helen) (Ludvigsen, 1974). The first
two tests consisted of eight spondee words presented in a forced choice situation. In the
first speechreading test (1-word), one word at a time was presented after a carrier
phrase, "Nu sager jag:" /barn-vagn/ ("Now I am saying:" /spond6e word/). In the next
speechreading test (3-words), the same eight words were used with the same carrier
phrase, but three of the eight words were said after each other. The subject had to repeat
the words in correct order. In the last speechreading test (Helen), the subject had to
speechread a simple question, for example "What's the name of the capital of Sweden"?
The criteria that the sentence was understood was that the subject was able to give the
correct answer. The tests were recorded on videotape.
C. Audiovisual tests to measure the ability to combine auditory and visual information.
The audiovisual test in noise (Subj.SIN;audvis) was the same as the above described
auditory test in noise, in which the subjects adjusted the noise to the level where they
could just understand a continuous discourse. In the audiovisual situation, however,
they could also see the speaker's face and thus combine auditory and visual information. The subject sat in front of a TV-monitor (20"), l 112 meter from the monitor. The
story was randomly presented in the audiovisual (AV) or auditory (A) situation, so that
six subjects were first tested in the AV- and seven in the A-situation. To get acquainted
with the test procedure, there was a pretest situation exactly as AV or A, so that the
subject's total test was "A-AV-A or "AV-A-AV". It was only results from the second
and third situation that was used. Based on the difference in result in the A- and AV-situation, the visual gain was calculated (Subj.gain;vis) expressed in dB signal-to-noise
ratio.
D. A Questionnaire to get information about the hard of hearing person's situation judged by himself.
When measuring hearing impairments, the standard audiological procedures have
some limitations as these procedures fail to measure the hard of hearing person's own
opinion and difficulties in a variety of listening situations. Several different scales for
this type of measurement have been developed, for example, Hearing Handicap Scale
(High, Fairbanks, & Glorig, 1964), Social Hearing Handicap Index (Ewertsen & BirkNielsen, 1973), Hearing Performance Inventory (Giolas, Owens, Lamb, & Schubert,
1979) and Communication Profile for the Hearing Impaired (Demorest & Erdman,
1987). At the Hearing Institute, the Hearing Performance Inventory ( H P I ) is used. In
HPI, 90 situations are described, and the hearing impaired person is asked to value how
often he experiences difficulties - due to the hearing handicap - by answering if the situation results in difficulties "Practically Always", "Frequently", "About Half of the
Time", "Occasionally", and "Almost Never". The answers are then converted to percent,
were 100% stand for maximum difficulty. The inventory items are divided into six sections: Understanding Speech with visual information (HPI audvis) and without visual
information (HPI aud), Intensity, detect the presence of different environmental sounds
(HPI intensity), Response to Auditory Failure, how problems in communication are
solved, (HPI react), Social (HPI soc), Personal (HPI pers) and Occupational Situations
(HPI voc). The last three items describe the subjects view on how his hearing impairment iduences different aspects of his life situation.
Results and discussion
In Table I, range, mean value, and standard deviation are presented for tests with the
13 subjects in the study. Table I1 shows the correlation coefficients between the test results.
Mean
Range
Duration of loss, years
min
max
7
42
PTA, dB
33.4
93.3
Disc, %
12.0
100.0
HPI aud, %
38.3
95.0
Subj.S/N;aud, dB
+6.0
+31.5
BH S/N,dB
-4.1
+18.0
HPI gainpis %
-0.5
48.6
Subj.gain;vis, dB
0.0
+17.0
1-word, %
45.8
100.0
3-words, %
11.7
76.4
0.0
70.0
HPI audvis, %
33.3
82.4
Subj.S/N;audvis, dB
+1.5
+18.0
HPI intens, %
30.0
100.0
HPI react,%
35.4
83.1
HPI SOC, %
35.4
83.1
HPI pers, %
48.0
97.8
HPI voc, %
1.o
69.3
Helen, %
SD
Table I. Test results. Range, mean value and standard deviation. N=13.
The number of subjects in this study is low, and the range of the degree of loss is
wide, which might introduce measuring difficulties with some of the tests. The results
must then be interpreted with caution.
In the following, however, some interesting significant correlations (p<.05) are discussed. All tests in this study, which are supposed to measure visual ability, correlate
with Duration of loss. The longer a person has been hard of hearing, the more training
the person gets in using the visual information in speech as a support system to the distorted auditory information. A significant correlation between Duration of loss and
speechreading ability is thus what can be expected. Notably though, is that Jeffers &
Barley (1978) only show one study by Simmons where Duration of loss correlated with
AVl
Dcratlori o! loss
A1 PTA
A2 Disc
A3 HI'I aud
AV2
Ql
Q2
43
Q4
Q5
1.OO
0.22
0.62
0.24
-
V I HPI gaic;vis
VZ Subj.ga~n:vis
V3 I-word
V 4 3- words
V5 Hclc~l
Q1 HPI intcns.
a2 PI rcaci.
Q9 HPI soc.
Q4 tlPl pcrs.
Q5 PI voc.
Table II.
0.04
-0.21
-0.20
-0.20
- 0.23
Correlatiorzs henrveen Duration of loss (D.oL),results from auditory (A),Visual (V),audiovisual (AV)measurements,and HPI ( Q ) .
3
b
23
%
speechreading-ability. Of special interest in our study is that both the "objective"
speechreading-test ( 1-word 10.561, 3-words 10.621and Helen 10.704 and the "subjective"
tests (Subj.gain;vis 10.571 and HPI gainvis 10.600 show a significant correlation with
Duration of loss. Seven of the subjects in this study declared that the hearing loss was
acquired before the age of 10. Five of these belonged to the group of better speechreaders, which might indicate that the important factor is "age of onset" and not Duration of
loss. This will be studied in a coming article.
Two auditory tests, Disc 1-0.621and Subj.SIN;aud 10.731, show a significant comlation with Duration of loss. The correlation between Disc and Duration of loss is negative as could be expected. Subj.SIN;aud also shows a high correlation with Duration of
loss; that is the longer you have been hard of hearing the less noise do you tolerate in a
listening situation. No significant correlation was however found between BHSIN,
(speech perception in noise) and Duration of loss. The correlation between BHSIN and
Subj.SIN;aud is zero. This might indicate that the tests measures different aspects of the
speech perception difficulties. In a study made by Giolas and Epstein (1963), no significant correlation was found between results on word lists and continuous discourse. The
analysis made by Dancer, Davis, & O'Neil (1987) also indicates that "a word scoring
method does not necessarily reflect changes at the sentence or connected discourse levels."
HPI gain;vis 1-0.551,Subj.gain;vis 1-0.771and Helen 1-0.611correlate negatively with
Disc. Disc is an auditory test that measures a person's ability to understand PB-words in
quiet. The negative correlation between the visual tests and Disc is consistent with the
hypothesis that the more severe hearing handicap, the more you have to depend on
speechreading.
Disc 1-0.731 correlates with PTA, but the correlations between Disc and the two
"subjective" auditory tests: HPI aud 1-0.531and Subj.SIN;aud 1-0.551are not significant.
Disc is measured with phonetically balanced lists. The talker uses clear speech.
Picheny, Durlach, & Braida (1985) have presented evidence that there are intelligibility
differences between clear and conversonal speech. They reported that the difference
between clear and conversonal speech across talkers was 17 percentage points. The
questions in HPI are concerned with how the hearing impaired person perceives conversonal speech in different situations and in Subj.SIN;aud, the test material consists of
connected discourse spoken with conversonal speech. Another argument for using connected discourse spoken with conversonal language - instead of isolated words with
clear speech, when you want to develop a valid procedure for measuring the hard of
hearing person's difficulties outside the clinic - is that intelligibility scores for lists of
isolated words are poor predictors of scores for connected discourse (Giolas & Epstein,
1963). As the reality for the hard of hearing is to interpret connected discourse in most
situations, it is very important to develop a test which easily gives a measure of the hard
of hearing's perception of connected discourse. PTA and Disc colrelates highly, but
these tests show a low correlation with HPI-measures. These tests therefore do not seem
to reveal the most useful information about the hard of hearing person's difficulties in
perceiving speech in daily life.
The result from the test BHSIN 10.781correlates highly with PTA. HPI aud /0.60/,
HPI react 10.631and HPI soc 10.721 - all "subjective" answers to the HPI-questionnaire
- also give interesting correlations with BHSIN. Based on these preliminary results, the
BHSIN-test seems to give a good view of the hard of hearing person's problem in communication situations.
STL-QPSR 411988
Implication for the future
The new self-adjusted auditory and audiovisual test in noise did not show any significant correlation with the HPI-measurements. Our aim was to create a simple test
close to the hard of hearing person's every day listening situation. The subjects' spontaneous comments to the test indicated that they perceived the test as realistic. Based on
the results reported, it seems, however, that this test did not succeed to do this. One possible explanation might be that the manner in which the test was administered was not
optimal. Work on finding realistic measures for a communication profile is continuing.
If a valid and reliable test method could be developed for audiovisual speech perception
in noise, this would be very useful both in the rehabilitative work and for the hearing
aid fitting.
Acknowledgements
The authors would like to express their thanks to the staff and students at the Hearing
Institute in Uppsala for their participating in this study. This project has been supported
by a grant from the Swedish Council for Planning and Coordination of Research.
References:
Dancer, J.E., Davis, N.D., & O'Neil, J. (1987): "Word recognition versus sentence comprehension on a speechreading task," A.A.D, March, pp. 43-45.
Demorest, M.E. & Erdman, S. (1987): "Development of the Communication Profile for the
Hearing Impaired," JSHD 52, pp. 129-143.
Egan, J.P. (1948): "Articulation testing methods," The Laryngoscope 58:9, pp. 955-991.
Ewertsen, H.W. & Birk Nielsen, H. (1973): "Social hearing handicap index," Audiology 12,
pp. 180-187.
Giolas, T.G. & Epstein, A. (1963): "Comparative intelligibility of word lists and continuous discourse," JSHR 6, pp. 349-358.
Giolas, T.G., Owens, E., Lamb, S.H., & Schubert, E.D. (1979): "Hearing performance inventory," JSHD 44:2, pp. 169-195.
Hagerman, B. (1984): "Clinical measurements of speech reception threshold in noise," Scand.
Audiology 13,pp. 57-63.
Hawkins, D.B., Montgomery, A.A., Meuller, H.G., & Sedge, R.K. (1988): "Assessment of
speech intelligibility by hearing-impaired listeners," pp. 241-246 in Noise a s a Public Health
Problem. Vol. 2, Swedish Council for Building Research, Stockholm.
High, W.S., Fairbanks, G., & Glorig, A. (1964): "Scale for self-assessment of hearing handicap," JSHD 29:3, pp. 215-230.
Jeffers,J. & Barley, M. (1978): Speechreading, Charles Thomas Publisher, Springfield. USA
Ludvigsen, C. (1974): "Construction and evaluation of an audio-visual test (The Helen lest),"
pp. 67-75 in (H. Birk Nielsen & E. Karnpp, eds.) T/isual and Arrdio-vis~nl/7cr.c.cl,fiort of
speech, Sixth Danavox Symposium, Scandinavi'm Audiology Suppl. 4.
Middelweerd, M.J. & Plomp, R. (1987): "The effect of speechreading on the speech-reception
threshold of sentences in noise," J.Acoust.Soc.Am. 82 :6, pp. 2145-214.
O'Neill, J.J. (1957): "Recognition of intelligibility test materials in context and isolation,"
JSHD 22, pp. 87-90.
Picheny, M.A., Durlach, N.I., & Braida, L.D. (1985): "Speaking clearly for the hard of hearing
I: Intelligibility differences between clear and conversonal speech", JSHR 28, pp. 96-103.
Plomp, R. (1978): "Auditory handicap of hearing impairment and the limited benefit of hearing
aids,"J.Acoust.Soc.Am. 63: 2 , pp. 533-549.
Risberg, A. & Agelfors, E. (1978): "Information extraction and information processing in
speech-reading,"STL-QPSR 2-3/1978, pp. 62-81.
World Health Organization (1980): International classification of impairments, disabilities and
handicaps, p. 28 (WHO+, Geneva).