A Novel Noise-induced Annoyance Measurement
Method
Huan Zhou, Rongshan Yu, Ying Song
Signal Processing Department
Institute for Infocomm Research, Singapore
Abstract—This paper proposes a novel method of measuring
noise-induced annoyance used by jury subjective tests. Currently,
noise-induced annoyance is typically measured by either rating
scale or discrimination scale. The former is able to provide
quantitative results, but it is very challenging to achieve the
desired accuracy. The latter in general could provide more
accurate results, but it is not able to provide quantitative
results that are useful in many applications. To overcome these
limitations, we propose a new listening test method. The proposed
method introduces a reference stimulus with adjustable sound
level as a benchmark for noise annoyance assessment, and the
final annoyance index is reported based on the level of the
reference stimulus that matches annoyance level of the sample
under test. Our experiments show that the proposed method
is more reliable and useful than existing methods for noise
annoyance measurement.
I. I NTRODUCTION
The environmental noise pollution is a growing concern
worldwide as a cost of rapid economic growth and development. Among the health effects due to environmental noise,
one of the most salient and direct effects of noise on humans is
annoyance [7]. Such noise-induced annoyance generally refers
to a person’s individual adverse reaction (including dissatisfaction, bother, annoyance and disturbance due to noise) to
noise [3], which might be caused by a complex mix of
noise exposure quantities, non-acoustical factors and potential
psycho-acoustical factors. Thus, it is challenging to measure
noise annoyance, although it is a common and universal
sensation for most people.
Depending on various research objectives, two different
approaches are commonly used to collect noise-induced annoyance information. One approach is via social surveys,
where questionnaires or interviews are conducted in situ by
asking people who have been exposed to the noise environment
for a long time to respond their experienced annoyance. The
degree of noise annoyance is usually quantized using two
indices, a 5-point verbal rating scale (not at all / slightly
/ moderately / very / extremely) and a 11-point numerical
rating scale (0-to-10 with even distribution). If noise exposure information (like day-night average sound level 𝐿𝑑𝑛 )
is available, a quantitative model that relates the overall
noise annoyance to noise exposure can be developed, such
as the well-known dose-response curve in [9]. And if nonacoustical factors (like noise sensitivity, social-demographic
factors, etc.) are available, a qualitative model that accounts
for how physiological, psychological and social factors affect
noise perception can be built, such as the sensory pleasantness
model in [19].
The other approach to collect annoyance information is via
jury subjective tests where subjects report their perceived annoyances of short-term noise exposures under a controlled laboratory environment following a pre-defined annoyance level
metrics. By associating such information with the psychoacoustical features (like loudness, roughness and sharpness
etc.) of noise stimuli, the so-called psycho-acoustic annoyance (PA) model can be developed and used for annoyance
prediction (e.g. [11], [17]).
Regarding the annoyance measurement methods, the above
mentioned 5-point verbal rating scale and the 11-point numerical rating scale are mainstream methods used in both
social surveys and jury tests. We noted that comparing to the
numerous researches on investigating annoyance influencing
factors, there are much less studies on the topic of annoyance
measurement. In this research, we are going to investigate this
topic. In particular, we are interested in how to evaluate shortterm noise annoyance that used in jury subjective tests.
The rest of this paper is organized as follows. Firstly,
an overview of existing methods on annoyance measurement
is given in Section 2. Then a new annoyance measurement
method is proposed in Section 3 to overcome the limitations
of existing methods. In Section 4, results of a series of jury
tests are reported to verify the effectiveness of the proposed
method. At last, Section 5 concludes the paper.
II. E XISTING M ETHODS OF N OISE - INDUCED A NNOYANCE
M EASUREMENT M ETHOD
Although there is considerable diversity of opinions on how
subjective annoyance should be measured, existing methods
can be typically categorized into either rating scale (RS) based
method, or discrimination scale (DS) based method.
The RS method requires the subjects, after exposure of a
short noise stimulus, represent the degrees of their short-term
annoyances using either verbal rating scale or numerical rating
scale. Examples of those different scales are:
∙
∙
Semantic scale [2]: annoyance indexes are identified with
five labels - ”not at all”, ”slightly”, ”moderately”, ”very”
and ”extremely”, which are equidistant from each other.
Numerical scale [3]: A popular numerical scale example
is an 11-point categorical scale from 0 to 10, as illustrated
in Fig. 1, which has been recommended by ISO/ITS as
a standard of ”assessment of noise annoyance by means
of social and socio-acoustic surveys”.
Not
at all
0
Extremely
1
2
3
4
5
6
7
8
9
10
Fig. 1. The eleven-point numerical scale, as recommended by ISO/ITS 15666:
2003.
The most advantage of the RS method is that it provides
absolute annoyance scale. This makes it as a convenient index
for annoyance analysis, comparison and modeling.
However, as pointed out by other researchers (e.g. [8]),
since the noise perception is subjective, there lacks a systematic mechanism for the listener to associate either verbal rating
scale or numerical rating scale with their perceived level of
annoyance of the listening test samples, which possibly generates confusion between ratings. As a result, it is challenging
to obtain statistically significant results from this type of test.
On the other hand, the DS method is generally based on
paired comparison, which requires the subjects, after exposure
to a pair of noise stimuli, determine which of the two noises
is more annoying. Examples include:
∙
∙
Pair comparison method [8]: after the presentation of
each sound pair, the participants judged which sound
stimulus was more annoying or if they were equal. Also,
they were asked to give an annoyance rating based on
how much more annoying one stimulus was over the other
one in two methods, with scores distributed as 2-0, 0-2
or 1-1 for a pair.
Relative magnitude estimation method [1]: the participants rated annoyance index of each stimulus numerically
relative to a pre-defined reference sound which was
assigned an arbitrary rating of 100 (so that each reported
annoyance index was referred to 100).
One of the benefits of DS method is that it adopts a forcedchoice procedure, which generally produces less ambiguous
results for untrained subjects compared to the RS method.
In fact, similar psychological evidence [22] shows that
human prefers to evaluate images by comparing candidate with
benchmark rather than assess with numerical scores. This is
preferred considering normally only non-experts are present in
the jury-panels. But the downside of this method is that quantitative results on absolute annoyance indexes are unavailable,
which makes it less useful for further noise/annoyance analysis. Another downside of this method is the time consumption
issue. Because all possible pairs have to be presented to the
jury subjects, 𝑛 ∗ (𝑛 − 1)/2 pair comparisons are required for
the tests with a set of 𝑛 sounds, which may most likely exhaust
the subject before completing the test.
Among the above annoyance measurement methods, the
ISO standardized 11-point numerical annoyance scale [3] is
the most widely adopted one in this research field. Therefore,
it is chosen as a representative of prior arts and will be tested
in our experiments as a benchmark method.
III. P ROPOSED A NNOYANCE M EASUREMENT M ETHOD
From previous section, it can be seen that the all existing
noise annoyance measurement methods have respective limitations. To overcome those limitations, a novel noise annoyance
measurement method is proposed in this paper.
The new method introduces a reference stimulus with
adjustable sound level (in dB) as a benchmark for noise
annoyance assessment. Like previous jury subjective tests
using DS, the subject firstly experiences short-term noise
exposure from one sound pair, the reference and a target noise
stimulus. Then, instead of directly reporting a score of the
comparative annoyance difference, in our method, the subject
is required to adjust sound level of the reference to reduce
the annoyance difference between the reference and the target
noise until the final sound pair brings the same or similar
sensation to the subject, or the adjusted sound level reaches its
range. Lastly, the final bipolar value of sound level adjustment
on the reference is recorded as an annoyance index to reflect
the annoyance degree of the target noise stimulus.
It’s clear that the proposed method potentially combines the
merits of both two types of existing methods. In particular, it
embeds the following advantages: 1) less ambiguous results
(because it adopts a forced-choice procedure); 2) less timeconsuming (on average, 2.4 ∗ 𝑛 pair comparisons for the test
with 𝑛 sounds)1 and 3) quantitative annoyance index (absolute
value of sound level adjustment). Last but not least, since all
noise stimuli are compared to the same reference stimulus,
it can be expected that the proposed method can provide
more reliable annoyance index than existing RS or DS based
methods.
IV. E XPERIMENT S ETUP AND R ESULTS
To verify the effectiveness of the proposed method, jury
subjective tests with 15 subjects are conducted. Both the
proposed method and the 11-point numerical scale based ISO
15666 method [3], are used in the tests. Herein, like many
previous studies [20] [4], the 11 numerical scale is further
conveniently translated into a scale, from 0 to 100, based on
the assumption that a set of annoyance categories divides the
range into equally spaced intervals.
A. Experiment Setup
The jury tests are conducted in an anechoic chamber. A total
of 15 subjects (8 males and 7 females), with normal hearing
and aged between 31 ∼ 55 years old, are selected for the tests.
The experiments are approved by the local ethic committee,
Institutional Review Board, National University of Singapore;
and all jury subjects give informed and written consent.
20 noise stimuli, with duration of 10 seconds each, are chosen as a miniature set of typical urban noises, including traffic,
airplane, community and construction noises, as illustrated in
Table I.
The selection of reference signal is under investigation
when the listening tests were conducted. As a makeshift, a
1 statistical conclusions drawn from the feedbacks of our following jury
experiment
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Noise name
highway traffic noise
electric train noise
toilet flush noise
heavy traffic noise
commercial jet noise
train passing noise
cars passing by noise
grinding floor noise
plane flying noise
demolition noise
school playground noise
pneumatic drill noise
neighbor drilling noise
whining airplane engine noise
solid steel hammering noise
semi passing noise
wet traffic noise
knocking door noise
remote control airplane noise
vacuum cleaning noise
Noise type
traffic
traffic
community
traffic
airplane
traffic
traffic
construction
airplane
construction
community
construction
construction
airplane
construction
traffic
traffic
community
airplane
community
C. Experiment Result Analysis
The experiment results are evaluated based on both statistical means and confidence intervals (CIs) at 95% confidence
interval. Box-and-Whisker technique is used for screening, to
remove invalid data (i.e., outliers) in raw results. The valid
screening results show that the mean distribution of all noise
stimuli is similar by both methods. However, observing the CIs
of each noise stimulus, it can be found that CIs of proposed
method (see Figure 2) are distinctively shorter than the CIs of
prior art (see Figure 3). In detail, the averaged CI is 8.30dB
over range of 50 in proposed method, and 31.14 points over
range of 100 in prior art. This verifies our expectation that the
proposed method can provide less ambiguous results, that is,
more consistent annoyance index with less variation.
Annoyance Scales using the Proposed Method
30
25
TABLE I
S TIMULI L IST
B. Experiment Procedure
In each test round, subject is asked to imagine himself at
home while relaxing (reading, watching television or doing
other common relaxing activities). After the instruction explanation, the 20 noise stimuli then randomly played one by one.
After hearing one noise stimulus, the subject is required to:
1) rate the annoyance of the noise stimulus with a grade of
100-point numerical scale.
2) subsequently, play the reference stimulus. Then based
on his own perception, fine tune the sound level bar of
the reference, until the annoyance differences between the
adjusted reference and the noise is imperceptible or the range
of level bar is reached.
Both the above rating score and final value of sound level
adjustment (in logarithmic scale, dB) are recorded as the
annoyance indexes of the noise stimulus under test.
Following the procedure, the overall time consumption for
the 20 noise test turns out to be less than 30 minutes.
Adjusted Scale
15
10
5
0
−5
−10
−15
−20
0
5
10
15
Noise Item No.
20
25
Fig. 2. Annoyance levels using the proposed method.
Annyoance Scores using Prior Arts
100
90
80
Annoyance Score
construction drilling noise (also with duration of 10 seconds),
with 50 dBA, is selected as the reference stimulus. Its sound
level can be continuously adjusted (in dB) from 20 dB less to
30 dB more, with default setting as 0 dB at the beginning of
the experiment.
A Matlab based Graphical-User-Interface (GUI) is designed
to facilitate the experiments. It adopts MUSHRA [21] style
so that the subject can access multiple noise stimuli and
annoyance indexes on one integrated screen. This allows
subjects to modify their decisions during the evaluation when
it is necessary.
A loudspeaker (Genelec 1032A) is placed in front of the test
subject at the distance around 4.5 meter. All noise stimuli and
reference signal are monaurally presented via the loudspeaker.
20
70
60
50
40
30
20
0
5
10
15
Noise Item No.
20
25
Fig. 3. Annoyance levels using prior art.
One direct implication of this advantage is that it becomes
easier to identify whether one stimulus is more annoying
than another even based on small number of test results. To
illustrate this merit, we further analyze the results based on
different noise types. Due to page limitation, only comparison
results on traffic noises are illustrated in Figure 4.
As an example, we analyze the annoyance indexes of traffic
noises. Firstly, it is interesting to note that the noise rankings
are consistent by both methods. Secondly, using prior art (right
Traffic Noise
1
0.9
0.9
0.8
0.8
Normalized Annoyance Score
Normalized Adjusted Scale
Traffic Noise
1
0.7
0.6
0.5
0.4
0.3
0.7
0.6
0.5
0.4
0.3
0.2
0.2
0.1
0.1
0
1
2 4 6 7 16 17
Noise Item No.
0
1
2 4 6 7 16 17
Noise Item No.
Fig. 4. Annoyance levels of traffic noises.
Noise type
Traffic
Airplane
Community
Construction
Reference
noise
#1
#1
#3
#1
High annoyance noise
Prior art Proposed method
#3, #7
#2 ,#3, #5, #6, #7
#4
#2, #3, #4
#2, #4
#2, #4
#3, #4
#2, #3, #4, #5
TABLE II
A NNOYANCE C OMPARISONS
panel), it can be observed that noise #3 and #7 are significantly
more annoying than noise #1 (with separated CIs). While using
the proposed method (left panel), more noises (noise #2 , #3,
#5, #6 and #7) demonstrates higher annoyance than noise #1.
Similar founding can be observed from other noise types as
well, as concluded in Table II.
V. C ONCLUSIONS
From the above observations, it could be concluded that
the proposed listening test method can produce more reliable
and less ambiguous annoyance test results compared to the
widely adopted ISO 15666 method, which makes it a more
preferable method for noise annoyance measurement tasks. To
further verify it, we plan to evaluate the proposed method in
a more large-scale experiment with hundreds of noise stimuli
and experiment subjects. The test results will be reported upon
the completion of the experiments.
ACKNOWLEDGEMENT
This material is based on research/work supported by the
Singapore Ministry of National Development and National
Research Foundation under L2 NIC Award No. L2NICCFP12013-7.
R EFERENCES
[1] Antonio J. Torija, Ian H. Flindell, Rod Self, frequency weightings based
on subjectively dominant spectral ranges, EuroNoise (2015), 1645–1649.
[2] J.M. Fields, R.G. De Jong, T. Gjestland etc. , Standardized noise reaction
questions for community noise surveys: research and a recommendation,
Journal of Sound & Vibration, 242(2001), 641–679.
[3] ISO/TS 15666: Acoustics - Assessment of noise annoyance by means of
social and socio-acoustic surveys, (2003).
[4] Daniel L. Steele, Song Hui Chon, A perceptual study of sound annoyance,
AudioMostly, (2007).
[5] M. Birgitta Berglund, Ratio Scaling of Psychological Magnitude - In
Honor of the Memory of S. S. Stevens, Hillsdale New Jersey, 1991.
[6] Bert De Coensel etc., Experimental investigation of noise annoyance cuased by high-speed trains, ACTA Acustica united with Acustica,Vol.93(2007), 589–601.
[7] Fritschi L. etc., Burden of Disease from Enviromental Noise: Quantification of Healthy Life Years Lost in Europe, World Health Organization,
Regitional Office for Europe: Copenhagen, Denmark, 2011.
[8] M. Shafiquzzaman Khan, etc., Subjective Annoyance Response to Diesel
Engine Sound During Idling COnditions, Int. Journal of Occupational
Safety and Ergonomics, Vol.2(1996), 16–26.
[9] T.J. Schultz, Synthesis of social surveys on noise annoyance, J. Acoust
Soc Am., Vol.64(1978), 377–405.
[10] Miedema HM, Vos H., Exposure-response relationships for transportation noise, J Acoust Soc Am., 1998 Dec, 3432–3445.
[11] E. Zwicker and H. Fastl, Psychoacoustics: Facts and Models, Springer
(2007).
[12] Miedema, H.M.E. and Vos, H.,Demographic and attitudianl factors that
modify annoyance from transportation noise, Journal of Acoustical Society
of America, Vol.105(1999), 3336-3344.
[13] Maarten, K., Eric, J.E., Bert, V.W.,Testing a theory of aircraft noise
annoyance: A structural equation analysis, Journal of Acoustical Society
of America, Vol.123(2008), 4250-4260.
[14] Ouis, D., Annoyance from road traffic noise: a review, Journal of
Enviromental Psychology, Vol.21(2001), 101-120.
[15] I. Alimohammadi, P. Nassiri, M. Azkhosh and M. Hoseini, Factors
affecting road traffic noise annoyance among white-collar employees
working in Tehran, Journal of Enviromental Health. Sci. Eng., Vol.7(2010),
25-34.
[16] Weinstein, N. D., Individual differences in reactions to noise: A longitudinal study in a college dormitory, Journal of Applied Psychology,
Vol.63(1978), 458–466.
[17] Marquis, F.C., Premat, E., Aubree, D. and Vallet, M.,Noise and its effects
- a review on qualitative aspects of sound. Part II: Noise and Annoyance,
Acta Acustica united with Acustica, Vol.91(2005), 626-642.
[18] Hans Jurgen Eysenck and Sybil B. G. Eysenck,Manual of Eysenck
Personality Questionnaire, London: Hodder and Stoughton.
[19] Taylor S.M.,A path model of aircraft noise annoyance, Journal of Sound
and Vibration, Vol.96(1984), 243-260.
[20] Kryter KD.,Acoustical, sensory, and psychological research data and
procedures for their use in predicting effects of environmental noises,
Journal of Acoustical Society of America, Vol.122(2007), 2601-2614.
[21] ITU-R BS.1534-1,Method for the subjective assessment of intermediate
quality level of coding systems, International Telecommunication Union Radiocommunication Sector, 2001.
[22] M H. Waugh.,Personal narratives from psychological assessment, Journal of Constructivist Psychology, 2015.