AV 1461/01 Page 1 of 7 + Appendix REPORT DELTA Danish Electronics, Light & Acoustics Building 356 Akademivej DK-2800 Kgs. Lyngby Denmark Tel. (+45) 45 93 12 11 Measurements and Judgments of Sound in relation to Human Sound Perception Client: Nordtest Fax (+45) 45 93 19 90 www.delta.dk The report must not be reproduced, except in full, 23 August 2001 without the written approval of DELTA. 80rap-uk-a AV 1461/01 Page 2 of 7 Title Measurements and Judgments of Sound in relation to Human Sound Perception Journal no. AV 1461/01 Project no. 1519-00 Our ref. THP/CLF/lm Client NORDTEST Postbox 116 FIN-02151 Esbo Finland Summary It is recognised that A-weighted measurements of sound pressure levels and sound power levels alone are insufficient for measurements in relation to product sound quality and noise annoyance potential. This has led to an increasing interest in objective and subjective methods giving results in closer agreement with human sound perception. As a result of the work in this project a guideline consisting of two parts has been worked out. Part 1 gives instructions on how to supplement the A-weighted data from standardised measurements of sound pressure and sound power with meaningful results of measurements of loudness. Part 2 describes principles for making effective and reliable listening tests. For some topics the guideline gives specific recommendations, for other topics the text has the character of a list of items to remember or to consider. A number of definitions within the fields of product sound quality and listening tests are given. DELTA, 23 August 2001 Torben Holm Pedersen Acoustics & Vibration AV 1461/01 Page 3 of 7 Contents 1. Background.............................................................................................................. 4 2. Aim ........................................................................................................................... 5 3. Project Group .......................................................................................................... 6 4. Work in the Project Group .................................................................................... 7 5. The Result of the Project ........................................................................................ 7 Appendix: Nordtest Guideline Draft Measurements and Judgments of Sound in relation to Human Sound Perception Common Introducion (AV 1458/01) Part 1: Measurement Positions for Measurement of Loudness (AV 1457/01) Part 2: Guidelines for Listening Tests (AV 1456/01) AV 1461/01 Page 4 of 7 1. Background There is an increasing recognition that in a number of contexts A-weighted noise levels are insufficient for assessment of noise problems and sound from products. This has led to an increasing interest in measurement and evaluation methods which agree with human sound perception to a higher degree than earlier. Loudness is an obvious example of an objective goal which is closer to the sound perception than the A-weighted levels normally used. dB Pink noise Bandpassfiltered noise Tone Air hammer Organ harmony Electronic warning signal Electric high speed tool Figure 1 Illustration of the relevance of supplementing A-weighted measurements with measurements of loudness. Upper curve: A-weighted sound pressure levels with time weighting F (uncalibrated) Lower curve: Loudness versus time of the same sounds AV 1461/01 Page 5 of 7 Figure 1 illustrates the relevance of supplementing the A-weighted measurements with measurements of loudness. It is seen that for different noises with the same A-weighted level (upper curve) significant differences in the loudness (and the perception of the intensity) are found. Several instruments and analysers are now available which can measure loudness in a homogeneous and simple way, so the time is ripe for supplementing the (standardised) measuring methods with the possibility of supplementarily and comparably stating e.g. values of loudness. It might also be relevant with other psychoacoustically related measures as e.g. sharpness, roughness, etc. Measuring positions etc. are not defined in present standards with this end in view. This applies especially to standards for measurement of sound power. Therefore there is a need that standards etc. are supplemented with directions determining relevant measuring positions etc. for this purpose. Correspondingly the interest in using listening tests both as objective and subjective measuring tools is increasing. Unfortunately there are no commonly accepted guidelines in this field, so the results are seldom comparable. It has not been possible at present to determine detailed guidelines for performing listening tests. But within the scope of this project guidelines are given as to what should be taken into consideration and reported when carrying out listening tests. It is meant as a valuable contribution in the process of making listening tests an operational measuring tool. Many are of the opinion that the use of A-weighted levels in time should be replaced by other and more relevant measures, but as the major part of our empirical and legislative basis is founded on A-weighted values, changes have been avoided so far and for very good reasons. It is our hope that the Nordic countries could be the initiators of an appropriate development by creating the possibility of – in a homogeneous way– being able to supplement the traditional measures with measures, which conform better to the knowledge we now have of human sound perception. 2. Aim The aim of this project was to give a supplement to existing standards for measurement of sound power and sound pressure levels in the shape of: 1. Instructions/directions for measurement of loudness and other psychoacoustic parameters. 2. Guidelines for topics, which should be considered when making a listening test. The work is delimited to relevant ISO and NORDTEST standards and perhaps a few IEC standards. AV 1461/01 Page 6 of 7 3. Project Group The project group consisted of: Torben Holm Pedersen, project leader DELTA Acoustics & Vibration Akademivej, Building 356 DK-2800 Kgs. Lyngby Denmark [email protected] Torben Holm Pedersen prepared Part 2 ”Guidelines for listening tests” of the method. Truls Gjestland SINTEF Telecom and Informatics N-7465 Trondheim Norway [email protected] Mendel Kleiner Chalmers University of Technology Room acoustics group S-412 96 Göteborg Sweden [email protected] Panu Maijala VTT Automation Safety Engineering Tekniikankatu 1 P.O.Box 1307 FIN-33101 TAMPERE Finland [email protected] The group was supplemented with: Carsten L. Fog DELTA Acoustics & Vibration Akademivej, Building 356 DK-2800 Kgs. Lyngby [email protected] Carsten L. Fog prepared Part 1 “Measurement Positions for Measurement of Loudness“ of the method. AV 1461/01 Page 7 of 7 4. Work in the Project Group The project group had two meetings at DELTA in Lyngby. Before the first meeting drafts of the two parts of the methods were sent to the participants. Principles and methods were discussed. On the basis of the discussions a revised draft was sent to the participants for further comments. These comments were taken into account, and in general the drafts were brought to a more complete stage before the second and final meeting. At the second meeting the drafts of the method were discussed, and after the meeting the report and final drafts were sent to the participants for approval. 5. The Result of the Project The result of the work is a draft for a NORDTEST guideline consisting of two parts (Part 1: AV 1457/01 and Part 2: AV 1456/01) with a common introduction (AV 1458/ 01). The guideline is enclosed. Nordtest Guideline Draft 2001-08-23 AV 1458/01 Measurements and judgments of sound in relation to human sound perception Part 1: Measurement positions for measurement of loudness Part 2: Guidelines for listening tests Introduction This NORDTEST guideline consists of two independent parts: Part 1: Measurement positions for measurement of loudness It is recognised that A-weighted measurements of sound pressure levels and sound power levels alone are insufficient for measurements in relation to product sound quality and noise annoyance potential. This has led to increasing interest in measures in closer agreement with human sound perception as e.g. loudness, sharpness roughness ..… Equipment, which gives easy and reliable measurements of loudness and other psychoacoustic parameters, is now available, and therefore there is a need to supplement existing measurement methods with procedures for measurement of loudness etc. In the existing ISO-standards for measurements of A-weighted sound pressure levels and sound power levels, measurement positions are not defined with the purpose of giving meaningful measurements of e.g. loudness. Part 1 of this NORDTEST guideline gives instructions on how to supplement the A-weighted data with meaningful results of measurements of loudness. Part 2: Guidelines for listening tests The interest in using listening tests as both an objective and a subjective measuring tool is increasing. There are no generally accepted guidelines for making listening tests so the results are seldom comparable, and there is a risk that ”newcomers” in this field make fundamental methodological errors. Part 2 of this NORDTEST guideline is meant as a help to make effective and reliable listening tests. For some topics the guideline gives specific recommendations, for other topics the text has the character of a list of items to remember or to consider. As new concepts within acoustics are introduced, the section with definitions and terms is rather comprehensive. (page 1) Nordtest Guideline Draft 8th working draft 2001-08-23 AV 1457/01 Part 1 prepared by Carsten L. Fog Measurements and judgments of noise in relation to human sound perception Part 1: Measurement positions for measurement of loudness Contents 1. 2. 3. 4. Scope ................................................................................................................................................3 Field of application ............................................................................................................................3 Normative references........................................................................................................................3 Definitions..........................................................................................................................................4 4.1 Sound pressure level, in decibels, Lp .........................................................................................4 4.2 Energy-equivalent sound pressure level, in decibels, Leq,T ........................................................4 4.3 Sound intensity level, in decibels, LI ..........................................................................................4 4.4 Sound power level, Lw ................................................................................................................4 4.5 Loudness....................................................................................................................................5 4.5.1 Loudness ............................................................................................................................5 4.5.2 Loudness level....................................................................................................................5 4.6 Other metrics related to human hearing ....................................................................................5 5. Related EN and ISO standards.........................................................................................................5 5.1 General overview .......................................................................................................................5 5.1.1 Noise declaration measurements .......................................................................................6 5.2 Determination of (stationary) loudness ......................................................................................6 5.3 Determination of sound pressure level ......................................................................................6 5.4 Determination of sound power levels.........................................................................................7 5.4.1 Determination of sound power level based on sound pressure .........................................7 5.4.2 Determination of sound power level based on sound intensity ..........................................7 6. Measurement guidelines ...................................................................................................................7 6.1 Types of sound sources.............................................................................................................7 6.2 Instrumentation ..........................................................................................................................7 6.3 Test environment .......................................................................................................................8 6.4 Specification of installation and operating conditions ................................................................8 6.5 Specification of measurement positions ....................................................................................9 6.5.1 At the operator’s position ....................................................................................................9 6.5.2 Enclosed work station position ...........................................................................................9 6.5.3 At the bystander position ....................................................................................................9 6.5.4 Measurement positions in other cases ...............................................................................9 6.6 Measurements ...........................................................................................................................9 6.6.1 Measurement time interval .................................................................................................9 6.6.2 Steady noise .................................................................................................................... 10 6.6.3 Non-steady noise............................................................................................................. 10 6.6.4 Measurements in frequency bands ................................................................................. 10 6.6.5 Repetition of measurements............................................................................................ 10 6.6.6 Impulsive sound............................................................................................................... 10 6.6.7 Supplementary measurements........................................................................................ 10 6.7 Accuracy ................................................................................................................................. 11 7. Information to be recorded and reported ....................................................................................... 11 7.1 Information to be recorded...................................................................................................... 11 (page 2) 7.1.1 Product under test ........................................................................................................... 11 7.1.2 Test conditions................................................................................................................. 11 7.1.3 Acoustic environment ...................................................................................................... 11 7.1.4 Instrumentation ................................................................................................................ 11 7.1.5 Location of specified positions......................................................................................... 12 7.1.6 Noise data........................................................................................................................ 12 7.2 Information to be reported....................................................................................................... 12 8. Bibliography.................................................................................................................................... 12 (Part 1, page 3) Part 1: Measurement positions for measurement of loudness 1. Scope The traditional physical descriptor, the A-weighted sound pressure level, LPA, is often not an adequate objective descriptor for perceived sound. The sounds of different products with the same LPA do not always sound equally intense to the user. So, objective descriptors are needed, which correlate better with subjective perception, as a supplement to the A-weighted based parameters sound pressure and sound power. The scope of this Nordtest Guideline is to supplement existing standards for measurements of sound pressure levels and sound power levels with guidelines for measurements (and calculations) of loudness and other psychoacoustic descriptors for product sound quality. 2. Field of application This guideline is related to the Machinery Directive (89/392/EU) and the new “Outdoor” Directive (2000/14/EU) and their requirements on low-noise design as an integral part of machinery safety and environmental protection - “Machinery must be so designed that risks resulting from noise are reduced to the lowest level taking account of technical progress and the availability of means of reducing noise, in particular at the source”. This guideline is a supplement to existing noise source measurement guidelines described especially in the EN ISO 3740 to 3747 and the EN ISO 11200 – 11204 standards. Specific guidelines on how to perform such measurements in the field are found also in product-relevant standards (Noise Test Codes) for the specific product. 3. Normative references The following normative documents contain provisions, which constitute - through reference in this text - provisions of this Nordtest Guideline. Parties using this Nordtest Guideline are encouraged to investigate the possibility of applying the most recent editions of the normative documents indicated below. For undated references, the latest edition of the normative document referred to apply. Nordtest as well as members of ISO and IEC maintain registers of currently valid International Standards. 1. EN ISO 4871, “Acoustics - Declaration and verification of noise emission values of machinery and equipment” 2. EN ISO 3740-3747, “Acoustics - Determination of sound power levels of noise sources – guidelines and using” 3. EN ISO 9614, “Acoustics - Determination of sound power levels using intensity” 4. EN ISO 11200-11204, “Acoustics – Noise emitted by machinery and equipment – Guidelines for the use of basic standards for the determination of emission sound pressure levels at a work station and at other specified positions” 5. EN ISO 12001, “Acoustics - Noise emitted by machinery and equipment – Rules for the drafting and presentation of a noise test code” 6. Noise test codes 7. IEC Publication 61672, “Electroacoustics - Sound level meters”1) 8. IEC Publications 60651 and 60804, “Sound level meters” 9. IEC Publication 60942, “Acoustic calibrators” 10. IEC Publication 61260, “1/3 and 1/1 octave filters” 11. ISO 532-1975 (E), “Acoustics – Method for calculating loudness level, Section two” 12. ISO 226, 1987 (E), “Acoustics – Normal equal-loudness level contours” 1) To be published, replaces IEC 60651 and IEC 60804. (Part 1, page 4) 4. Definitions 4.1 Sound pressure level, in decibels, Lp The sound pressure level Lp is given by p Lp = 10 lg p0 2 dB p = Root mean square sound pressure, in pascals p0 = Reference sound pressure, 20 µPa 4.2 Energy-equivalent sound pressure level, in decibels, Leq,T2) The value of the A-weighted sound pressure level of a continuous steady sound that within a specified time period has the same mean square sound pressure as a sound whose level varies with time. It is defined as t L eq,T = 10 lg 1 2 p2 ( t ) dt t 2 − t1 ∫t p 02 dB 1 Leq,T = Energy-equivalent sound pressure level for the time interval T, starting at the time t1 and ending at the time t2, in decibels Reference sound pressure, 20 µPa Instantaneous value of the sound pressure of the noise signal, in pascals p0 p(t) = = 4.3 Sound intensity level, in decibels, LI The sound intensity is defined as I I0 LI = 10 log dB I0 = Reference sound intensity, 1 pW/m2 I = Sound intensity, in watt/m2 4.4 Sound power level, Lw The sound power is defined as w w0 Lw = 10 log dB w = Sound power, in watt w0 = Reference sound power, 1 pW LWA is the A-weighted sound power level. 2) The expression “equivalent sound level” may be used in order to simplify a text. If A-weighting is used, the term is L Aeq,T (Part 1, page 5) 4.5 Loudness 4.5.1 Loudness Loudness belongs to the category of intensity sensations for sound. The unit for loudness N is Sone. A 1 kHz tone with a sound pressure level of 40 dB re 20 µPa is defined as having a loudness of 1 sone. Signals perceived as being twice as loud have a loudness of 2 sone. In the mid-frequency range and above sound pressure levels of 40 dB, an increase of the level of 10 dB leads to a multiplication of loudness by a factor of two, see Normative references 11 of Part 2. 4.5.2 Loudness level The loudness level LN of a given sound, measured in phon, is the sound pressure level of a reference sound, consisting of a sinusoidal plane progressive wave of frequency 1000 Hz coming directly in front of the listener, which is judged by otologically normal persons to be equally loud as the given sound, see Normative references 11 and 12 of Part 2. 4.6 Other metrics related to human hearing Many specialised (product) sound quality metrics and combinations of metrics have been developed such as Sharpness, Fluctuation Strength, and Roughness. These are further described in the Appendix of Part 2 of this Nordtest Guideline. 5. Related EN and ISO standards The requirements of the Machinery Directive that the noise level shall be so low as can reasonably be expected have led to European and international standardisation organisations preparing several guidelines regarding noise attenuation. Manufacturers must now in many cases make available specified quantitative information on noise emitted under defined operation, e.g. noise declarations. Furthermore, in this way manufacturers are made active participants in low-noise design and noise and vibration control. 5.1 General overview In order to help manufacturers make the required specified quantitative information, the CEN (CENELEC) has prepared general harmonized standards and guidelines. These comprise the following issues: • Noise declaration measurement guidelines - Sound pressure level at operator's position • Sound power level for the product/equipment • Guideline: Preparation of Noise Test Codes • “Declaration” of noise-attenuating equipment. Measurement guidelines for: - Silencers - Enclosures - Screens • Guideline - Design of low-noise work stations • Guideline - Design of low-noise machinery • Guideline - Systematic collection and comparison of noise data (Part 1, page 6) • Guideline - Preparation of the “noise sections” in specific machinery safety standards 5.1.1 Noise declaration measurements Noise declaration can be defined as a specification of the sound radiation from a given source (indication of noise data of the source). This implies that noise declaration guidelines do not automatically also include specification of limit values for sound radiation. As previously mentioned this must be left to individual agreements between supplier and purchaser or to rules stipulated by national authorities. Noise declaration may serve a number of purposes: 1. Comparison of sound data for corresponding products of different makes 2. Calculation of the influence of a sound source on its surroundings 3. Control that a product is in accordance with given limit values as regards sound 4. Determination of noise-attenuating effort, if needed 5. Evaluation of the effect of noise-attenuating measures 5.2 Determination of (stationary) loudness The method for determination of stationary loudness is described in ISO 532 B – it is, however, a little complicated and time-consuming to do it manually. It is easier to use a sound level meter which has this option or a postprocessing method using recordings. A standardised method for calculation of non-stationary loudness is on its way as an addendum to DIN 45631. Several measurement systems have the feature for calculating non-stationary loudness, but the results are very dependent on what system is used. 5.3 Determination of sound pressure level The EN ISO 11200-11204 standards describe guidelines for the determination of the sound pressure level at operator's position or at another specified position. Determining which of the guidelines is relevant, depends on the measurement accuracy desired, the acoustic environment of the measurement site, and of course also on the product in question. EN ISO 11200 gives guidelines for the use of basic standards for determination of emission sound pressure levels at a workstation and at other specified positions. EN ISO 11201 describes the measurement of emission sound pressure levels at a workstation and at other specified positions – engineering guideline in an essentially free field over a reflecting plane – precision grade 2. EN ISO 11202 describes the measurement of emission sound pressure levels at a workstation and at other specified positions - survey guideline in situ - precision grade 3. EN ISO 11203 describes the determination of emission sound pressure levels at a workstation and at other specified positions based on the sound power level, grade 2 or 3. EN ISO 11204 describes the measurement of emission sound pressure levels at a workstation and at other specified position where environmental corrections are required - grade 2 or 3. (Part 1, page 7) 5.4 Determination of sound power levels 5.4.1 Determination of sound power level based on sound pressure The EN ISO 3740-3747 describes methods for determination of the sound power level of noise sources. The methods of EN ISO 3740-3747 are based on measurements of sound pressure level. 5.4.2 Determination of sound power level based on sound intensity EN ISO 9614 is based on the use of sound intensity measurements to determine the sound power. The standard consists of two parts: Part 1: Measurement at discrete points and Part 2: Measurement by scanning. These standards are more complicated to use, and advanced measurement equipment is necessary. 6. Measurement guidelines 6.1 Types of sound sources The methods specified in this guideline are applicable to all types of sound sources both moving and stationary for indoor and outdoor use. For different types of sound sources the following descriptors should be measured: Small units • Loudness level at operator's position • Sound pressure level at operator's position • Sound power level (measured in laboratory) Single machines • Loudness level at operator's position • Sound pressure level at operator's position • Sound power level (measured in industrial environment) Large/composite plants • Loudness level measured in specified positions • Sound pressure level measured in specified positions • Sound power level for sub-components 6.2 Instrumentation Measurement equipment applied for this measurement guideline can be performed either using monaural or binaural techniques, i.e. using one microphone or a mannequin (artificial head / artificial head and torso). For comparison between measurements from different laboratories, measurements based on pressure microphones are normally preferred. Under certain conditions, particularly where the sound sources are close to the (operator’s) head, there may be an essential difference between monaural and binaural measurements. If a mannequin is used, then the quality of the microphones should conform to the relevant standard for microphones and the mannequin. In diffuse field a correction for the influence of the head should be applied. The type of mannequin and correction should be specified in this case. In order to perform postanalysis (other sound quality metrics) it is recommended to carry out recordings. (Part 1, page 8) A linear recording device should be used. If codecs are used, care should be taken by making comparative measurements with a linear device. Normally, an integrating-averaging sound level meter complying with IEC 804 shall be used to measure the loudness (and emission sound pressure level). If it can be shown that the sound pressure level fluctuations measured with the time-weighting characteristic S, are less than ± 1 dB, a conventional sound level meter complying with IEC 651 may be used. In this case, the sound pressure level is taken to be the average of the maximum and minimum levels during the period of observation, measured with the time-weighting characteristic S. All the equipment shall be calibrated according to relevant regulations and guidelines before the actual measurement. Ambient condition may have an adverse effect on the system used for the measurements. Such conditions (e.g. strong electromagnetic fields, wind, temperature, etc.) shall be avoided by proper selection or positioning of the system. 6.3 Test environment The test environment should resemble the typical environment of a typical user situation as much as possible. At least the following information should be described: a) If indoors, description of physical treatment of walls, ceiling and floor; make a sketch showing the location of the product under test and room contents; acoustical qualification of room in accordance with 6.2 of EN ISO 11201. b) If outdoors, make a sketch showing the location of the product under test with respect to surrounding terrain, including 1) physical description of test environment 2) air temperature in degrees Celsius, barometric pressure in pascals, and relative humidity as a percentage 3) wind speed, in metres per second In both cases it is recommended to specify all other relevant details and if possible take a picture! It is recommended that the background noise of the loudness measurements should be more than 20 phons below the value of the total measured loudness level of the sound source under test. If it is less than 10 dB, special care should be taken. Regarding background noise of other measurements it should be at least 6 dB and preferably more than 15 dB below the level due to the sound source under test. 6.4 Specification of installation and operating conditions The manner in which the sound source under test is installed and operated may have a significant influence on the loudness levels at the specified positions. Relevant instructions of a noise test code, if any exists for the family of sound source under test, should be followed. The operating conditions used during the measurements shall be representative of normal use of the product following a given noise test code. If there is no test code, the product under test shall, if possible, be operated in a manner which is typical of normal use. In such a case, one or more of the following operating conditions shall be selected: a) Sound source under specified load and operating conditions b) Sound source under full load (if different from the first condition above) c) Sound source under no load (idling) (Part 1, page 9) d) Sound source under operating conditions corresponding to maximum sound generation representative of normal use e) Sound source with simulated load operating under carefully defined conditions f) Sound source under operating conditions with characteristic operational cycle Loudness levels at specified positions shall be determined for any desired set of operating conditions (i.e. temperature, humidity, device speed, etc.). These test conditions shall be selected beforehand and shall be held constant during the test. The product under test shall be in the desired operating condition before any noise measurements are made. Use of standardised operating conditions gives a possibility of a general, correct comparison of noise data of different makes. It is also important to note that the standardised operating conditions cannot be adequate for all relevant applications of the machine in question. If, when purchasing a machine, there is a need for noise data for specific applications, it is necessary to perform measurements under the operating conditions in question. 6.5 Specification of measurement positions 6.5.1 At the operator’s position One or more operator positions shall be specified for equipment which requires operator attention while in operating mode – at a typical listening position. 6.5.2 Enclosed work station position When the operator is located in an enclosed cab or in an enclosure remote from the product under test, the cab or enclosure is regarded as an integral part of the machine under test and, consequently, sound reflections inside the cab or enclosure are considered contributions to the emission sound pressure level. No environmental corrections are permitted. During measurements, doors and windows of the cabin or enclosure shall be open or closed as defined in the noise test code for the machinery or equipment being measured. If the work station or bystander’s position of the product is located inside a cab or a cabin, an additional “conventional” work station or bystander’s position outside the cab or cabin (e.g. for maintenance) in the vicinity of product under test shall be specified. 6.5.3 At the bystander position For equipment which does not require operator attention while in operating mode, an operator position need not to be specified. But a ”conventional” work station for services, maintenance, etc. or one ore more bystander positions shall be defined and stated in the noise test code. 6.5.4 Measurement positions in other cases In this case at least 4 bystander positions shall be defined. Typically one will place the measurement equipment 1 m away from a virtual reference box as defined in EN ISO 3744 or EN ISO 3745 at a height of 1.5 m above the ground. 6.6 Measurements 6.6.1 Measurement time interval The measurement time interval shall be chosen in such a way that the loudness level and, as required, the time characteristics of sound emission at specified positions can be determined for the specified operating conditions. (Part 1, page 10) For products with a specified operational cycle, it is usually necessary to extend the measurement time interval to an integral number of consecutive operational cycles. The measurement time interval shall correspond only to the operational periods for which the emission sound pressure level and, as required, the time characteristics of sound emission are desired. Values of the measurement time interval, possible sub-measurement time intervals and number of operational cycles contained in the measurement time interval are usually to be found in the noise test code specific to the family of machinery or equipment to which the product under test belongs, if any exists. In any case, these values shall be identical to those defined for determining the sound power level of the product under test. 6.6.2 Steady noise If the sound emission at a specified position is steady for the specified operating conditions, the measurement time interval shall be at least 15 seconds. 6.6.3 Non-steady noise If the sound emission at a specified position is not steady for the specified operating condition, the measurement time interval and operational periods of the product under test shall be carefully defined and reported in the test results. They are normally specified in the relevant noise test code, if any exist. 6.6.4 Measurements in frequency bands If supplementarily, measurements are to be made in octave or 1/3 octave bands, the minimum observation shall be 30 seconds for the frequency bands centred on or below 160 Hz, and 15 seconds for the frequency bands centred on or above 200 Hz. 6.6.5 Repetition of measurements In order to reduce the uncertainty of the determination of loudness, levels at the specified positions, it may be necessary, for a specific type of product, to repeat the measurement a number of times as specified in the noise test code for the family of products or equipment to which the product under test belongs. The value to be used after repeated measurements shall be that defined in the noise test code, if any exists. Repeating measurements involves the following procedure: a) The product under test is turned off and on again, if feasible b) The microphone or mannequin is moved away and set again at the specified position c) The measurement is carried out again in the same environment, with the same instrumentation over the same measurement time interval and for the same mounting and operating conditions 6.6.6 Impulsive sound If the sound emission is impulsive, particular care shall be taken when measuring to ensure that the dynamic range of the instrumentation is sufficiently large, and the sound level meter / mannequin system is equipped with an overload indicator. For measurement of the time characteristics of impulsive sound emissions, additional to the repetition procedure described in section 6.6.5, the measurement time interval shall include at least 10 impulsive events, unless other is specified in the noise test code, if any. In order to determine whether or not the sound emission contains impulsive components, one of the procedures given in Bibliography [8] may be used. 6.6.7 Supplementary measurements If technically possible, it is recommended also to measure and include the A-weighted level and 1/3 octave bands as supplementary information. (Part 1, page 11) 6.7 Accuracy Noise data should always also include information on measurement accuracy, as the measurement accuracy should be taken into consideration when indicating guarantee values and at control measurements. The uncertainty rises from several factors, some associated with the environmental conditions, some with the instrumentation, and some with performing the measurement. Different degrees of accuracy can be obtained - depending on what is asked for - engineering or survey precision, see EN ISO11201 or 11202 section 4. When using a measurement system with a mannequin bias from transmission, disturbance of the sound field, the diffuse field correction etc. should be taken into account. If digital equipment is used, the influence of resolution should be stated. 7. Information to be recorded and reported 7.1 Information to be recorded The following information, when applicable, shall be compiled and recorded for all measurements made in accordance with this guideline. Rounding of computed data values shall occur only after performing the final computational step before reporting. 7.1.1 Product under test Description of the product, including its - type technical data dimensions manufacturer product serial number year of manufacture 7.1.2 Test conditions a) Precise quantitative description of operating conditions and, if relevant, operational periods and cycle b) Mounting conditions c) Location of product in the test environment d) If the product under test has multiple noise sources, a description of the sources in operation during the measurements. 7.1.3 Acoustic environment Description of the test environment with relevant sketches and photos, see also section 6.3. 7.1.4 Instrumentation a) Equipment used for measurements, including name, type, serial number and manufacturer b) Method used for verifying the calibration of the measuring system; the date, place and result of calibration shall be recorded c) Characteristics of windscreen (if any) (Part 1, page 12) 7.1.5 Location of specified positions A precise quantitative description shall be recorded of all positions where the loudness levels have been measured. 7.1.6 Noise data • All measured loudness level data • Other supplementary sound pressure level data • Background noise levels and possible correction at each specified position. If required, background noise levels and correction in frequency bands. 7.2 Information to be reported Results of the loudness measurements at the specified positions. The loudness N shall be reported with one decimal in sone and the loudness level LN with one decimal in phon. If a standardised noise test code exists for the product under test, the supplementary data to be reported are probably specified in this. Additionally, the following information shall be given: • State that the measurement has been performed in accordance with the specifications in the present Nordtest Guideline • Recording and analysis equipment, type, make and model including calibration • Place and date when the measurements were performed, and person responsible for the test, date 8. Bibliography 1) Bernsen, J. et al., “Sound in design”, Danish Design Centre & DELTA Acoustics & Vibration, 1999. 2) Blauert, J. & Jekosch, U., “Sound Quality Evaluation – a multilayered problem”, EEA-Tutorium, Antwerp, 1996. 3) Bodden, M., “Importance of binaural hearing”, Psych. Acoust., Editor A. Schick., Universität Oldenburg, 1993. 4) Fog, C.L., “Optimal Product Sound: Design and Construction Guidelines for Developing Products with Desirable Sound Characteristics and Minimal Noise”, Report SPM 144 (in Danish), DELTA Acoustics & Vibration, Copenhagen, 1998. 5) Järvinen, A. & Maijala, P., “On the use of real head recordings in product sound design”, InterNoise 1997 Proceedings, Vol. II, p. 1143-1146. 6) Maijala, P., “Better binaural recordings using the real human head”, Inter-Noise 2001 Proceedings, Vol. II, p. 1135-1138. 7) Pedersen, T. Holm & Fog, C.L., “Optimisation of perceived product quality”, Euronoise, 1998. 8) Pedersen, T. Holm, “Impulsive noise - Objective method for measuring the prominence of impulsive sounds and for adjustment of LAeq, Inter-Noise 2001 Proceedings, p. xx-xx. 9) Porter, N.D. & Berry, B.F., “A study of standard methods for measuring the (product) sound quality of industrial products”, MPL report CIRA(EXT)021, January 1997. 10) Zwicker, E. & Fastl, H. “Psychoacoustics, Fact and Models”, Springer Verlag, 1999. (Part 2, page 1) Nordtest Guideline Draft 5th working draft, 2001-08-23 AV 1456/01 Part 2 prepared by Torben Holm Pedersen Measurements and judgments of sound in relation to human sound perception Part 2: Guidelines for listening tests Contents 1. 2. 3. 4. Scope ................................................................................................................................................2 Field of application ............................................................................................................................2 Normative references........................................................................................................................2 Definitions and terms ........................................................................................................................3 4.1 Listening tests ............................................................................................................................3 4.2 Product sound ............................................................................................................................6 4.3 Psychoacoustic metrics .............................................................................................................7 4.4 Metrics related to sound pressure levels ...................................................................................7 5. Instrumentation..................................................................................................................................8 5.1 Recording and playback equipment ..........................................................................................8 5.2 Measuring equipment for physical measurements ....................................................................8 6. Types of listening tests......................................................................................................................8 6.1 Semantic differential test............................................................................................................8 6.2 Paired comparison test ............................................................................................................10 7. Selection of words ...........................................................................................................................10 8. Selection and training of test persons.............................................................................................10 8.1 Test jury ...................................................................................................................................10 8.2 Expert panel .............................................................................................................................11 9. Planning and preparing the listening test........................................................................................12 9.1 Form of test ..............................................................................................................................12 9.2 Modes of listening ....................................................................................................................12 9.3 Randomising ............................................................................................................................12 9.4 Listening conditions .................................................................................................................12 9.5 Duration of tests .......................................................................................................................13 9.6 Recording and reproduction ....................................................................................................13 9.7 The sound samples..................................................................................................................14 9.8 Announcements .......................................................................................................................14 10. Running the test sessions ...........................................................................................................14 10.1 Introduction to listening test .....................................................................................................14 10.2 Instruction of test persons........................................................................................................14 10.3 Sample answering sheets and pretest.....................................................................................15 10.4 Listening test ............................................................................................................................15 10.5 Debriefing.................................................................................................................................15 11. Physical measurements ..............................................................................................................15 12. Statistical methods ......................................................................................................................15 13. Accuracy and significance ...........................................................................................................16 14. Information to be reported ...........................................................................................................16 15. Bibliography .................................................................................................................................16 16. Appendix – Psychoacoustics.......................................................................................................19 16.1 Psychoacoustic metrics ...........................................................................................................19 16.2 Psychoacoustics ......................................................................................................................20 16.3 Psychometric methods.............................................................................................................20 (Part 2, page 2) Part 2: Guidelines for listening tests 1. Scope It is the intention of these guidelines to provide a tool to increase effectiveness and reliability of listening tests. The guidelines are also meant as a help to perform comparable listening tests. One of the essential items when performing listening tests is to distinguish between objective and subjective tests. This matter is clarified in the guidelines. Since new concepts within acoustics are introduced, section 4 Definitions and terms is rather comprehensive. For some topics in this guideline specific recommendations are given. For other topics the text has the character of a list of items for consideration because the relevant choices depend on the specific test situation. 2. Field of application This guideline is related to listening tests in the field of product sound (e.g. for optimising product sound quality) and of psychoacoustic testing. The principles may also be utilised in measurements of the annoyance potential (see definitions) of environmental noise and noise at workplaces. In general annoyance of environmental noise should be performed as surveys according to ISO/DTS 15666, see section 3 Normative references. The results of listening tests according to these guidelines may be used for comparison of different products, situations and/or working conditions. The results may also be a supplement to physical measurements of the sound/noise or may be used for finding the relevant physical measure (metric) for a certain sound characteristic or situation by correlating the results from the listening tests with measurements using technical instrumentation (physical measurements). 3. Normative references The following normative documents contain provisions that constitute - through reference in this text this Nordtest Guideline. Parties using this Nordtest Guideline are encouraged to investigate the possibility of applying the most recent editions of the normative documents indicated below. For undated references, the latest edition of the normative document referred to applies. Nordtest as well as members of ISO and IEC maintain registers of currently valid International Standards. IEC Publication 61672, “Electroacoustics - Sound level meters”1) IEC Publications 60651 and 60804, “Sound level meters” IEC Publication 60942, “Acoustic calibrators” IEC Publication 61260, “1/3 and 1/1 octave filters” IEC Publication 60268 series, “Sound system equipment” IEC Publication 60268-13, “Listening tests on loudspeakers” ISO 1996/1, “Acoustics - Description, measurement, and assessment of environmental noise. Part 1: Basic quantities and assessment procedures” 8. ISO/CD 1996-2, “Acoustics - Description, measurement, and assessment of environmental noise. Part 2: Determination of environmental noise levels”, 2001-05-21 9. ISO/DTS 15666 Draft Technical specification, “Acoustics - Assessment of noise annoyance by means of social and socio-acoustic surveys”, ISO/TC 43/SC 1, N1313, 2001-04-06 10. ISO 532-1975 (E) “Acoustics - Method for calculating loudness level, Section two” 11. ISO 226, 1987 (E) “Acoustics - Normal equal-loudness level contours” 12. ISO 5492, ”Sensory analysis – Vocabulary” 1. 2. 3. 4. 5. 6. 7. 1) To be published, replaces IEC 60651 and IEC 60804. (Part 2, page 3) 13. 14. 15. 16. ISO 4121, “Sensory analysis - Methodology - Evaluation of food by methods using scales” ISO 4121, “Sensory analysis – Methodology – Paired comparison test” ISO 6658, “Sensory analysis – Methodology – General guidance” NORDTEST Method proposal 1516-00, “Impulsive noise, Objective method for the measurements of prominence of impulsive sounds and for the adjustment of LAeq”, 2001-05-01 4. Definitions and terms 4.1 Listening tests 4.1.1 Listening test A listening test is a test where one or more persons in a systematic way are presented to samples of sound and requested to give their evaluations/response in a prescribed manner. Listening tests may be either objective (perceptive - what do the persons hear) or subjective (affective - what do the persons prefer or dislike) as illustrated in figure 1. It is essential for the results that a clear distinction between these two types of tests is made. Physical masurement Auditive measurement Filter 1 Filter 2 M2 M1 Physical stimulus Affective test Preferences Perceived stimulus Sensory sensitivity and selectivity Objective M3 Mood Interest Emotion Background Expectation Subjective Figure 1. Simplified illustration of human sound perception and affection. Filter 1 symbolises our senses (e.g. hearing). Filter 2 symbolises our mental processing of sensory perceptions. M1-M3 illustrates different measuring points. Measurements at each of these points may be made independently of each other, and for a certain purpose measurements in any point(s) without the others may be made. M1: physical measurements (sound pressure levels, loudness …), M2: perceptive measurements (psychoacoustics), and M3: affective tests. 4.1.2 Perceptive measurements Perceptive tests are objective tests (sensory evaluations, perceptive analysis) where humans are used as measuring instruments. The characteristics of the perceived stimulus are rated in objective terms without asking the test persons for any preferences. The tests are usually made with a panel of trained experts. The persons are trained to express their sensory perception in terms that have been well defined to the test persons in advance. The panel may be trained for a specific purpose. The tests may be discriminative (e.g. paired comparison) or descriptive (e.g. semantic differential test). (Part 2, page 4) 4.1.3 Auditive measurements - objective listening tests Auditive tests are perceptive measurements with sound as the stimulus. They are objective listening tests where the test persons express their perception of the sound characteristics in terms which describe the sound as e.g. “loud”, “soft“, “audible tones”, “sharp”, “rough”, “pulsating”, etc. The main purpose is to give information about the character of the sound as perceived by humans. 4.1.4 Affective tests – subjective listening tests Affective tests are subjective listening tests normally performed with a group of naive (untrained and without experience in listening tests) test persons who are representative of the relevant user group – “a consumer jury”. As they may use other words than acousticians for the attributes of the product they hear, the relevant words for expression of the heard sound often have to be “found” before the answering forms for the listening tests can be made. This can be done e.g. by interviews or focus group discussions. Answers as an immediate response of the persons’ judgments are preferred. Affective tests are used when the preferred characteristics or the “dislikes” of a product are sought (preference tests). The main purpose is to give information about humans in relation to the sound in a given context. 4.1.5 Assessor (sensory) / Test person / Subject Any person taking part in a sensory test. NOTE – A naive assessor is a person who does not meet any particular criterion in relation to testing. An initiated assessor is a person who has previous knowledge relevant to the sensory test. 4.1.6 Selected assessor / Selected test person Assessor chosen for his/her ability to perform a sensory test. 4.1.7 Expert / Expert listener In the general sense, a person who, through knowledge or experience, has competence to give his/her opinion in the fields about which he/she is consulted. In sensory analysis there are two types of expert, i.e. the “expert assessor” and the “specialised expert assessor”. 4.1.8 Expert assessor Selected assessor with a high degree of sensory (listening) sensitivity and experience of sensory (listening) methodology, who is able to make consistent and repeatable sensory (listening) assessments of various products or sounds. 4.1.9 Specialised expert assessor Expert assessor who has additional experiences as a specialist in the product and/or process, acoustics and/or marketing, and who is able to perform sensory analysis of the product and to evaluate or predict effects of variations relating to design changes, sound propagation, influence of other (sound) characteristics, etc. 4.1.10 Panel / Jury Group of assessors chosen to participate in a sensory test. 4.1.11 Consumer Normally a person who uses a product (user). It may also be a person decisive for the purchase of products (manager, buyer). 4.1.12 Neighbour For the purpose of this guideline a neighbour is defined as a person who is exposed to the sound/noise from a product without being a consumer of that particular product. (Part 2, page 5) 4.1.13 Expert panel A group of experts who participate in an objective (or perceptive) listening test. 4.1.14 Test jury A group of persons (users/ buyers/neighbours) who participate in affective (or preference) listening tests. 4.1.15 Acceptability State of a product or sound favourably received by a given individual or population, in terms of its attributes or its judged confirmation with standard(s) or stated requirement(s). 4.1.16 Acceptance The act of a given individual or population of finding that a product or sound answers satisfactorily to his/her/its expectations. 4.1.17 Preference Expression of the emotional state or reaction of an assessor which leads him/her to prefer a specific product (or sound) to other products (or sounds) of the same type or function. 4.1.18 Noise annoyance Noise-induced annoyance is a person’s adverse reaction to noise. The annoyance caused by noise is a complex relationship between the noise and other physical variables as well as personal, psychosocial, socio-economic and other non-physical variables. Noise annoyance may e.g. be measured by socio-acoustic surveys among people who have been exposed to the noise in a certain context (home environment or workplace) for a period of time (months). 4.1.19 Annoyance potential Annoyance potential is the annoyance of sounds measured under laboratory conditions. The listeners may (recommended) or may not be presented to a scenario for the sound. NOTE - Reliable measurements of noise annoyance may not be obtained under laboratory conditions, and it is uncertain to which degree annoyance potential correlates with noise annoyance. If the test persons are presented to the context of the sound by instructions, by pictures, by simulated surroundings or similar, it is believed that the ranking of the annoyance potential of different sounds correlates with the annoyance. 4.1.20 Scenario A context for the perception of the sound that is simulated for the test person (by pictures, video …) or a context that the test person is asked to imagine (imagine that you hear this sound while sitting …). 4.1.21 Auditory attributes Acoustic characteristics of a sound rendered by a perceptual analysis. 4.1.22 Antagonism Joint action of two or more stimuli, whose combination elicits a level of sensation lower than that expected from superimposing the effects of each stimulus taken separately. 4.1.23 Synergism Joint action of two or more stimuli, whose combination elicits a level of sensation in excess of that expected from a simple addition of the effects of each stimulus taken separately. (Part 2, page 6) 4.1.24 Masking Decrease in the intensity or change in the quality of the perception of one stimulus by the simultaneous exposure of other stimuli. 4.1.25 Contrast effect Increase in response to differences between two simultaneous or consecutive stimuli. 4.1.26 Convergence / Assimilation effect Decrease in response to differences between two simultaneous or consecutive stimuli. 4.2 Product sound 4.2.1 Product For the purpose of this guideline a product is defined as the item under investigation. The term product is to be understood in a broad sense, the product might be a household article, a car, a train, a plane, a room, a factory ... Even events (as e.g. traffic) may be defined as a product for the purpose of this guideline. 4.2.2 Perceived product quality Perceived product quality is a collection of features that confer the product’s ability to satisfy stated or implied needs. This is evaluated on the basis of the totality of perceived features and characteristics of the product, with reference to the expectations and implied needs that are apparent in the users’ cognitive and emotional situations. 4.2.3 Product sound quality Product sound quality refers to the adequacy of the sound from a product. The adequacy is evaluated on the basis of the totality of the auditory characteristics of the sound, with reference to the set of desirable product features that are apparent in the user’s cognitive and emotional situation. Product Sound Quality as defined above is a concept to emphasise that we are concerned with the characteristics of the product. Product Sound Quality is not an inherent quality of the sound or the product, but is a result of the perceptive and mental processes when a listener is exposed to the sound from a product. Product sound quality is not an absolute parameter, but oriented towards a certain product and customer. NOTE - Evaluation of product sound quality depends on three main areas of knowledge and experience: − Acoustics, especially related to physical design and measurements − Psychoacoustics: i.e. the relationship between the acoustic input and the perception of the sound − Psychology, effects and rules, which govern the judgment process by which the product sound quality is assessed Meaningful and relevant listening tests shall take these three areas into account. The term Product Sound Quality indicates that we are concerned with the characteristics of the product. 4.2.4 Sound quality The sound quality is the quality (e.g. the fidelity of music, the intelligibility of speech) of reproduced sound or of generated signal sounds (e.g. warning signals). The term Sound Quality indicates that we are concerned with the characteristics of the sound. 4.2.5 Passive sounds Passive sounds are the sounds that are produced when the product is touched (knocked, pressed, etc.), switches are operated …. Examples: the door slamming of cars, refrigerators, etc., the sounds when you operate a PC keyboard, the crunching sound of Cornflakes. (Part 2, page 7) 4.2.6 Active sounds Active sounds are generated by the product itself. These active sounds can be further categorised as: Transition sounds (or action sounds), e.g. when a washing machine goes from washing stage to spinning stage. The sound has to do with the function and is not intended as a direct signal. This kind of sounds can be continuous or impulsive, but is not generally stationary over long periods of time. Operative sounds (or running sounds) are generated when a product is in a given part of its cycle (washing, spinning, rinsing, etc.). The running sounds may vary with the different stages of the cycle, and they may be continuous, stationary, or irregular. Signal sounds are direct signals to the user or surroundings (a washing machine may give a humming sound to indicate that the machine has finished). The signal sounds may be further subdivided into alarm, warning, activating, feedback (may also be passive sounds), and informative sounds and may also take the form of auditory icons (“earcons”). 4.2.7 Metric Metrics or measures for the sound are the result of a physical measurement. The metrics may be any relevant traditional noise measures, may be psychoacoustically related measures (see section 4.3), other measures (e.g. the sound pressure level within a specified frequency range, rise time and level difference of impulsive sounds), or any combinations of these. 4.3 Psychoacoustic metrics Psychoacoustic metrics are a means of quantifying sound characteristics in a way that correlates well with human sound perception. Relevant metrics are: − − − − − − − − Loudness (sone) Loudness level (phon) Specific loudness (son/bark) Sharpness (acum) Fluctuation strength (vacil) Roughness (asper) Prominence of audible tones Prominence of impulses Brief definitions of these metrics are given in the Appendix section 16.1. 4.4 Metrics related to sound pressure levels Examples of relevant metrics are: − − − − − − Sound pressure level Frequency-weighted sound pressure level (weightings: A, B, C, or D) Time-weighted sound pressure level (weightings: S, F, or I) Energy-equivalent A-weighted sound pressure level 1/1- and/or 1/3-octave band sound pressure levels Band-limited sound pressure levels Metrics related to sound pressure (and sound power) level, in decibels, are defined in part 1, section 4 of this guideline. (Part 2, page 8) 5. Instrumentation This section gives a description of appropriate types of measurement equipment to be used for measurements as described in sections 9.1 and 11. 5.1 Recording and playback equipment If recording and playback equipment is to be used for presentation of the sound, a high quality of reproduction is normally preferred to substitute the natural sources and environments. Only in special cases, e.g. where the topic of the investigations is detection of defects of the reproduced sound, limited quality should be accepted. The types of equipment used should be stated in the measurement report together with the technical data. At least data for frequency response, non-linear distortion, dynamic range / digital resolution, the equivalent A-weighted acoustic background noise level due to electric noise in the measuring set-up, and the similarity of stereo channels should be stated. The technical data shall be given with reference to the IEC 268 series (see Normative references 5). All recording and playback equipment should be essentially linear. If non-linear equipment is used, e.g. equipment using codecs, tests should be made in advance so that the codecs do not subtract or add audible signal components. For binaural recordings mannequins or microphones in real ears may be used. 5.2 Measuring equipment for physical measurements Measurement equipment applied for the physical measurements of sound pressure related measures shall be Class 1 and shall comply with appropriate IEC requirements, see Normative references 1-5. At present no standards exist for equipment for psychoacoustic measurements. For stationary loudness and sharpness there is generally good agreement between different makes of equipment; this is not always the case for non-stationary loudness and the derived metrics Fluctuation strength and Roughness. It is therefore essential that the make, type numbers and software versions of the equipment are stated in the measurement report. The equipment shall be calibrated according to relevant regulations and guidelines for the actual type of measurement. 6. Types of listening tests Many types of listening tests and test methods exist. A number of methods as well as guidelines for the number of test persons are given in Normative references 15. A list of psychometric test methods is given in the Appendix section 16. In this guideline only two main types of test, namely Semantic differential test and Paired comparison test, will be described. NOTE - The type of assessments or judgments used in listening tests depends on the type of test: There may be absolute or relative assessment. The form of assessment may be dominance judgments or similarity judgments. The methods of assessment may be: detection, ordering (ranking), rating, recognition, magnitude judgments … 6.1 Semantic differential test The semantic differential test is designed to give an assessment in relation to words for sound characteristics without comparison to other sounds. As no direct comparison is intended, pauses between sound samples shall be at least 10 seconds. The duration of the sounds is typically longer than sound (Part 2, page 9) samples for paired comparison tests and sounds (e.g. non-stationary) with long duration (minutes) may be assessed by this method. A graphical scale as shown in figure 2 (e.g. 15 cm long with marks 2 cm from the ends) is recommended. It does not restrict the test person to specific points and has the advantage of absence of any numerical value associated with the response plus the limited use of words to minimise word bias. After the test the answers are measured e.g. in cm from the left end of the line. Weak Absent Strong Dominant Figure 2. Examples of 15 cm long answering scales for semantic differential tests. The upper scale has open ends, the lower scale has one closed end. In paper-based versions of the test the subjects place a vertical line at the place that reflects the intensity of the characteristic. For computerised tests the subjects move a slider on a video display unit with the mouse. Typically two anchors reflect a continuum from low to high intensity. The scales in figure 2 are unipolar in the sense that they describe the intensity of a characteristic. The scales may also be bipolar so that the words at the ends describe opposite characteristics. For bipolar scales it shall be considered if the midpoint is meaningful. Numeric scales may also be used, e.g. seven- or nine-point scales. They are fast to read after the test, but do not allow the test persons to choose numbers between the fixed points. The nine-point hedonic scale shown in figure 3 is often used for measurements of product acceptance or preference (an affective test). Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely Figure 3. Example of a nine-point hedonic scale. The subjects’ task is to place a cross in the box that best represents their attitude to the product. The responses are converted to numerical values for computational purposes: Like extremely = 9; dislike extremely = 1. Many other scales exist, see Bibliography [6] and [27] and Normative references 6 and 13. Special scales and procedures are used for measurement of noise annoyance, see Normative references 9. These scales may be adapted for measurements of annoyance potential (defined in section 4.1.19). (Part 2, page 10) 6.2 Paired comparison test In paired comparison tests the sounds are presented in pairs for comparison of auditory attributes. Each sound is successively compared to all other sounds. Two comparisons are to be made for each pair (A, B): A comparison with sound A presented first and another comparison with sound B first. The number of comparisons to be made are n(n-1), where n is the number of sounds. The paired comparison test gives the listeners the ability to detect small differences of a specified attribute for different sounds. The result will typically be of the type “higher” or “lower” without information about the absolute placement (in the “high” end or the “low” end) of the sounds. Pauses between sounds to be compared shall be less than one second. The duration of the sounds is typically shorter than sound samples for semantic differential tests. 7. Selection of words The words used for questions and/or to scale the responses must be familiar, easily understood, and unambiguous to the subjects. The words must be readily related to the product and the task, and they must make sense to the subjects as to how they are applied in the test. Words that have specific and useful meaning to the requestor and professional acoustician may be much less meaningful to subjects, especially if they are not trained subjects. A method of obtaining relevant words is a focus group discussion. This is a guided discussion with a small group of persons without too big differences in background and age. The purpose is to get the response of the group on a limited subject (e.g. finding words describing the product sound, e.g. sharp pulsating …, for expert panels or words, e.g. powerful, luxurious …, relating product attributes to the perceived sound for jury tests). Individual interviews with a few experts knowing the product and or the market or a few individuals representative of the users/neighbours is a convenient and relative fast method of obtaining relevant words. Care (e.g. by pilot tests with thorough debriefing) shall be taken that the selected words have the same meaning to other persons. For the interview technique (as well as eliciting words) special techniques exist, see Bibliography [14] and [16]. Words characterising the quality of reproduced sound may be found in Normative references 6, and a number of already existing dimensional analyses of large adjective pools for various products as well as some more common dimensions exist, see Bibliography [17] and [18]. 8. Selection and training of test persons 8.1 Test jury 8.1.1 Selection of members of a jury The test persons are normally a representative group of potential buyers, users, neighbours or other persons relevant for the actual situation. The persons should be qualified based on typically desired demographic characteristics (gender, age, education, cultural and ethnic background, income …), physiological criteria (e.g. hearing ability) and usage criteria. In general, persons involved with the product as sales persons, technicians, designers, and employees at the company producing the product under test should be avoided. If such persons are used, their attitudinal responses should be compared with data from other populations to determine that the subjects are comparable, at least in their preferences towards a specific product or sound characteristic. Test juries are used in affective or preference tests. At these tests the immediate response is often sought. Therefore naïve listeners are preferred for these types of test. (Part 2, page 11) 8.1.2 Training The training of the persons in a test jury should be limited to training in the test procedure and to pretests with the purpose of ensuring that the words or other descriptors of the sounds presented are familiar and unambiguous to the subjects. It should be checked that the individual responses attain a satisfactory consistency. 8.1.3 Number of test persons Depending on the type of product and the accuracy and representability of the test in relation to the relevant market segment or other groups of persons, the number of test persons should be in the range 10 to several hundred. Reliable tests are seen with 16-32 persons. The variance of the test results will give guidance for the reliability of the test. Guidelines for the number of test persons are given in Normative references 15. The necessary number of test persons may also be found from statistical methods, see Bibliography [4]. NOTE - Individual differences: To minimise variance the test persons may be checked for noise sensitivity, noise/ product attitudes, personality, current mood, experience, etc. Standardised scales exist. It is clearly most important when using naïve participants, see Bibliography [11]. 8.2 Expert panel The test persons are trained listeners. The tests are perceptive measurements (i.e. objective measurements with humans as measuring instruments). The users are not representative of any specific group, and preferences are not asked for. 8.2.1 Selection Expert listeners may be recruited from amongst the office, plant or laboratory staff of the organisation involved in production or testing the product. Still it is advisable to avoid persons too personally involved with the products being examined, as they may cause bias. If such persons participate, their assessments should be compared with data from other groups in the panel to determine that the results are comparable for the specific sound characteristic under assessment. The expert listeners may also be recruited outside the organisation by advertisement, among persons visiting the organisation or among personal acquaintances. The criteria for selection may be: audiometric tests, reliability of judgments of pretest sessions, and criteria for listening experience. 8.2.2 Training The effect of training may be a change in an absolute threshold, in the ability to detect differences between stimuli, or the consistency of ratings. Such changes in the course of an experiment must be minimised as they represent an extra error variance if the data of successive experiments are pooled. The main reason for training the expert assessors for a certain type of judgment is therefore to ensure that different aspects of judgment performance have reached a stable level before the listening tests start. This will ensure that the assessments made in the listening tests are as precise and cost- and time-effective as possible. Special efforts shall be made to ensure that the words and the scales for the responses are familiar and unambiguous to the subjects. All words describing sound characteristics shall be well defined, and the corresponding sound characteristics shall be demonstrated to the listeners during training sessions. 8.2.3 Number of test persons Ideally it is not desirable to operate with a panel of less than 10 selected assessors, and it is necessary to recruit more persons (e.g. 2 or 3 times as many) than shall constitute the final panel. (Part 2, page 12) For survey measurements acceptable results may be obtained with less than 10 experts. Successful tests are seen with only 2 specialised expert listeners. The variability between selected and trained assessors is less than between naïve listeners. 9. Planning and preparing the listening test 9.1 Form of test At first the form of the test should be considered: − − With or without a test leader Individual listening or group listening At this point the presentation mode should also be considered. It shall be decided whether the experimenter or the subject control the: − − − − − 9.2 Order of stimuli Duration of test signals Repetitions of test signals Pauses between test signals Time for answering Modes of listening If possible and practical, it is recommended that listening tests are performed with natural (live) sound sources in relevant sound fields and surroundings. Binaural recordings presented over calibrated stereo headphones may give the best approximation to the live situation. For specific purposes loudspeaker listening (mono or stereo) may be used. In that case any room acoustic influence on the perceived sound should be taken into account. 9.3 Randomising Ideally the order of the sound samples shall be randomised so that no test person is presented to the samples in the same order as the others. If e.g. group testing is used, this is not possible. In that case a number of groups, where the samples are presented in different order, should be used. Principles of randomising sound samples and test persons may be found in Bibliography [23] and [15]. 9.4 Listening conditions 9.4.1 Bias Normally we want the listening test to be a robust process, i.e. a process minimally affected by external sources of variability. By careful planning of the tests, by selection of relevant assessors, by randomising the presentations of sound samples etc., we try to minimise the bias from these sources. In psychoacoustic testing non-acoustic sources of variance (e.g. visual or tactile cues) are normally regarded as unwanted bias. This is not the case in listening tests for product sound and annoyance potential, where knowledge of the product and the context are necessary premises for obtaining relevant assessments of the sound characteristics. (Part 2, page 13) This implies that it is essential to make it clear what type of test you are planning and what sources of variability you will include as part of the context for the listening test and what sources you will regard as bias and try to minimise. In cases where more sound characteristics are present and/or where multimodal sensory inputs are the normal conditions for perceiving the sound, any combination effects (see definitions 4.1.22-4.1.26) shall be considered and taken into account during planning of the listening test. 9.4.2 Test environment and context The relevance and influence of a specific test environment for a listening test shall be carefully considered in each case. The surroundings and context of the test may be: real world (natural) conditions, simulated conditions, or pure laboratory conditions. In general the test situation and the test environment shall be as similar as possible to the real life situation for using a specific product. Practical limitations may imply deviations from this ideal situation. It may e.g. be necessary to replace a product with pictures of the product. For measurements of annoyance potential of a given environmental situation pictures may be used instead of bringing the test persons to a specific environment. The relevance and the influence from non-acoustical parameters (combined stimuli): visual cues (products, video, pictures) tactile cues, smell, taste, shall be considered. 9.5 Duration of tests It is recommended that the duration of a listening session is not exceeding 2 hours including pauses. The maximum duration of the listening rounds must depend on the complexity of the listening task, but should be less than 20 minutes. Each round should be followed by a pause of approximately the same duration where the test persons can relax from the listening task. Normally no listener should be engaged for more than 2 hours of listening per day. 9.6 Recording and reproduction If the listening tests are not performed with natural (live) sound sources, some considerations about recording and reproduction of the sound shall be made. Recordings with real or artificial heads and torsoes (binaural recordings) should be used for headphone presentations of sound samples. Special care (by calibration) shall be taken to ensure that the reproduced sound is presented at the same sound pressure levels as the original sources. Frequency equalization of the reproduction system is recommended. For headphone listening psychoacoustic calibration may be used. By psychoacoustic calibration the test person(s) equalizes the listening level in the headphones to the listening level of the (same) natural sound source. Investigations imply, see Bibliography [28] that the level in the headphones shall be 3-6 dB louder to give the same intensity impression as from the live source. Especially for headphone listening procedures for protecting the listeners’ ears against unintended high sound levels should be observed. A level limiting system is recommended. NOTE 1 - Real head recordings may have some advantages to recordings with mannequin heads. One of the advantages is recordings in the driver’s seat during handling and operation of the machines. Special care should be taken against unintended head movement during the recording. NOTE 2 - For headphone listening the test persons should be instructed not to move their heads while listening. Alternatively head tracking may be used to obtain stable localisation of the sound sources. (Part 2, page 14) Audible noise caused by electric noise in the measuring set-up and other unwanted background noise sources should be avoided. Care shall be taken to ensure that the system is not overloaded during measurement. 9.7 The sound samples The test stimuli may be real (natural), reproduced (recorded) sound or synthetic sound. The direction and distance to different sound sources at the presentation should be the same as for the real sound. Especially for recordings and synthetic sounds this calls for care and consideration. The length of sound samples shall be sufficient to give a stable, representative impression of the product, process, or sound under test. Durations of a few seconds to 1 or 2 minutes may be used depending on the sound source and the type of test. Durations longer than 2 minutes are seldom necessary or relevant. It is important that continuous sound samples are given soft starts and ends (rise times of e.g. 200500 ms) so that no unintended transient sounds occur. 9.8 Announcements Recorded announcements given in connection with presentation of the sound samples (e.g. announcement of sample numbers) should be recorded at relevant distances and directions and in the same acoustic surroundings as the sound samples in order to avoid confusion or other effects of different acoustics for sound samples and announcements. The announcements should also be presented at a natural and calibrated level. For announcements of high-level sound sources, shouted announcements could be used, and for low-level sources whispered announcements could be used. 10. Running the test sessions 10.1 Introduction to listening test Before the test the test persons should have a short presentation of the test organisation, test leader, purpose, listening conditions, safety, ethics (see Bibliography [25]), their rights, what is expected and that they have the right to terminate their participation whenever they want , the expected duration and pauses, and other practical information. The form of this presentation and the rest of the listening session should be considered. A serious and relaxed form is recommended. 10.2 Instruction of test persons The test persons should receive an aural and/or written instruction that describes: − − − − Test situation Test conditions The sounds and the scenario The test persons’ task Stress that there are no correct answers, that what is wanted is “their answer”. (Part 2, page 15) 10.3 Sample answering sheets and pretest For jury tests sample answering sheets (or an initiated computer) should be given for a pretest. After the pretest there should be an opportunity to ask questions and get answers. Rating sheets are collected, and it is checked that all test persons understand their tasks and the test procedures. Expert listeners have gone through this point during their training. 10.4 Listening test The new answering sheets are distributed or the computer(s) is/are reset, and the listening test is performed as planned. 10.5 Debriefing Experience and unexpected knowledge may be gained by asking for the test persons’ observations and experience after the listening test. Remember to thank the test persons for their help and to give them gifts or a salary or make appointments for sending it. Give some written information about the experiment and the contact person (usually the experimenter) in case the participants have questions or comments later on. 11. Physical measurements Physical measurements are traditionally made by omnidirectional calibrated measuring microphones. Depending on the measurement and the analysis system, measurements and signals recorded with binaural technique may be made by use of a free-field or diffuse-field correction filter. If this possibility does not exist, it is necessary to supplement the binaural recordings with measurements or recordings from a measuring microphone. The equipment applied for the measurement shall be specified in the report. Special equipment shall be described. Consider the influence and state corrections (specify the used HTRF: Head-related transfer function - a free-field transfer function of sound from a certain point to the ear of a subject or mannequin) made for transformation of binaural recordings. 12. Statistical methods Statistical design and analysis of the listening test are important issues, which are not covered in this guideline. Text books, e.g. Bibliography [15] and [27][4], should be consulted. First, a check for missing data should be made. Next, an evaluation of listeners’ qualifications on the basis of the test results and decisions of excluding listeners or outlying results should be made. A general warning of exclusion of results without a specific reason should be given. In general graphical methods will give a good overview of the data. X-Y plots of the mean values and standard deviations of the listeners’ response to the different sound samples or sound characteristics are recommended. Correlation matrices with all data may in some cases give hints about connections between data. Multiple linear regression analysis may be a relevant tool for modelling e.g. the results from the listening tests by the data from the physical measurements. This tool may also give information of the accuracy and significance of results. Both correlation and multiple linear regression analysis is available in spreadsheet programs. (Part 2, page 16) ANOVA-analysis and principal component analysis (PCA) is a more advanced tool used for analysing this kind of data. 13. Accuracy and significance The statistical computations of mean standard deviations and confidence intervals of the results may give information of the accuracy of the results. The influence of working conditions and stability of the sound sources may be estimated by measurements or estimated by experience. Considerations about how representative the tests and the result are of the situation or product under investigation should be made before any generalisation of test results. 14. Information to be reported • State that the measurement has been performed on the basis of this guideline. • Purpose of the measurements and involved parties. • Type of listening test (perceptive or affective). • Descriptions of measuring objects / sound sources and their normal use. • Recording playback and analysis equipment, type, make and model. Special equipment shall be described. • Description of the sound samples (recording technique, editing, announcements, length and pauses). • Criteria for selection of test persons, number of persons. • The method of the listening test, training of the test persons, instructions to test persons, amount of information given to the test persons about the sound sources/situations of use, duration of the listening sessions, etc. • The test conditions (surroundings, non-acoustical cues, individual or group test, etc.). • Results of listening tests, validation of results, test persons, and the statistics employed. Guidelines for both written and graphical presentation of results may be found in Bibliography. • Results of physical measurements. • Any correlation between physical measurement results and the results of the listening test, other types of statistical data processing. • Uncertainty and significance of the main results. Representativity of the results in relation to “nontest” situations. • Photographs of the recording and playback situations are advantageous. 15. Bibliography [1] Bech, S., “Selection and Training of Subjects for Listening Tests on Sound-Reproducing Equipment, J. Audio Eng. Soc. Vol. 40 No. 7/8 1992. [2] Bernsen, J. et al., “Sound in design”, Danish Design Centre & DELTA Acoustics & Vibration, 1999. [3] Blauert, J. & Jekosch, U., “Sound Quality Evaluation – a multilayered problem”, EEA-Tutorium, Antwerp, 1996. (Part 2, page 17) [4] Cohen, J., “Statistical Power Analysis for the Behavioral Sciences”, Lawrence Erlbaum Assoc., ISBN: 0805802835, 1988. [5] Fog, C.L., “Optimal Product Sound: Design and Construction Guidelines for Developing Products with Desirable Sound Characteristics and Minimal Noise”, Report SPM 144 (in Danish), DELTA Acoustics & Vibration, Copenhagen, 1998. [6] Guski, R., “Psychological methods for evaluating sound quality and assessing acoustic information”, Acta Acustica, 83, p. 765-774, 1997. [7] IEC Report 268-13, “Sound system equipment, Listening test on loudspeakers”. [8] ISO/CD2 1996-2, “Acoustics – Description, measurement and assessment of environmental noise – Part 2: Determination of environmental noise levels, Annex C - Objective method for assessing audibility of tones in noise”. [9] ITU Recommendation 562-3,”Subjective Assessment of Sound Quality”, 1990. [10] ITU Recommendation BS.1116-1, “Methods for the Subjective Assessment of Small Impairments in Audio Systems including Multichannel Sound Systems”, 1997. [11] Job, R.F.S., “Noise sensitivity as a factor influencing human reactions to noise”. Noise and Health, 3, p. 57-68, 1999. [12] Järvinen, A. & Maijala, P., “On the use of real head recordings in product sound design”, InterNoise 1997 Proceedings, Vol. II, p. 1143-1146. [13] Maijala, P., “Better binaural recordings using the real human head”, Inter-Noise 2001 Proceedings, Vol. II, p. 1135-1138. [14] Mason R., Ford N., Rumsey F. & de Bruyn B., “Verbal and Non-Verbal Elicitation Techniques in the Subjective Assessment of Spatial Sound Reproduction”, Audio Engineering Society, Preprint, 109th AES Convention, preprint no. 5225, 2000. [15] Montgomery, D.C., “Design and Analysis of Experiments”, Arizona State University, John Wiley & Sons Inc., 2001. [16] Muckel, P., Ensel, L., Chouard, N. & Schulte-Fortkamp, B.: “Sound Descriptions as a Matter of Evaluation in Sound Design”, Proceedings of the 6th International Congress on Sound and Vibration, Kopenhagen, Danemark, 1999. [17] Osgood, C.E., “The nature and measurement of meaning”, Psychological Bulletin, 49, p. 192-237, 1952. [18] Osgood, C.E., Suci, G. & Tannenbaum, P., “The measurement of meaning”, University of Illinois Press: Urbana, 1957. [19] Pedersen, T. Holm, “Objective Method for Assessing the Audibility of Tones in Noise”, Inter-Noise 2000 Proceedings. [20] Pedersen, T. Holm, “Impulsive noise - Objective method for measuring the prominence of impulsive sounds and for adjustment of LAeq, Inter-Noise 2001 Proceedings, p. xx-xx. [21] Pedersen, T. Holm & Fog, C.L., “Optimisation of perceived product quality”, Euronoise, 1998. [22] Porter, N.D. & Berry, B.F., “A study of standard methods for measuring the (product) sound quality of industrial products”, MPL report CIRA(EXT)021, January 1997. [23] Poulsen, T. “Psykoakustiske målemetoder”, (in Danish), Version 3.3, Acoustics Technology, Ørsted•DTU, Technical University of Denmark, 2001. [24] “Publication Manual of the American Psychological Association” (4th edition), American Psychological Association (www.apa.org.), Washington DC. [25] Sales, B.D. & Folkman, S. (Eds.), “Ethics in research with human participants”, American Psychological Association (www.apa.org), Washington DC, 2000. (Part 2, page 18) [26] Sensorisk studiegruppe, “Sensorisk analyse, bedømmelse af næringsmidler” (in Norwegian), Universitetsforlaget, Oslo, 1997. [27] Stone, H. & Sidel, J.L., “Sensory evaluation practices”, Academic Press, 1993. [28] Søndergaard, M., “Psykoakustisk opfattelse af lydtrykniveauer ved anvendelse af hovedtelefoner” (in Danish), AV 1393/01, DELTA Acoustics & Vibration, July 2001. [29] Zwicker, E. & Fastl, H. “Psychoacoustics, Fact and Models”, Springer Verlag, 1999. (Part 2, page 19) 16. Appendix – Psychoacoustics 16.1 Psychoacoustic metrics Psychoacoustic metrics are means of quantifying sound characteristics in ways that correlate well with human sound perception. This section gives a brief overview. Bibliography [29], [19], and [20] should be consulted for more comprehensive and precise definitions. 16.1.1 Loudness (sone) Loudness belongs to the category of intensity sensations of sound. The unit for loudness is Sone. A kHz tone with a sound pressure level of 40 dB re 20 µPa is defined as having a loudness of 1 sone. Signals perceived as being twice as loud have a loudness of 2 sone. In the mid-frequency range and above sound pressure levels of 40 dB, an increase of the level of 10 dB leads to a multiplication of loudness by a factor of two, see Normative references 10. 16.1.2 Loudness level (phon) The loudness level of a given sound, measured in phon, is the sound pressure level of a reference sound, consisting of a sinusoidal plane progressive wave of frequency 1000 Hz coming directly in front of the listener, which is judged by otologically normal persons to be equally loud as the given sound, see Normative references 10 and 11. 16.1.3 Specific loudness (son/bark) The specific loudness is the loudness per critical band of a certain sound. If the sound does not have a low level, it produces a specific loudness in other critical bands than those corresponding to the physical sound. 16.1.4 Sharpness (acum) The sensation of sharpness is produced by high-frequency components in acoustic signals. A narrow band of noise around 1000 Hz with a bandwidth less than 150 Hz and a sound pressure level of 60 dB are said to have a sharpness of 1 acum. Two views on level dependence exist: According to a Zwicker definition sharpness is slightly dependent on the sound pressure level, a level increment from 30 to 90 dB increases sharpness with a factor of two. According to an Aures definition sharpness is almost level independent. 16.1.5 Fluctuation strength (vacil) Fluctuation strength is the sensation associated with amplitude modulation at very low frequencies (usually less than 15 Hz). The sensation reaches its maximum at modulation frequencies of about 4 Hz. Fluctuation strength is not greatly dependent on the sound pressure level of the signal. A pure tone of 1 kHz with a sound pressure level of 60 dB, whose amplitude is modulated at a frequency of 4 Hz and a degree of modulation of 1, is said to have a fluctuation strength of 1 vacil. 16.1.6 Roughness (asper) The sensation of roughness results from amplitude or frequency modulation of a sound at frequencies between 15 Hz and 300 Hz. The maximum roughness sensation is at around 70 Hz. A pure tone of 1 kHz with a sound pressure level of 60 dB, whose amplitude is modulated at a frequency of 70 Hz and a degree of modulation of 1, is said to have a roughness of 1 asper. Roughness is only slightly dependent on the sound pressure level of the signal. 16.1.7 Audible tones The prominence of tones in noise may be characterised by the tonal audibility, ∆Lta, which is a measure for the level of tones in relation to the masking threshold. ∆Lta is defined in Normative references 8 (ISO/CD 1996-2) and Bibliography [19]. (Part 2, page 20) 16.1.8 Prominence of impulses An impulse is defined as the sudden onset of a sound. The prominence of impulses may be characterised by the predicted prominence P. P is a measure, calculated from the onset rate and the level difference, which gives good correlation with listening tests on how prominent impulsive sounds are perceived. P is defined in Bibliography [20]. 16.2 Psychoacoustics Psychoacoustic measurements deviate in a number of aspects from measurements of product sound quality, where other sensory inputs than the acoustical may be essential to the measurement result. In psychoacoustics such cues are normally regarded as unwanted bias. Generally psychoacoustic measurements are characterised as follows: − − − − − − − Determination of the relations between the physical sound and the auditory perception in a reliable and representative way Humans are used as measuring instruments Tests are usually performed in laboratories or clinics Emotionally stable test persons Suppression/control of modifying factors Expectations and emotions related to the sound should be avoided Psychometric measurement methods are used 16.3 Psychometric methods Method of adjustment In this method, the subject has control of the stimulus. Method of tracking In the method of tracking, the subject also controls the stimulus; however, in contrast to the method of adjustment the subject controls only the direction in which a stimulus varies. Magnitude estimation In this method, stimuli are assigned numbers corresponding to the perceived magnitude in some dimension. It is sometimes useful to present a standard, which is called the anchor sound. In this case, pairs of stimuli are presented, and the first stimulus of each pair is kept constant. This standard, or anchor, is assigned a numerical value. Magnitude production In this case, the subject is given a ratio of numbers and has to adjust a second stimulus in such a way that the ratio of psychoacoustical magnitudes corresponds to the ratio of numbers given by the experimenter. Yes-no procedure In this method, the subject has to decide whether a signal was present or not. Two-interval forced choice In this procedure, the subject is presented with two intervals and has to decide whether the signal occurs in the first or second interval. Adaptive procedures The stimulus presented in a trial depends on the answers given by the subject in preceding trials. These procedures are also called “up-down” procedures. Comparison of stimulus pairs If the effects of variations along different stimulus dimensions are to be evaluated, the method of comparison of stimulus pairs has to be used.
© Copyright 2026 Paperzz