Recording a High-Quality German Speech Database for the Study of Speaker
Personality and Likability
Laura Fernández Gallardo
Quality and Usability Lab, Telekom Innovation Laboratories, TU Berlin, Germany
[email protected]
Abstract
The ongoing process for recording a personality and likability
database in German is motivated and described. Overall, highquality and consistency among recordings is pursued, in order
to avoid possible biases when rating speaker characteristics and
low performance when automatically detecting them. Prescribed and spontaneous human-human dialogs are recorded using
three different microphones in an acoustically-isolated room
with 48 kHz sampling frequency. Natural and neutral speech
is recorded, controlling the absence of background noises. So
far, 101 German speakers without accent have participated and
two listening tests have been conducted with part of these data.
Our goal is to extend the number of recorded participants to at
least 200, conduct more listening tests, and share the speech files, metadata, and associated labels, features, and analyses with
the scientific community.
Index Terms: database recording, voice likability, speaker personality
1. Motivation for a New Database
Speech-based human-machine communications are no longer
constrained to the detection of the users’ message (automatic
speech recognition) and to speech synthesis but also the automatic characterization of the users has been gaining attention
over the last decades. Of particular interest is the detection of
speakers’ social characteristics such as personality and voice likability [1, 2]. Successful automatic personality traits and likability recognition—correlating well with human judgments—
can enable the systematic prediction of social human behavior
and the improvement of speech synthesis for human-machine
interactions.
Generally, client-server architectures are being incorporated in speech-based services, which imply that coded speech
signals are transmitted to a server to process the information
from the signals. However, the influence of different telephone
transmissions on the automatic detection performance of voice
personality and likability has been overlooked. Commonly, the
data are collected through telephone communications or present
a sampling frequency of 8 kHz (see Section 2), filtering out the
possibly relevant high-frequency speech cues. The aim of our
ongoing research is to determine whether and to which extent
transmission channel degradations affect the human and the automatic detection of speaker personality and likability.
To investigate the influence of transmission channels, our
approach is to systematically apply channel degradations to
speech recorded in clean conditions. Therefore, two main requirements for the speech data of our study are 1) clean speech,
that is, recorded through high-quality microphone in quiet conditions, and 2) sampling frequency of at least 16 kHz, needed
for wideband (WB, 50–7,000 Hz) transmissions, from which
possible benefits are to be contrasted to conventional narrowband (NB, 300–3,400 Hz) transmissions. A sampling rate of at
least 32 kHz would be needed for super-wideband (SWB, 50–
14,000 Hz) conditions to be studied. Since no speech database
has been found to meet these requirements (see review in Section 2), a new high-quality database of German speech is being
recorded, and is presented in this paper. The aim is to record the
voices of 200–300 speakers (approx. half of them female) over
high-quality microphones and with 48 kHz sampling frequency.
This database will serve not only for our study of speaker personality and likability but also for any speech-related research
requiring high-quality clean recordings in German.
2. Brief Review of Existing Databases
Unfortunately, the publicly available speech databases employed for personality and likability research seem not to be sufficient for our intended investigation. Objections to these datasets
for our work are that their segments are already transmitted or
have a sampling frequency lower than 16 kHz (SSPNet Speaker
Personality Corpus (SPC) [3], Personable and Intelligent virtual
Agents (PersIA) [4], Electronically Activated Recorder (EAR)
Corpus [5], and Speaker Likability Database (SLD) [2], VoiceClass [6]), that the language is not German (SPC, PersIA,
EAR, AMI, Maptraits), or that the dataset contains a very reduced number of speakers (EmoDB [7]). PhonDat [8] presents
read speech, which may difficult the manifestation of personality traits. No previous study on speaker personality has been
found that employed read speech. Datasets like Smartkom or
other emotional databases (such as VAM, SEMAINE, etc.) contain very little neutral content, which also makes them unsuitable for our research.
Other corpora from the speech recognition and speaker verification areas present one or more drawbacks as well. These
are: different microphones and recording environments (Voxforge), NB telephone quality (Yoho, NIST SRE data, Switchboard, Mixer, Phonebook, Call-home, VeriDat, SpeechDat), too
little material (OLLO), English databases (TIMIT, XM2VTS,
CSR, AusTalk, BANCA, etc.), and non-native speakers of the
recording language (MOBIO).
Differently, the T-Labs Personality Database (TPDB) was
recorded at a sufficient sampling rate (44.1 kHz) and contains
German conversational speech. It was shown to be useful for
the detection of personality characteristics by humans [9]. The
dataset, however, only contains 29 speakers recorded over a table microphone and 35 speakers recorded over a headset and is
hence too small for training and testing regression and classification systems. An approach similar to that used for the recording of TPDB has been adopted in our recording procedure
to elicit spontaneous speech. Both databases could be used to-
gether for research on speech since the recording conditions are
comparable.
It should be noted that the review presented in this section
does not intend to be exhaustive.
A closer look is taken at the SPC [3] and SLD [2] databases,
proposed in the Interspeech Speaker Trait Challenge [10] organized in 2012 (IS12) for the personality and for the likability
sub-challenges, respectively. A total of 640 speech clips of 10 s
or shorter were selected for the SPC database from 322 speakers (263 males and 59 females) of French. The spontaneous
speech enabled the speakers’ personality to be labeled by 11 raters using the Big-Five inventory (BFI)-10 questionnaire [11].
However, the speech data, extracted from transmitted radio signals and sampled at 8 kHz, present channel distortions and low
quality. Besides, the radio speaking style is not conversational,
in contrast to a desirable scenario involving human interactions.
The SLD database is a subset of the larger aGender database [6], recorded over telephone lines presenting different degradations, and sampled at 8 kHz. It contains 800 speakers divided into 6 groups according to their age and gender, and one
sentence per speaker. The sentences, with an average length of
3.05 s, have been used to obtain likability ratings in [2]. Besides the low and variable quality of the recordings, an important
drawback of this database for the study of voice likability is that
the content of the utterances varies across speakers. In [2], the
authors state: ”We’re aware of the fact that the meaning of the
words might affect the perceived likability and it would have
been better to have the same text spoken by all test speakers,
(...)”.
The design of our data collection procedure aimed at embracing the revealed advantages from these databases while
avoiding their drawbacks. The maximum unweighted accuracy (UA) reached by the participants of the IS12 challenge
using SPC and SLD was only UA=69.0% for personality and
UA=65.8% for likability, both in binary classification tasks [12].
This rather modest performance could partly be attributed to some of the mentioned disadvantages.
3. Database Material
3.1. Speech
Different dialogs (human-human) were chosen as recording material, in order to promote a communications scenario, the focus of the future investigations with these data. The contents of
the utterances recorded are all related to information requests
over the telephone. In order to collect the same segments from
all speakers (and thus avoid biases in the later perceptions of
likability [2]), it was proposed that the speakers read dialog
turns maintaining the given wording (prescribed speech). On
the other hand, because personality traits are more easily manifested in non-read speech, the speakers are also asked to hold
free conversations given its structure and information that speakers should provide and collect (spontaneous speech).
A female German speaker of 26 years old assists this project conducting all recording sessions. She was born and raised
in northern Hesse, Germany, and claims not to have a noticeable German accent deviating from the standard High German
dialect.
A recording session is divided into two parts. In the first
part, the speakers are asked to read turns from four different
dialogs. The dialogs simulate telephone calls held with the recording assistant, who always plays the role of a contact person
or agent. The recorded speech, from the client’s side, includes
Tabelle 1: Recording material for our database.
Dialog 1: Health insurance (Table 2)
Dialog 2: Mobile phone rate plan
Dialog 3: Car rental—inquiry
Prescribed speech
Dialog 4: Real estate agency
Dialog 5: Car rental—booking
Dialog 6: Pizza
Dialog 7: Book from the library
Spontaneous speech
Dialog 8: Doctor’s appointment
Tabelle 2: Dialog 1 (prescribed speech). A: agent turns (recording assistant), B: client turns (speaker).
A: Guten Tag, DKS Versicherungen. Mein Name ist
Heinmüller, was kann ich für Sie tun?
B: Guten Tag, mein Name ist Schmidt. Ich hab eine Frage
bezüglich meiner Krankenversicherung.
A: Geben Sie mir bitte Ihre Vertragsnummer durch.
B: Meine Vertragsnummer ist die 4035.
A: Einen Moment bitte. Alles klar, wie lautet Ihre Frage?
B: Ich hab von meinem Hausarzt ein Rezept über 10
Physiotherapie-Anwendungen bekommen. Deshalb wollte ich
mal fragen, ob Sie die Kosten in vollem Umfang übernehmen.
A: Ich verstehe. Ich leite Sie eben zu Ihrem persönlichen
Sachbearbeiter weiter.
B: Alles klar, Dankeschön!
different inquiries about some information: to a health insurance company (Table 2), a mobile telecommunications company,
a car rental company, and a real estate agency, for each dialog,
respectively. The speakers are asked to read the exact given text
as naturally as possible, yet without emotions or exaggerated
friendliness. To the extent possible, it was avoided that the utterances sound like read speech.
While the first recording part comprises prescribed texts,
spontaneous speech is elicited in the second part. Four spontaneous telephone dialogs are held between the speaker and the
recording assistant: renting a car, ordering a pizza, ordering a
book from the library, and making an appointment at the doctor’s. These dialogs follow the scenarios known as Short Conversation Tests, found in the International Telecommunication
Union (ITU)-T Rec. P.805. Again, the participants play the client’s role and the recording assistant the agent’s role. A summary of the recorded speech contents is given in Table 1.
The speakers are instructed to always talk naturally and
neutrally, without hesitations, and with the correct pronunciation (i.e. without accent and not tongue-tied). Special attention
is paid to avoiding any background noise, e.g. papers rustling,
and any disturbance in the speaker’s voice, e.g. croaky or harsh
voice, clearing one’s throat, snickering, etc. Unsatisfactory dialog turns (specially for prescribed speech) are always repeated
until an acceptable turn is uttered.
When 1 s silences are inserted between dialog turns, the
prescribed dialogs have an approximate mean duration of 18 s,
and the spontaneous dialogs of 45 s.
3.2. Metadata
Socio-demographic data about the speakers are systematically
collected when the recording session is completed and stored
as metadata. This information comprises: age, gender, place of
birth, chronological places of residence and duration of stay,
place of birth of the mother, place of birth of the father, highest
education level, educational background, main occupation, past
occupations (if any), years of work experience (if any), and selfassessed personality traits (the speakers are asked to fill in the
BFI-10 questionnaire [11]).
These data, collected from every speaker, may be useful for
future investigations of effects on speech idiosyncrasies. All data are pseudo-anonymized, i.e. the mapping between the speakers’ true names and their speech files and metadata is securely
stored. This is only done for the solely purpose of being able to
delete a speaker’s participation if he/she wishes at any time.
Before the recording session starts, the speakers are asked
to carefully read information about the project and to sign a
consent form. They are informed that:
• their voices will be recorded and that socio-demographic
data will be collected in the session
• the data are stored and pseudo-anonymized for scientific
analyses which can be published
• the data will be treated confidentially and be part of a database that may be publicly released under license agreement for exclusive use in scientific studies
• the participation is voluntary and it is possible to withdraw from the study at any time without penalty
Any question they may have is answered and the participants only proceed to the recording session if they agree with
the consent form. They keep a copy of the signed form and of
the project’s information sheet with our contact details.
4. Speakers’ Recruitment
Most of the speakers who participate in the recording process
are recruited via the ’Probandenportal’ of the Technische Universität (TU) Berlin, mailing lists and paper announcements.
The requisites for the participation are:
• Native German speaker without dialect or accent
• 20–35 years old
• No hearing or speech disorders
• No sickness or cold at the time of the recording
Special effort has been made to only recruit speakers of the
standard High German dialect, since pronunciation has been
shown to have an important effect on the impressions of voice likability. It has been reported that listeners seem to prefer to
hear voices with their own accent [13] or with no accent [14].
Some persons who register to participate and attend the agreed
time slot but from whom a marked accent can be perceived by
the conductor of the recordings are, regrettably, not accepted for
the participation.
The age range was restricted to young adults in order to
avoid inconsistencies in likability or personality ratings caused
by age differences. Using the SLD data [2], it was found that
senior speakers (55–80 years old) were rated as less likable than
adult speakers (25–54 years old).
The duration of the recording session is approximately
30 minutes and the speakers are compensated with 6 e.
5. Recording Setup
The recording sessions are conducted in the acousticallyisolated room Nautilus, at the TEL-building of the TU Berlin.
The room’s dimensions are 2.75 m x 2.53 m x 2.10 m, and RT60
= 0.08 s at 2 kHz. The speaker sits alone in this room, where
three microphones are placed. These microphones are:
Abbildung 1: Room Nautilus with the recording setup. In the
position of a speaker is Laura Fernández Gallardo.
• Stand-up microphone: AKG C 414B-XLS (frequency
range 20–20,000 Hz)
• Table microphone: Audio-Technica U851R (frequency
range 30–20,000 Hz)
• Headset microphone: Sennheiser HMD 46 (frequency
range 100–12,000 Hz)
Figure 1 shows the recording setup. The approximate distances from each microphone to the speaker’s mouth are
35 cm, 50 cm, and 3 cm for the stand-up, table, and headset
microphone, respectively.
The three signals corresponding to each microphone are
amplified and then digitalized employing the RME Fireface
UCX Audio Interface, conducted to the outside of the room
via a USB port, and recorded using the software Cubase 4 with
48 kHz sampling frequency and 32-bit quantization. The recording assistant sits outside of the Nautilus room, from which she
can listen to the speaker, give the pertinent instructions, and
hold the dialogs. She employs headsets also conducted to the
interior of the room, to the speaker’s headset.
6. Listening Tests conducted with the
Recorded Data
So far, two listening tests have been conducted using part of the
recorded speakers and the signals acquired through the stand-up
microphone.
6.1. Likability and Personality-based Social Relations
The first listening test was part of our study presented in [15]. A
group of 30 speakers of our database were asked to participate
in a listening test where the voices from the group were played,
following a round-robin design. Hence, the participants mutually rated each another in terms of voice likability and personality. These ratings were collected by means of continuous scales
with labels at their ends. Speech signals transmitted through a
narrowband channel (codec G.711 at 64 kbit/s), through a wideband channel (codec G.722 at 64 kbit/s), and clean speech
(sampled at 44.1 kHz) were used as stimuli. The Social Relations Model (SRM) [16] was then employed as a statistical
approach to analyze the mutual perceptions.
In short, it was found and reported in [15] that persons
which are perceived as extroverted and agreeable are also rated
with a higher likability. In addition, people similar in agreeableness and neuroticism tend to rate each other’s voice likability more positively. WB voices, with respect to NB, were si-
gnificantly higher rated in terms of likability on average, led
to lower variance among perceivers’ rating tendencies, and allowed listeners to better distinguish between non-likable and
likable speech. It could also be shown, using a reduced set of
acoustic features, that a better model to predict the likability ratings could be built using WB speech, compared to that using
NB speech.
6.2. Paired-Comparison Listening Test
A second listening test [17] was conducted with partly the same speech stimuli and the same listeners as the experiment
in [15]. Instead of using a continuous scale as in [17], a pairedcomparison listening test was proposed for obtaining listeners’
likability ratings. The intention was to detect and assess possible advantages of this method with respect to the direct scaling
test, which typically contributes to undesirable low agreement
between judges. In the paired-comparison task, the rater was
presented with voice stimuli in pairs and was asked each time
to determine which of each voice is preferred and to which extent, i.e. how much likable is the preferred utterance over the
other.
It could be asserted that paired-comparison constitutes a reliable method for voice likability assessment while facilitating
simple comparative judgments. The Bradley-Terry-Luce (BTL)
probabilistic choice model [18, 19] could be successfully applied and ratio scale preference measures were derived. These
preference measures correlated with R2 = 0.90 with those obtained by the direct scaling test. While the direct scaling test
leads to a somewhat lesser agreement between raters, it may still
be generally preferred over the paired-comparison approach, given the considerable test length of the later.
7. Future Directions
A total of 101 speakers (47 males, 54 females) have already
been recorded as of June 27th , 2016. Their mean age is 26.3
years with standard deviation 3.76. Out of the 101 participants,
68 are students, 20 are employed, and 13 gave other answers:
’Auszubildender’, ’Arbeitssuchende’, etc.
Our intention is to reach the recording of at least 200 speakers, which may suffice for training and testing automatic classification or regression systems in the frame of our project. It
will be controlled that about half of them are female. The next
step after completing the recordings is to label all data in clean
conditions via listening tests regarding personality and likability. Then, part of the data (the speakers presenting extreme likability values and personality traits), will be transmitted through
communication channels and the effects of the distortions on the
subjective ratings will be analyzed. Furthermore, the influence
of the channel degradations on the performance of automatic
systems will be investigated.
We are very open to various forms of collaborations. Once
our data recording is accomplished, they will be freely released
to the scientific community under a license agreement. Also the
results derived from these data such as raw scores, speech features, or other analyses can be shared with interested researchers.
8. Acknowledgements
The database recording and its associated studies are supported
by the German Research Foundation (DFG, Grant FE 1603/1-1
to Laura Fernández Gallardo).
9. Bibliographie
[1] A. Vinciarelli and G. Mohammadi, “A Survey of Personality
Computing,” IEEE Transactions on Affective Computing, vol. 5,
no. 3, pp. 273–291, 2014.
[2] F. Burkhardt, B. Schuller, B. Weiss, and F. Weninger, “’Would
you Buy a Car From Me?’ – On the Likability of Telephone Voices,” in Interspeech, 2011, pp. 1557–1560.
[3] G. Mohammadi and A. Vinciarelli, “Automatic Personality Perception: Prediction of Trait Attribution Based on Prosodic Features,” IEEE Transactions on Affective Computing, vol. 3, no. 3, pp.
273–284, 2012.
[4] A. V. Ivanov, G. Riccardi, A. J. Sporka, and J. Franc, “Recognition of Personality Traits from Human Spoken Conversations,” in
Interspeech, 2011, pp. 1549–1552.
[5] M. Mehl, S. Gosling, and J. Pennebaker, “Personality in Its Natural Habitat: Manifestations and Implicit Folk Theories of Personality in Daily Life,” Journal of Personality and Social Psychology,
vol. 90, no. 5, pp. 862–877, 2006.
[6] F. Burkhardt, M. Eckert, W. Johannsen, and J. Stegmann, “A Database of Age and Gender Annotated Telephone Speech,” in Language Resources Evaluation Conference (LREC), 2010, pp. 1562–
1565.
[7] F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and
B. Weiss, “A Database of German Emotional Speech,” in Interspeech, 2005, pp. 1517–1520.
[8] F. Schiel and A. Baumann, “Phondat 1, corpus v. 3.4.” Bavarian
Archive for Speech Signals (BAS), Tech. Rep., 2006.
[9] T. Polzehl, “Personality in Speech - Assessment and Automatic
Classification,” Ph.D. dissertation, Technische Universität Berlin,
Germany, 2014.
[10] B. Schuller, S. Steidl, and A. Batliner, “The INTERSPEECH 2012
Speaker Trait Challenge,” in Interspeech, 2012, pp. 254–257.
[11] B. Rammstedt and O. P. John, “Measuring Personality in One Minute or Less: A 10-Item Short Version of the Big Five Inventory in
English and German,” Journal of Research in Personality, vol. 41,
no. 1, pp. 203–212, 2007.
[12] B. Schuller, S. Steidl, A. Batliner, E. Nöth, A. Vinciarelli, F. Burkhardt, R. van Son, F. Weninger, F. Eyben, T. Bocklet, G. Mohammadi, and B. Weiss, “A Survey on Perceived Speaker Traits:
Personality, Likability, Pathology, and the First Challenge,” Computer Speech & Language, vol. 29, no. 1, pp. 100–131, 2015.
[13] N. Dahlbäck, Q. Wang, C. Nass, and J. Alwin, “Similarity is More
Important than Expertise: Accent Effects in Speech Interfaces,” in
Conference on Human Factors in Computing Systems, 2007, pp.
1553–1556.
[14] B. Weiss and F. Burkhardt, “Is ’Not Bad’ Good Enough? Aspects
of Unknown Voices’ Likability,” in Interspeech, 2012, pp. 510–
513.
[15] L. Fernández Gallardo and B. Weiss, “Speech Likability and
Personality-based Social Relations: A Round-Robin Analysis
over Communication Channels,” in Interspeech, 2016.
[16] D. A. Kenny, Interpersonal Perception: A Social Relations Analysis. New York, U. S.: Guilford Press, 1994.
[17] L. Fernández Gallardo, “A Paired-Comparison Listening Test for
Collecting Voice Likability Scores,” in Informationstechnische
Gesellschaft im VDE (ITG) Conference on Speech Communication, 2016.
[18] R. A. Bradley and M. E. Terry, “Rank Analysis of Incomplete
Block Designs: I. The Method of Paired Comparisons,” Biometrika, vol. 39, no. 3/4, pp. 324–345, 1952.
[19] R. D. Luce, Individual Choice Behavior: A Theoretical Analysis.
New York, USA: Wiley, 1959.