Timbral Data Sonification from
Parallel Attribute Graphs
Dale Parson, Danielle Emily Hoch,
Hallie Langley - Kutztown University
PACISE 2016, Kutztown University of PA
The Problem
• How to classify data records into one of N distinct
sets of records by turning their attribute values into
properties of sound.
• Inspired by Alfred Inselberg’s Parallel Coordinates
approach to data visualization.
• Inspired by Schedel’s & Yager’s work in timbral
sonification of x-ray scattering data in metals using a
direct approach to mapping application data to
waveforms (that is different from ours).
Parallel Coordinates Plot of
Programming Project Records
Mean Values for 5 Co-varying
Attributes for 3 Classes
Audio Voltage Levels for 2 Classic
Waveforms across Time
http://faculty.kutztown.edu/parson/spring2016/sounds/triangle440.wav
http://faculty.kutztown.edu/parson/spring2016/sounds/sawtooth440.wav
Sweet & Sour Values for <=1, <=2, and > 2
standard deviations from Set 0 Mean
For each record including the set mean records, compute the distance from
the mean of Reference Set 0 as a step function.
Waveform approach maps sweet &
sour distances to sound levels
Waveform sounds from bottom
row of previous slide
• Set 0 reference waveform sound
http://faculty.kutztown.edu/parson/spring2016/sounds/waveform_0_-1.wav
• Set 1 reference waveform sound
http://faculty.kutztown.edu/parson/spring2016/sounds/waveform_1_-1.wav
• Set 2 reference waveform sound
http://faculty.kutztown.edu/parson/spring2016/sounds/waveform_2_-1.wav
Harmonic generates 5 tones as mix
of sweet & sour pitches & timbres.
Harmonic means for each set
• Set 0 reference harmonic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/harmony_0_-1.wav
• Set 1 reference harmonic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/harmony_1_-1.wav
• Set 2 reference harmonic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/harmony_2_-1.wav
Melodic sequences same 5 notes
as harmonic across time.
Melodic means for each set
• Set 0 reference melodic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/melody_0_-1.wav
• Set 1 reference melodic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/melody_1_-1.wav
• Set 2 reference melodic sound
http://faculty.kutztown.edu/parson/spring2016/sounds/melody_2_-1.wav
Responses for fall’15 sonic surveys
Sonification
Harmonic
Harmonic
Harmonic
Harmonic
Melodic
Melodic
Melodic
Melodic
Waveform
Waveform
Waveform
Waveform
Category
All 3 sets
Reference Set 0
Set 1
Set 2
All 3 sets
Reference Set 0
Set 1
Set 2
All 3 sets
Reference Set 0
Set 1
Set 2
Correct responses
55.8%
65.5%
41.4%
60.5%
55.4%
47.5%
50.9%
67.9%
61.4%
74.8%
57.8%
51.5%
Conclusions & Future work
• Waveform is generally better than other two.
• We need to improve Set 1 versus 2 discrimination.
• Currently surveying 3 variants of waveform.
– Two identical waveforms displaced by an octave.
– Two identical waveforms displaced by consonant interval.
– Two identical waveforms displaced by dissonant interval.
• Mapping parallel coordinate plots to high pass
filtered waveforms by mapping distance-from-mean
to waveform inflection points works.
• The approach is data-domain neutral.