3D Audio: aesthetical and technical aspects of
recording and mixing in 3D
Andreas Werner, Prof. Martin Neukom, David Bollinger,
Lars Dölle, Simon Könz
Abstract
Being restricted to stereophonic or two-dimensional formats for creating a surrounding
space is unrewarding. The surrounding space
that gives us the impression of being in a
church, a concert hall or a theatre is missing.
Important characteristics of contemporary
music, contemporary theatre as well as sound
art with a focus on spatial elements can not
be reproduced properly. Also, in new media
like 3D cinema or television, video games and
new applications for the creation of virtual
spaces, appropriate systems for playback
and recording are necessary.
Considering today‘s technical capabilities, it
is very difficult for the sound engineer to find
an appropriate approach for the reproduction
of a specific acoustic situation. Empirical
knowledge is rather diffuse, theoretical
approaches and technical means are rarely
based on clear criteria. An official vocabulary
for describing and qualifying three-dimensional
acoustic events is missing. The intention
of the project is to work out methods for the
reproduction of three-dimensional sound
environments, based on certain defined
criteria. Considering different listening situations, the process of recording, mixing and
playback as well as the reception by the listener
will be examined. In every stage of this research,
the focus is on reproducing spatial properties
with specific characteristics.
The project is focused on recording and mixing.
Developing a systematic vocabulary for threedimensional audio complements the project.
The project draws upon many years of experience from the sound engineering department of the Zurich University of the Arts.
Furthermore, the ICST supports the project
with its expertise in 3D-audio technology and
artistic research. Collaborating with various
orchestras, ensembles, with well-known
composers and sound artists gives the project
a solid base for the production and evaluation
of the recordings. In order to give significant impetus to the development of a modern and
content-oriented practice, a practical research
approach needs to be developed. The intention
is to give sound engineers and artists a reference and model for developing further
technologies.
Listening situation
After evaluating the most common playback
systems for three-dimensional sound, we
decided to use a 5.1 loudspeaker array with
five additional height speakers. These extra
speakers are top left, top right, top left surround,
top right surround and one directly above the
listener, pointing downwards. To facilitate the
comparison of different recording and mixing
ch. 1
ch. 2
ch. 3
...
S
M
S
M
S
M
S
M
mix plugin
mixing and
spacialisation
Legend:
system excuded busses (plugin side chain)
system internal busses
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
S S MM
S S MMM
S
S S MM
S S MMM
S
S
M
S S MM
S S MMM
S
S S MM
S S MMM
S
S
M
Input
1
Input
Input
11
Input
Input
11
Input
1
Input
1
Input
Input
11
Input
Input
11
Input
1
Input
Input
11
Input
Input
11
Input
1
Input
1
Input
Input
11
Input
Send
1 1
input channels
(main and spot microphone signals)
send channels
hold the 3D send plugin
RECEIVE
MIX Plug
S
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
SEND
S S MM
S S MMM
S
S S MM
S S MMM
S
S
M
M
Input
1
Input
Input
11
Input
Input
11
Input
1
Input
1
Input
Input
11
Input
Receive
L1
Mix
mix channel
holds the 3D mix plugin
receive channels
L, R, Ls, Rs, C, LFE, Lh, Rh, Lsh, Rsh and Top
hold the 3D receive plugin
S S MM
S S MMM
S
S S MM
S S MMM
S
S
M
Input
1
Input
Input
11
Input
Input
11
Input
1
Input
1
Input
Input
11
Input
Output
L1
output channel
used for recording descrete,
loudspeaker related audio tracks (bounce)
Panning software
Positioning the sound sources within the
loudspeaker array is done by a special pan
ning software. As for today‘s standard, digital
audio workstations (DAWs) only allow panning
in the horizontal plane. Some tools exist
that allow to overcome this limitation, in order
to pan sources in the vertical dimension as
well. This is done by a plug-in that bypasses
the DAWs standard mixing structure and
replaces parts of it with it‘s own panning and
mixing engine (see figure above).
Prototype recordings
10.1 loudspeaker setup
(ITU 5.1 surround plus hight layer and top-loudspeaker)
techniques, the playback system stays the
same for the whole duration of the project (see
Van Daele und Van Baelen 2012, pages 11-14)
Expert opinions
construction for the 10.0 microphone array, Tonhalle November 2013
signal flow diagram for
three dimensional mixing
within the DAW
There is hardly any literature on 3D-audio.
Therefore, close collaboration with experts
who have practical experience in this field
and are ready to share their thoughts is
essential. Some of the experts we are contacting have more of a technical background,
like sound engineers who have developed
their own methods for recording and mixing
in 3D. We are planning to exchange our recordings with them, as their feedback will
be most valuable to our work. In order to
get as much information from these experts as possible, an interview guideline has been made. Other experts have more
of a musical background: musicians, conductors and sound artists are helpful in judging
recordings on an aesthetic level, especially
if they were involved in the recording. We
will also be collaborating with a composer
who will create a sound installation specifically for us to record in 3D.
For this kind of project a practical approach has
to be taken. Prototype recordings of spatially
interesting content will provide the basis for
the development of recording and mixing techniques. Furthermore a vocabulary to clearly
describe the properties of three-dimensional
sound events will be developed.
The recordings will include:
• Symphony Orchestra in a large hall
• Small Ensemble, including electro-acoustic
events
• Pop music, remix of a stereo
production to 3D
• Singer/narrator who is moving
around the listening position
• Rock-concert (Live-recording)
• Jazz recording, musicians spread
around the listening position
• Recording of a three-dimensional
sound installation
updated: 19/01/2014
[email protected]
Institute for Computer Music and Sound Technology
www.icst.net