WP4 – Sound Object Identification
WP5 – Enriched Access Tools
Workpackage 5 - Tasks
WP5 Partner
Role
DIT (Lead)
- Pitch / timescale modification, - Source Separation**,
- EQ, - Noise Reduction.
QMUL
- Looping and marking, - Video TSM,
NICE
- Speech enhancement tools, - subjective quality tools
Total 80 PM
**developed in WP4 but ultimately an access tool also.
18 Month Biannual
Technical Meeting
Deliverables and Milestones
Month
1
You Are
Here
D1
5
T5.1 Looping and Marking
10
D5.1 M9
100% Complete
15
D2
20
25
30
i
D5.2 M27
90% Complete
T5.2 Time and Pitch Scale Modification
D4.2 M14
90% Complete
T4.2 Sound Source Separation
x
i
M5.2 Initial prototype integrated with interfaces (Month 18)
18 Month Biannual
Technical Meeting
i
D5.3 M30
60% Complete
T5.4 Sound Enhancement & NR
5
i
Video + Audio TSM
90% Complete
T5.3 Multimedia Alignment
1
D3
10
15
20
25
= integrated
30
x
EASAIER HOST APP.
Real-time engine integration
Real-time Engine
Load Frame
Real-time Access Tools
Source Separation
Interpolate
Audio
PSM & TSM
factor
UI control
Window + FFT
Noise Reduction
Is Transient ?
Yes
i
Update Phases
18 Month Biannual
Technical Meeting
IFFT + window
IFFT + window
75% RT Overlap
75% RT Overlap
++ SynHop x TSM
++ SynHop x 4
Clock Feedback
Equalisation
Workpackage Progress
 TSM/PSM – successfully integrated and operational
 Video synchronised TSM integrated and operational
TSM
i
VTSM
18 Month Biannual
Technical Meeting
Further Modification
 Link option for time and
pitch scaling
 Pitch change via
transposition (requires
Key sig estimate)
 Time change via tempo
change (requires tempo
estimate)
18 Month Biannual
Technical Meeting
Workpackage Progress
Sound Source Separation
 Source separation tool is fully developed and ready for
integration
Problem: DIT IP regulations will not
allow code to be open source
Solution: DLL for windows release and
shared library for Linux release
18 Month Biannual
Technical Meeting
Workpackage Progress
Equalisation / Enhancement
 Equalisation tool is fully developed and ready for
integration
The equalisation tool can be used to filter out tones and
hums which are often present in archival formats such as
magnetic tape.
Alternately, the tool can be simply used to tailor the sound
to a listeners taste and playback system
18 Month Biannual
Technical Meeting
Workpackage Progress
Equalisation / Enhancement
• Simple mode interaction uses 5
controls to increase or decrease
the level of 5 fixed bands
• Advanced mode allows user to
view the spectrum and draw the
desired curve onto spectrum
• Demo to follow
• Integration – Mid December 07’
18 Month Biannual
Technical Meeting
Upcoming Work Plan Months 19-30
• Integrate EQ module
• Integrate Source Separation
• Work is ongoing w.r.t. “intelligent noise reduction” algorithms
• Evaluation / Benchmarking / User testing.
• Journal Publication planned for VTSM
18 Month Biannual
Technical Meeting
Demonstrations
1. Integrated pitch and timescale modification
2. Integrated video/audio time and pitch scale modification
3. Prototype restoration EQ and simple EQ
18 Month Biannual
Technical Meeting
Questions ?
Discussion
18 Month Biannual
Technical Meeting