Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Media madness I:
PARADISEC video trial 2006
Linda Barwick
Sydney Conservatorium of Music
University of Sydney
DELAMAN IV, London 2 November 2006
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
PARADISEC
Collaborative digital research resource set up
by University of Sydney, University of
Melbourne & Australian National University,
2003. (UNE joined 2004)
75% funding from Australian Research Council
LIEF Scheme (3 successful applications)
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
2
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Why video?
 From 1980s, portable battery-powered video
cameras increasingly used by fieldworkers
 Primary data for ethnographic studies of
gesture, dance, performance, signed
language, social interaction
 Highly valued by cultural heritage
communities
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
3
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Threats to video
 Obsolescence of consumer video formats
(VHS, super-8, Hi-8 etc)
 Media deterioration much more advanced that
audio media (estimate 15 years)
 Kept in private research collections rather than
deposited in archives (hence less than ideal
storage conditions, lack of metadata)
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
4
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Deterrents to digitisation
 Lack of internationally agreed archival formats and
standards for digital video, hence researcher fear of
making wrong choice
 Difficulties of storage and management once digitised
- huge filesizes difficult to manage on desktop
computer
 Lack of user-friendly and robust tools to allow
researchers to analyse video data and collaborate
 Ethical and rights obligations - fears of compromising
security and anonymity of consultants by outsourcing
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
5
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Preservation and Digital Sustainability
 the only way to guarantee sustainability of digital
audiovisual media is secure mass storage with
well-structured metadata and an ongoing digital
migration strategy, using authenticated streaming
for access.
Kevin Bradley, APSR Sustainability Issues Discussion Paper, National Library of
Australia, 2005.
International Association of Sound and Audiovisual Archives (IASA) Guidelines on the
Production and Preservation of Digital Audio Objects (IASA-TC04). Aarhus,
Denmark: International Association of Sound and Audiovisual Archives (IASA),
2004.
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
6
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Purposes of trial
 Feasibility study for video ingestion
 select appropriate standards
 test impact on existing systems
 design archival workflows
 model costs for future funding applications
 Preserve significant collections
 Provide access copies for researchers and communities
 Provide test data for EthnoER online annotation projects
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
7
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Equipment and data
 Outsourced ingestion to professional video-
maker (Paul Cockram) using Final Cut Pro 4
on Mac computer, ingesting video data from
professional decks over firewire 400 cable.
 Video formats tested: VHS, Hi-8, miniDV
 Video files saved on 500GB LaCie firewire
drive for transport
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
8
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Formats and standards adopted for trial
 Master copy: .dv standard (maximum quality coming
across firewire into Final Cut Pro) (saved inside
Quicktime .mov wrapper) - approx 15GB per hour
 720x576 pixels, 25 fps, PAL
 Titles: basic metadata plus test signal at beginning
of file, “end file” title at end of file.
 No chapter markers unless depositor provides list
of timecodes for this purpose
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
9
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Access copies
 Also sent to archive because of time taken to process
(will review later)
 DVD copy (MPEG 2) - compress item to approx 4GB.
Save as muxed .mpg file or disk image .img
 Chapter markers inserted every 5 minutes
 Web copy (.mov or .mp4)
 frame size 320 x 240 (480 x 360)
 30 frames per second
 keyframe every 300 frames
 data rate 40 - 50 KB/sec (320 - 400 kbps) or 100 - 120
ckTim e™ a nd a
KB/sec (800 - 960 kbps) for larger frameQuisizes.
2 November 2006
TIFF (Uncompressed ) deco mpre ssor
Barwick - Media madness
are need ed to s ee thi s pictu re.
10
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Issues (1)
 Damage to videos (dropped frames) may cause stall in
ingestion, need to set FCP to tolerate
 Some VHS tapes (1980s) had too many tracking errors to
digitise
 Unstable picture (both hi8 and VHS) leads to higher filesize
in mpeg-2 (image stabiliser needed?)
 Communication issues with contractor led to wrong
filenames being used in titles, hence requiring reprocessing
 Compression to DVD and web copies very slow (8 hours to
process a 3-hour file) , ties up machine - better done inhouse by batch processing
Qui ckTim e™ a nd a
2 November 2006
TIFF (Uncompressed ) deco mpre ssor
Barwick - Media madness
are need ed to s ee thi s pictu re.
11
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Issues (2)
 Data deluge at time of HD shuttle
 500GB needing to be copied onto system - took a day
 swamped storage (ill-prepared)
 server partitions 200GB - not large enough, hence loss
of time in copying across partitions
 Backup scripts at Sydney and APAC became out of
sync, leading to data not being written to tape in
Sydney, or being removed from server before APAC
could mirror
 timeconsuming diagnosis and repair, but systems now
redesigned
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
12
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Transcoding for web delivery
 EthnoER project working with CSIRO’s Annodex
platform to deliver streamed video
 Uses Ogg Theora - best transcoded from full-
resolution version (currently done by CSIRO)
 Online delivery of annotations via EOPAS (EthnoER
online annotation standard) - developed with
University of Queensland Vannotea team building on
Michel Jacobson’s ITE work (LACITO)
 More details at our December conference!
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
13
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Costing and feasibility
 Final report on data trial in preparation - we
will post on our website and blog
 Costing only of production (staff time,
consumables) - model at this stage does not
take into account long-term storage and
networking costs, equipment deterioriation
 Funding application for equipment in 2007
unsuccessful - we will regroup
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
14
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Video data in PARADISEC collection (30/10/06)
Type
Files
.dv
.img
.mov
.mp4
.mpg
TOTAL vid
TOTAL collection
%
23
8
61
6
19
117
21,098
0.555
Size (GB)
Avge (GB)
23.44
1.019
24.750
3.094
485.53
7.960
2.22
0.370
11.92
0.627
547.86
4.683
2909.24
0.138
18.832
 0.5% of collection files account for 19%
of the data volume
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
15
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
are need ed to s ee thi s pictu re.
Thanks to:
Participating Universities
Australian Research Council
Australian Partnership for Advanced Computing
Australian Partnership for Sustainable Repositories
GrangeNet
Further information:
http://paradisec.org.au
http://blogs.usyd.edu.au/elac
http://ethnoer.unimelb.edu.au
http://conferences.arts.usyd.edu.au/index.php?cf=11
2 November 2006
Barwick - Media
Qui ckTim e™ a nd a
TIFF (Uncompressed ) deco mpre ssor
madness
are need ed to s ee thi s pictu re.
16