Digital Cinema Mastering
Considerations for Higher Frame Rates
Nick Mitchell
V.P. Mastering Services
Technicolor
Creative Services - Digital Cinema
Agenda
• Acronyms, TLAs, FLAs and Important Info
• The Digital Cinema Mastering Workflow
• How Higher Frame Rates changes things
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Important Information
• DCP – Digital Cinema Package
• CPL – Composition Playlist
– XML file within a DCP that defines how a composition is played and uniquely
identifies the version
– Specifies what portion of the track files are to be played
• HFR – High Frame Rate
• MXF – Material Exchange Format
– Container format for media defined by SMPTE
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Important Information
• DCDM – Digital Cinema Distribution Master – Uncompressed
source, specific to DC Mastering:
2K or 4K - 16bit X’Y’Z’ tiff files
24bit/48K Broadcast wav files
DC specific subtitle XML files
Typically delivered as reels
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Ingest
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
DCDM Picture Reels:
~25,000 frames (24 fps @ 2K) =
3D-24 fps ~50,000 frames (48 fps @ 2K) =
3D-48 fps ~100,000 frames (96 fps @ 2K) =
3D-60 fps ~125,000 frames (120 fps @ 2K)=
Distribution
~250GB
~500GB
~1TB
~1.25TB
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Total
~26 TB
Ingest
More data…
2D
~2 TB
3D 24fps
4.5fL ~4 TB
3D 24fps
7fL ~4 TB
HFR 3D
48fps
4.5fL ~8 TB
DCI
Deprecated
HFR 3D
48fps
7fL ~8 TB
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Conform
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
Sync Picture and Sound
Define the entry points and durations of the track files that make up
the composition
Single frame delays are twice as hard to detect in HFR
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Compression/Packaging
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
HFR Stereo – SMPTE DCP only
250Mbps – Defined as the current maximum bitrate for 2K and 4K @24fps
DCI recommendation for HFR Stereo (Near-term) – 500Mbps
Because of the increased temporal resolution HFR, 500Mbps is not enough
for all content – Stress tests will show artifacts are visible at 2K 48fps 2D
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Compression/Packaging
Ingest
Conform
Create
DCP
QC
4X = ~1TB for 4K & HFR
Stereoscopic
Replication
Distribution
~25,000*.j2c files
@ 2K ~20 GBytes
@ 4K ~21 GBytes
Feature DCDM ~25,000*.tiff
files per reel
@ 2K ~ 250 GB
2X = ~500GB for
Stereoscopic
Create
distribution
packages
JPEG2000
Compression
120 min 2K movie:
~120 GBytes
~150GB for Stereoscopic
~450GB for HFR 48fps Stereoscopic
@ 500 Mbps
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Compression/Packaging
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
HFR not much different aside from Frame Rate and Edit Rate
MXF file Creation – Proposed UL for High Frame Rate track files
P-HFR JP2K @ 2k: 06.0E.2B.34.04.01.01.0D.0E.16.02.02.03.01.01.03
Signals a device that the track file contains high bit-rate material
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
QC
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
Requires projection and playback equipment that can support HFR
HFR/High Bit-Rate is a paid upgrade for most devices
HFR is less fatiguing - 2D and 3D
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Creating distribution packages
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
HFR support requires a transition from the JPEG Interop (ad-hoc) to
SMPTE format DCPs
During the transition - in order to support legacy Interop content mixed format (SMPTE and JPEG Interop) media will be required
Changes the architecture of distribution media
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Replication
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
Time required to create and replicate hard drives is heavily
dependent on Bit-Rate
HFR Inventory will need to be manually managed on separate
media until all devices can handle HFR appropriately
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Distribution
Ingest
Conform
Create
DCP
QC
Create
distribution
packages
Replication
Distribution
Distribution of HFR content will need to be managed manually
on separate media until all devices are updated to handle the
content appropriately
Some smaller HDDs may need to be replaced with higher
capacity devices
Satellite distribution of HFR DCP at high bit-rates is challenging
© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Thank you
[email protected]
2012 SMPTE Symposium Program:
High Frame Rates for Digital Cinema
Compression and Distribution Challenges for
High Frame Rate Digital Cinema
Peter Wilson
www.hddc.co.uk
www.edcf.net
Contents
• Sampling Theory - Friend or Foe?
• JPEG 2000
• MTF = Energy
• HFR + Short Shutter = Energy
• Energy is the enemy of compression
• Temporal Axis Distortion in Display
• Converting HFR to LFR
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Sampling Theory - Friend or Foe?
• Sampling was used before the Digital Age
• SDTV is sampled at line rate and frame rate
• Undersampling creates artefacts
• Oversampling promotes quality at a cost
• Spatially content decides over or undersampling
• Temporally content decides over or undersampling
• Nyquist and Shannon were smart
• 24 Fps 85 years of Temporal Undersampling
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
JPEG 2000 (J2K)
• Digital Cinema uses JPEG 2000 Compression
The Free licence version is Part 1
• J2K is a wavelet compression system
- The wavelet system is layered groups of HP LP
- 5 levels max for 2K, 6 levels max for 4K
- This gives us the 2K & 4K compatibility
• J2K is Intra Frame only, each frame is separate
- VBR or variable bit rate allows us to spread data over time but the frames are still separate
More info here: http://www2.engr.arizona.edu/~bilgin/publications/ISCAS2006.pdf
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Stages of a wavelet transform
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
MTF = Energy
• Digital Data Channels have Energy templates
• Energy drops to zero at cut off
• Energy not at zero at cut off will Alias
• Film and Digital Cameras Energy slopes down
• 35 mm energy relatively low at 2K cut off
• New Cameras will blast through to 4K
• Top layer in J2K will be exercised increasing bit rate requirement
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
HFR + Short Shutter = Energy
• HFR improves motion rendition
• LFR is a big compromise on motion v blur
• HFR has less compromise
• Sharper images with less blur generate more energy
• High energy images need more bits
• The next slide illustrates this
• Top image 50 Fps, bottom image 300 Fps
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Effect of a short shutter @ HFR
1/50 s
With thanks to
Richard Salmon
1/300 s
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Energy is the Enemy of Compression
• New generation cameras generate more information
• HFR generates more information
• J2K is limited in future development
• Either bit rates increase or quality metrics need improving
Sharper images at higher frame rates will need proportionally more bits
• Who is the arbiter of quality?
DCPs today are being reduced in size to reduce satellite transmission costs
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Quality Metrics

No need to develop new compression techniques or standards
Existing compressors can do the job, but additional built-in meters and test point links
should be provided

Nearly all important parameters are known internally, but not reported

Built-in meters are much simpler and much more cost-effective
because all necessary data (eg; statistics or list of currently used quantization matrices)
are already there and in the appropriate formats – no need to format these data and
transfer them to remote measurement devices

Availability of additional measurement data
allows more efficient compression in terms of automatic profile optimization and
pre-processor settings control

Most importantly a demand value can best set by the Distributor.
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Temporal Axis Distortion in Display
• Many Cinemas use Single lens 3D systems
- These systems time sequentially display L + R
- These systems use multiple flashing to reduce flicker
- These Systems use multiple flashing to preserve Depth information
• Currently all systems will have difficulty meeting the latest DCI RP
for brightness without doubling up
• Multiple flashing causes temporal axis distortion which results in
Judder and object replication.
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Temporal Axis Distortion in Display
BACKGROUND IS PRESENTED
TO TRACKING EYE IN A
DIFFERENT PLACE
IN EACH FRAME
=
BACKGROUND
STROBING
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Converting HFR to LFR
• For regular release, HFR movies need a LFR version
- Converting from HFR to LFR is not trivial
- Compromising on shutter angle and motion for LFR dilutes the value of HFR
• Making multiples doesn't guarantee a simple conversion (eg; 96 - 24)
- There are often secondary effects
• Global motion blur addition can be highly damaging
- Sharp backgrounds should be preserved while objects in motion are
blurred in proportion to their speed
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
Effect of a short shutter @ HFR
1/50 s
With thanks to
Richard Salmon
1/300 s
© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org
The End
Peter Wilson
www.hddc.co.uk
www.edcf.net
2012 SMPTE Symposium Program: High Frame Rates for Digital Cinema Interna'onal V
ersioning Laurence Claydon www.deluxedigital.co.uk Contents •  Data Management •  DCDM Data Volumes •  Data i/o implica2ons •  Inventory Management •  SubFtling PosiFoning •  Dynamic Z-­‐Axis PosiFoning •  Future for In-­‐Theatre 3D SubFtles © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Data Management •  Moving and Storing Data •  48FPS 2K 3D DCDM is ~8TB •  60FPS 2K 3D DCDM is ~12TB
•  Data Throughput •  48FPS 2K 3D DCDM replay requires disk i/o 7.6Gb/sec •  60FPS 2K 3D DCDM replay requires disk i/o 9.6Gb/sec •  60FPS requires 16Gb/sec facility fibre infrastructure © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Data Management •  Mastering -­‐ Compression •  At Least 500Mb/sec required for 48FPS •  At Least 625Mb/sec required for 60FPS •  Larger DCP Package Sizes •  250-­‐500Gb •  Increased Cloning Time •  Increased Delivery Time via Satellite/Network •  Increased Load-­‐Time in the Cinema © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Inventory Management •  Adds to the MulFple Inventory ConsideraFons •  7.1/5.1/Atmos Audio •  4.5XL/7XL/10XL Light Levels •  24FPS and 48/60FPS Versions •  2D and 3D © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
SubFtle PosiFoning •  Current Hardware Overview HDMI 1.4a Plasma
3D Image Processor
DCP Player
LeT YCzCx 2K SDI Keyers Right YCzCx Network (Control) LTC Background Dual Head
Workstation
LeT SubFtle Right SubFtle Scalers Foreground LeT YCzCx Background Right YCzCx 1920 DVI-­‐2K SDI 1920 DVI-­‐2K SDI Foreground Series II Projector
Dual-DVI or 3G-SDI
Interface
© 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
SubFtle PosiFoning •  Hardware Issues for HFR 3D •  1.5Gb SMPTE 292 Interface Standards will not support HFR 3D •  Only Series-­‐II projectors capable of displaying HFR 3D •  HFR 3D capable replay system expensive (e.g. Clipster) •  Require 3G-­‐SDI capable Scalers/Keyers © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
SubFtle PosiFoning •  SoluFon 1 – PosiFon at 24FPS and Convert… CONSIDERATIONS •  Is the HFR cut same as 24FPS Cut? 24/30FPS •  Do HFR cuts always occur on ‘A-­‐frame’? 48/60FPS 1 2 3 4 A1 B1 A2 B2 A3 B3 A4 B4 •  Trim in points +1 frame, Trim out points -­‐1 frame? •  HFR preview not possible un2l DCP is made BENEFITS •  24/48, 24/60 and 30/60 sub2tle file conversions easily made •  Current 24FPS workflow may be used up to MXF-­‐Wrap stage © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
SubFtle PosiFoning •  SoluFon 2 – True HFR SimulaFon •  Integrate HFR DCP Replay into Worksta2on •  Output Interface Dual DVI or 3G-­‐SDI •  Interim ‘Frame Drop’ mode for connec2on to HDMI 1.4a displays •  48FPS -­‐> output 24P •  60FPS -­‐> output 30P 24/30FPS 1 2 3 4 48/60FPS A1 B1 A2 B2 A3 B3 A4 B4 •  Replay from DCP rather than DCDM to minimise disk i/o and capacity •  Requires dual GPU’s for HFR 3D JPEG2000 Replay © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
SubFtle PosiFoning •  Single WorkstaFon SoluFon OV DCP
(HFR 3D)
HDMI 1.4a Plasma
Create SingleReel DCP’s
(Including
Head/Tail)
Import and convert to HFR
timeline
D-CINEMA XML
(SMPTE 428-7)
24FPS
Series II Projector
Dual-DVI or 3G-SDI
Interface
3D D-CINEMA XML
48 or 60 FPS
PLUS SUBTITLE PNG
GRAPHICS
© 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Dynamic Z-­‐axis PosiFoning •  HFR provides greater temporal resoluFon •  Dynamic posi2oning an increasing requirement due to content •  Integer Pixel values insufficient for smooth Z-­‐axis transi2ons at HFR •  Sub-­‐pixel accuracy required •  Suitable An2-­‐aliasing must be applied to transi2ons © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Dynamic Z-­‐axis PosiFoning •  Dynamic posiFon implementaFon – Rendered SubFtles •  Keyframes provide Z-­‐axis posi2ons in 2meline •  SoXware interpolates posi2on for each frame •  Linear or S-­‐Transi2on •  Output Sub2tle file has 1 sub2tle instance per-­‐frame © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Dynamic Z-­‐axis PosiFoning © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Dynamic Z-­‐axis PosiFoning •  Dynamic posiFon implementaFon – Rendered SubFtles •  Keyframes provide Z-­‐axis posi2ons in 2meline •  SoXware interpolates posi2on for each frame •  Linear or S-­‐move •  Output Sub2tle file has 1 sub2tle instance per-­‐frame •  Z-­‐posi2on must be redefined to 0.001% accuracy for Subpixel Placement •  Sub2tle file TC must be HH:MM:SS:FF and not HH:MM:SS:Ticks © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Dynamic Z-­‐axis PosiFoning •  Dynamic posiFon implementaFon : In-­‐Theatre SubFtles •  Define Keyframes •  XML ‘Keyframe’ Anribute with Z-­‐Posi2on Value and Timecode •  Define TransiFon types •  XML ‘Z-­‐Transi2on’ Anribute with Type e.g. ‘Linear’, ‘Log1’, ‘Log2’ etc •  Projector/Media Block will need to be capable of: •  Posi2onal Interpola2on on Frame-­‐by-­‐Frame basis •  Sub-­‐Pixel Placement •  Generic Sub2tle File for 24/48 and 30/60? © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Summary •  Significant data impact to post producFon and distribuFon •  Inventory needs to be carefully managed against install base •  SubFtling process improvements required •  Current rendered-­‐in soluFon workable •  Further work required to extend SMPTE 429-­‐7 specificaFon •  Further work needed to define in-­‐theatre subFtle render-­‐
engine requirements © 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org
Thank You Laurence Claydon www.bydeluxe.com www.deluxedigital.co.uk