Your TV IS TOO SLOW
Dan Nichols
Head of Recording Services
Internet2 Multimedia Specialist
Northern Illinois University
Your TV IS TOO SLOW
Your Everything is too slow…..
- Cameras
- Displays
- Microphones
- Networks
Your etc. IS TOO SLOW
The Human Condition
&
Digital Systems
Your etc. IS TOO SLOW
Audio Comparisons
Nyquist & Pulse Code Modulation (1982)
- Data Driven Compromise
- Not Fault Tolerant
- Fixed Clock
- Good Frequency
Correlation
- Poor Transient Correlation
Your etc. IS TOO SLOW
Visual Comparisons
Your etc. IS TOO SLOW
Observable Latency Effects
- 5ms Unison Impulse Window
221 Samples (44.1 KHz); less than 1/64 Note at 120 BPM
- Video Frame Sync (-125 ms to 45 ms)
ITU Recommendation; Film is +/- 22 ms
Your etc. IS TOO SLOW
Latency Spirals
Positive Latency Spiral:
-
-
Player/Location A Plays Slightly Ahead
Audio Arrives Delayed at Location B (slightly)
Player/Location B Plays Slightly Ahead
Audio Arrives Delayed at Location A (slightly)
If players anticipation is reasonably synchronized with
latency, both will feel ahead but the tempo and feel will
stabilize
If either player is anticipates more than the latency, the
tempo will continue to increase as the players attempt to
maintain relative anticipation
Your etc. IS TOO SLOW
Latency Spirals
Follow the Leader:
- Player/Location A Plays Normally, Leading the Ensemble
- Audio Arrives Delayed at Location B (slightly)
- Player/Location B Plays in Unison with Player/Location A
- Audio Arrives Delayed at Location A (more so)
- Location B Receives Synchronous Musical Experience
- Location A Experiences Consistent (perhaps tolerable) Lag
Your etc. IS TOO SLOW
Latency Spirals
Negative Latency Spiral:
- Player/Location A Plays Normally, In Accordance with
Musical Instincts
- Audio Arrives at Location B Delayed
- Player/Location B Plays Synchronously with Location A
- Audio Arrives at Location A Delayed
- Player/Location A Plays Synchronously with Location B
- Tempo Slows Until Collapse or Stabilization (Dirge)
Your etc. IS TOO SLOW
In the Real World…
Video Delay is Negligible/Nonexistent
Audio Delay is Predictable:
- 1 ms of Acoustic Delay Originates from 1 ft of Distance
- 2.9 ms of Acoustic Delay Originates from 1 m of Distance
- Delays from Reflective Surface Integrate into Direct Sound
(Timbre-dependent) Under 40 ms
- Haas Effect/Law of the First Wavefront
- Everything Else is Considered Reverberation
(Agreeable Delay)
- Delays from Multiple Surfaces Produce Comb Filtering
Your etc. IS TOO SLOW
In Our Networked World…
Sources of Latency Include:
- Ingestion Latency
- Transmission Latency
- Reproduction Latency
Your etc. IS TOO SLOW
Where Does It Originate?
On Camera:
•
•
Exposure Latency
Double the Frame Rate/Halve the Latency
Buffering/Processing Latency
Collecting/Processing Frames into Compressed Output
Audio Interfaces:
•
Audio Buffer Size in Samples
Double the Sample Rate/Halve the Latency
Your etc. IS TOO SLOW
Where Does It Originate?
Ingestion/Reproduction:
•
•
•
•
•
Motherboard Architecture
Software Latency
Compression Latency
Reproduction Display Type
Video Repeaters/Switchers/Converters
Your etc. IS TOO SLOW
Practical, State-of-the-Moment Advice
Microphones Selection & Placement:
• Place Microphones as Close to Sources as Possible
• Consider Analog Dynamic Compression
• Supplement Close Mics with Nearfield/Midfield Mics
• Baffle Reflections to Reduce Echo/Reverb
Speaker Selection and Placement:
• Place Speakers where the Instruments would Naturally be
Staged in an Ensemble or Live Performance
• Use Diffusive Speaker Placement for Diffusive Instruments
& Directional Speaker Placement for Directional Instruments
• Consider Adding Reverb upon Reception of Remote Signal
Your etc. IS TOO SLOW
Practical, State-of-the-Moment Advice
General Audio Considerations:
• Employ Analog Mixers/Analog Audio Inputs
• Use Multichannel Audio for Different Sources
• Replicate Source and Destination Input Levels
• Employ a dB Meter or Spectrograph
• Use the Acoustic Feedback Test to Verify Latency is within
Interactive Thresholds
• Close one or both ends of the audio loop and listen for
Echo-Delay Feedback or conventional Feedback
Your etc. IS TOO SLOW
Practical, State-of-the-Moment Advice
Video Reproduction Considerations:
• Consider Video Reproduction Latency
CRTs – Less than 8 ms
LCDs/LEDs/OLEDs – Greater than 9 ms
Projectors – 16 ms or Greater
• Disable Image Processing
http://www.displaylag.com/display-database/
http://www.projectorreviews.com/the-art-of-gaming/
• Position Low Latency Display in Natural Location
• Face Audience and Performers
• Capture Both Local and Remote Display for Local
Projection/Display
• Repeat Video Signal after Local Reproduction
• Use Multiple Redundant Outputs on Local Machine for High Latency,
Large or Switched Video
Your etc. IS TOO SLOW
Practical, State-of-the-Moment Advice
Video Capture Considerations:
• Use Higher Frame Rates and Software Compression
• Higher Frame Rate can Reduce More Latency
than Compression Adds
• Higher Frame Rates/Shutter Speeds will Require More Light
Your etc. IS TOO SLOW
Development Vectors
•
•
•
Faster, More Robust Networks
Faster Computers
Higher Sample Rate, Fault-Tolerant/Redundant Audio
•
•
•
DSD (22.6 Mbps)
192 KHz/384 KHz
More Nimble Camera Designers
• Custom VTC Machine Vision Cameras
https://www.ximea.com/en/products/oem-custom-cameras
•
PTZ Control
http://www.bhphotovideo.com/c/product/1133719REG/syrp_0032_0001_genie_mini.html
http://www.bhphotovideo.com/c/product/1170815REG/cinetics_cinetics_axis360_pan_and.html/prm/alsVwDtl
http://www.bhphotovideo.com/c/product/1187283REG/cobracrane_ptu_proline_pan_tilt_head.html/prm/alsVwDtl
•
Lower Latency Displays
http://intelextrememasters.com/