Quality Metric for High
Dynamic Range Tone
Mapping
In the past few years, there have been tremendous advancements in high dynamic
range (HDR) imaging, processing, and display technologies. Industry leaders have
also been moving at a fast pace to deliver HDR video content to consumers’ home
and personal devices smoothly. A strong limiting factor in the delivery chain, however, is at the end, where current consumer HDR visualization devices are costly and
cannot reproduce the fascinating HDR visual effects that post-production studios
can. Fortunately, these factors can be mitigated by tone-mapping technologies,
which convert HDR to standard dynamic range (SDR) videos automatically. The
major technical challenge is how to design such tone-mapping operators (TMOs) so
the fine details in the HDR video content are preserved, while also preserving the
“HDR-like” visual experience.
What’s Missing in the Industry?
Many TMOs have been proposed, but they often behave inconsistently across different visual content,
making it difficult for practical systems to pick the best one. What’s missing is the right approach to quantify
the quality of tone-mapped videos. A straightforward solution is human subjective assessment, but it has
several drawbacks. It is cumbersome, slow, costly, and cannot be used for real-time monitoring. It is also
difficult to incorporate into automated design and optimization frameworks to improve TMO performance.
The best solution is to use an automatic objective quality metric that is trustworthy and fast, allowing
tone-mapped videos to be monitored on the fly and compared fairly. It also allows the development of
advanced TMOs to optimize perceptual quality. But, this kind of quality metric is currently missing in the
industry, and designing one is challenging. Standard methods (such as PSNR and SSIM) do not apply
because they can only compare images of the same dynamic range. Fortunately, recent research advancements have shed some light on how to design objective quality metrics of tone-mapped visual content.
SSIMWAVE INC. 295 Hagey Blvd, Waterloo, ON Canada N2L 6R5
http://ssimwave.com
Quality Metric for Tone-Mapped Videos
There are two fundamental aspects that a proper quality metric needs to include – structural
fidelity and statistical naturalness. The first aspect assesses how many of the fine details from the
HDR content are preserved after tone mapping, including the spatial and temporal artifacts generated during the tone-mapping process. The second aspect evaluates the naturalness or appealing
nature of the tone-mapped image, in a statistical sense.
(a) S - 0.9288 (S1 - 0.9371; S2 - 0.9642; S3 - 0.9524; S4 - 0.9158; S5 - 0.8286)
(b) S - 0.7980 (S1 - 0.8419; S2 - 0.8573; S3 - 0.8330; S4 - 0.7795; S5 - 0.6361)
Figure 1.
Two tone-mapped images and their corresponding multi-scale structural fidelity maps
Figure 1 illustrates these factors. On the left, there are two tone-mapped images – each from the
same HDR source, but created by two different TMOs. Most people would agree that the overall
bright appearance of image (b) is better. However, if this is the only method of evaluating image
quality, then the necessity of HDR is questioned, because HDR allows the same image to present
fine details at extremely dark and bright regions simultaneously. Such detail presentation capability
can only be tested by measuring the second factor, structural fidelity. For example, details in the
window regions at the top of picture are washed out in image (b), but are clearly visible in image
(a). Measuring structural fidelity can show exactly where such detailed structures are missing in
the images, as shown in the multi-scale structural fidelity maps to the right of the full image in
Figure 1. These maps, provided by the most advanced TMO quality metric, clearly show where the
details are lost, with darker pixels denoting a more severe structural information loss.
SSIMWAVE INC. 295 Hagey Blvd, Waterloo, ON Canada N2L 6R5
http://ssimwave.com
Quality metric
weighted fusion
Figure 2.
Two tone-mapped images, fused using a quality metric
Figure 2 illustrates a simple example where a quality metric can help to produce an optimal image. It
shows two images that were generated by two TMOs, from the same HDR image. The top image
preserves the details between the tree trunks, while the bottom image exhibits better overall brightness and preserves details in the darker regions. When this type of metric is used to guide a locally-adaptive fusion of the two images, an optimal image is created, with both natural appearance and
fine structural details.
Quality metrics for tone-mapping will continue to play an important role in the future development of
TMO technologies, as well as all associated video acquisition, processing, compression, transmission, and display systems. A handy, fast, and high-precision quality metric can be a powerful tool to
streamline the design of such systems.
For More Information
Extensive research on this topic has been conducted by Professor Zhou Wang’s research laboratory at The University of Waterloo. For more information, see the following publications:
H. Yeganeh and Z. Wang, “Objective quality assessment of tone mapped images,” IEEE
Transactions on Image Processing, vol. 22, no. 2, Feb. 2013.
K. Ma, H. Yeganeh, K. Zeng, and Z. Wang, “High dynamic range image compression by
optimizing tone mapped image quality index,” IEEE Transactions on Image Processing, vol.
24, no. 10, Oct. 2015.
H. Yeganeh, S. Wang, K. Zeng, M. Eisapour, and Z. Wang, “Objective quality assessment of
tone-mapped videos,” IEEE International Conference on Image Processing, Sept. 2016.
SSIMWAVE INC. 295 Hagey Blvd, Waterloo, ON Canada N2L 6R5
http://ssimwave.com
About SSIMWave
SSIMWave Inc., based in Waterloo, Canada, is a leading software and service
provider of video QoE measurement and QoE-driven bandwidth optimization solutions. Developed by world-renowned technology leaders and building upon more
than 15 years of cutting-edge research, SSIMWave supplies content producers,
broadcasters, cable companies, satellite companies, IPTV and OTT video service
providers, network service providers, and technology service providers with powerful software tools for accurate real-time QoE monitoring, QoE analysis, and
QoE-driven video stream optimization. Their customers benefit from award-winning, industry-leading products and services that significantly improve user experience and engagement, as well as reduce bandwidth cost. Leading organizations,
such as the American Society of Cinematographers (ASC), use SSIMWave’s
software tools to create testbeds for the assessment of the preservation of
creative intent during video distributions. For more information about SSIMplus
Analyzer, SSIMplus LiveMonitor, and other tools created by SSIMWave Inc., visit
www.ssimwave.com or email [email protected].
SSIMWAVE INC. 295 Hagey Blvd, Waterloo, ON Canada N2L 6R5
http://ssimwave.com