Approved for public release
Contrast Analysis for DMD-Based IR Scene Projector
Julia Rentz Dupuis, David J. Mansur, Samuel Grant, and Scott P. Newbry
OPTRA, Inc.
461 Boston St., Topsfield, MA 01983
phone: (978) 887-6600 fax: (978) 887-0022
[email protected]
www.optra.com
ABSTRACT
OPTRA has developed a two-band midwave infrared (MWIR) scene projector based on digital micromirror device
(DMD) technology; the projector is intended for training various IR tracking systems that exploit the relative intensities
of two separate MWIR spectral bands. Next generation tracking systems have increasing dynamic range requirements
which current DMD-based projector test equipment is not capable of meeting. While sufficient grayscale digitization
can be achieved with current drive electronics, commensurate contrast is not currently available. It is towards this
opportunity that OPTRA has initiated a dynamic range design improvement effort.
In this paper we present our work towards the measurement and analysis of contrast limiting factors including substrate
scattering, diffraction, and flat state emissivity. We summarize the results of an analytical model which indicates the
largest contributions to background energy in the off state. We present the methodology and results from a series of
breadboard tests designed to characterize these contributions. Finally, we suggest solutions to counter these
contributions.
Key Words: Two-band infrared scene projector, digital micromirror device, contrast
1. INTRODUCTION
Under a U.S. NAVY SBIR Phase II program, OPTRA developed a two-band midwave infrared (MWIR) scene projector
based on digital micromirror device (DMD) technology.1 The system employs two miniature thermal sources, a series of
MWIR lenses and spectral filters, and two DMDs – one for each spectral band. Spectrally separated MWIR images
projected by each DMD are fused and projected onto a unit under test (UUT). This technology was developed as
advanced threat detection test equipment with the unique ability to vary the relative intensity of simulated threats in the
two MWIR bands – loosely called “red” and “blue” – via pulse width modulation of each optical resolution element of
each DMD. The overall system supports realistic simulation of the spectral, spatial, temporal, and radiant intensity
properties of a host of threats for exercising and testing MWIR threat detection systems.
Under the Phase II program a point design was generated and a prototype system was built, integrated, and tested. The
initial application envisioned was flightline testing at a standoff range of about one meter. Figures 1a and 1b are
photographs of the integrated system; Figure 1c shows a simulated and projected image recorded using a FLIR SC6000
indium antimonide camera with a 250 mm focal length lens. The original image was visible grayscale but was projected
in the MWIR by our system. Table 1 gives the performance specifications.
Table 1: Prototype Performance Specifications
QUANTITY
VALUE
Spectral Bands
3.4-4.2 m (blue), 4.2-5.0 m (red)
Apparent Temperature
785 K
Grayscale Resolution
10-bits
Maximum Update Rate at 10-bits
40 Hz
Pixel Count
768 diameter
Maximum Scene Duration
54 s
Angular Resolution
225 rad
Image Registration
One Angular Resolution Element
Contrast Ratio
250:1
Copyright 2012 Society of Photo-Optical Instrumentation Engineers.
This paper was published in The Proceedings of Technologies for Synthetic Environments: Hardware-in-the-Loop Testing XVII
and is made available as an electronic preprint with permission of SPIE. One print or electronic copy may be made for personal
use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any
material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.
Approved for public release
Figure 1a: Two-Band Projector Prototype
(Cover On)
Figure 1b: Two-Band Projector Prototype
(Cover off)
Dichroic Beam
Combiner
Projector
Lenses
Projector
Lenses
“Blue” DMD
Blue
Illumination
Module
“Red” DMD
Power
Supplies
Figure 1c: Projected MWIR Image
In the process of identifying a suitable Navy platform for this promising technology, it became apparent that the dynamic
range of current DMD-based scene projectors (ours included) was limiting their utility for testing advanced IR tracking
technology. More specifically, the Navy is looking for 12-bits of grayscale resolution which applies to not just our
ability to digitize the grayscale range but also the contrast of the scene projector. DMD-based scene projectors in all
wavelength bands can already digitize beyond 16-bits, depending on the required frame rate and integration time,
however, the contrast, particularly in the MWIR spectral region, has been limited to around 250:1 or about 8-bits.
OPTRA was since presented with the opportunity to improve the contrast of our two-band DMD-based scene projector
thereby expanding the testing capabilities of this technology. The following paper summarizes an analytical model
predicting the overall contrast based on eight background light contributions. We also present the results from a series of
breadboard scattering/diffraction measurements. Finally, we present a new design approach motivated by the results of
the model and breadboard measurements.
2.
SUMMARY OF CONTRAST MODEL
The DMD community has two definitions of contrast:
Full-on:Full-off (or FO:FO) which is the ratio of lumens (or watts) projected with all micromirrors turned on
(full white screen) to the lumens projected with all of the micromirrors turned off (full black screen). In general,
Approved for public release
the FO:FO contrast is roughly an order of magnitude below what you could expect by replacing the DMD with a
mirror; this is due to the various device-related factors that will be discussed below.
ANSI checkerboard contrast which is measured by projecting a checkerboard pattern of black and white squares.
This measure of contrast includes the effects of the projection lens(es) and coating.
The model summarized here predicts the FO:FO contrast. The
following sources of background energy which have the effect
of degrading contrast were analyzed. Figure 2 illustrates the
DMD state geometry for reference.
Figure 2: DMD State Geometry
1.
2.
Self emission and reflection of the flat stat,
Source energy diffracted onto the flat state by the
micromirrors,
3. Source energy reflected onto the flat state by the
DMD window,
4. Source energy illuminating the flat state in transit
between on and off states,
5. Scattered source energy from the DMD substrate
(beneath the micromirrors) illuminating the
projector,
6. Source energy diffracted onto the projector by the
micromirrors,
7. Self emission of the DMD chip, and
8. Reflection of the projector optics.
The details of this model have been previously reported.2 Figure 3 shows the relative strengths of the eight background
contributions of our two-band scene projector assuming an IR source temperature of 1023 K and also a flat state
temperature of 233 K (i.e. thermoelectrically [TE]-cooled). With the flat state cooled, the largest contributions are due to
either scattered or diffracted source energy finding its way into the projector. If the flat state is not cooled, this
contribution is the third largest. It was therefore these IR source-related factors that we set out to measure and
subsequently address with the goal of lowering the overall background energy level and increasing the overall contrast.
Figure 3: Comparison of Background Contributions
1 10
8
1 10
9
1 10
Total background power
7
Source energy illuminating flat state in transit
1 10
)
Self emission of flat state
6
Source energy reflected to
flat state by window
1 10
Self emission of projector optics
5
Self emission of DMD
1 10
Source energy diffracted
onto flat state
4
Source energy diffracted into projector
1 10
Scattered source energy
Watts
3
1 10
10
0
2
4
6
8
10
Approved for public release
3. BREADBOARD MEASUREMENTS
3.1 Angular Scatter Patterns
We designed and built an apparatus to measure the angular scatter and diffraction pattern of the DMD in the MWIR.
The setup has the ability to vary the illumination angle in azimuth and the collection angle in both azimuth and elevation.
We also had the ability to implement asymmetrical aperture stops which is an accepted method for reducing the effects
of scattering and increasing the contrast at least in the visible spectral range. We used each of the two-band scene
projector telecentric Koehler illumination modules (in turn) and a chopper immediately following the last illumination
module lens. We used a lead selenide (PbSe) detector without a lens and a lock-in amplifier with the chopper driver as a
reference to recover the modulated signal. The measurement was therefore only sensitive to source-related contributions
(i.e. since the flat state was not modulated, the synchronous detection scheme was not sensitive to it). The angular
resolution of this measurement was limited by the width of the detector and the distance from the DMD: res = dd/R ≈
2/200 = 10 mrad (0.57º). Figures 4a and 4b are photos of the scattering setup.
Figure 4a: 2D Scattering Breadboard – View 1
Illumination
Module
Figure 4b: 2D Scattering Breadboard – View 2
DMD
Az/El
Rotation
Arm
DMD
PbSe
Detecto
Light
Blocking
Baffle
Chopper
Illumination
Rotation
Illumination
Module
Az/El
Rotation
Arm
PbSe
Detector
Using this setup, we measured angular scatter patterns for a series of different illumination angles and also experimented
with asymmetrical aperture stops, which, as reported in the literature3 can reduce the scattered background energy
contribution by on the order of a factor of 2. Figures 5a and 5b show some example data for a 26° illumination angle
using the blue illumination module over a collection range of -20° through +40° in azimuth and 0° through +10° in
elevation; Figure 5c shows the contrast calculated from Figures 5a and 5b. The data shows a reasonably uniform
distribution over the f/3 cone (±9°) although it starts to fall off at 7.5°. The diffraction efficiency is approximately 15%
as quantified by the ratio of the flat state energy (which is the zero diffracted order in this geometry) to the on-state
energy; the flat state / zero order energy distribution is pretty much independent of the micromirror positions as
expected. These observations were consistent with those reported in the literature for visible operation.4 The peak
contrast is located slightly off center towards the flat state which is comparable to what we observed with the single axis
measurements. The contrast is clearly dominated by scattering (as opposed to diffraction). Although the zero order is
present (and is taken into account by the model since it illuminates the flat state), it does not directly illuminate the
projector aperture as will a 1st diffracted order. The ± 1st orders should appear at ± 14º for the blue channel and ± 18º for
the red channel and should be a factor of about 1000× down from the zero order (keeping in mind these are relatively
broad spectral band channels). They are not discernible by our contrast measurement. Overall the data appears pretty
much as expected and supports the analytical predictions.
Contrast measurements including the asymmetric aperture stop also approximately agreed with results published in the
literature (i.e. we measured a factor of ~2× improvement), however, these improvements were insufficient to support the
dynamic range requirement of the next generation two-band scene projector. A more radical solution was required.
Approved for public release
Figure 5a: All Micromirrors On Surface Plot
El. Angle (°)
10
42
38
34
30
26
22
18
14
10
6
2
-2
-6
-10
-14
-18
5
0
Az. Angle (°)
0-100
100-200
200-300
300-400
400-500
500-600
600-700
700-800
10
5
42
38
38
34
30
30
26
26
22
22
188
4
14
0
10
6
6
2
2
-2
-2
-6
-6
-100
-144
0
8
-18
El. Angle (°)
Figure 5b: All Micromirrors Off Surface Plot (log scale)
Az. Angle (°)
0-100
100-200
200-300
300-400
400-500
500-600
600-700
700-800
10
5
42
38
34
30
26
22
18
14
10
6
2
-2
-6
-10
-14
0
-18
El. Angle (°)
Figure 5c: Contrast Surface Plot
Az. Angle (°)
0-50
50-100
100-150
150-200
200-250
250-300
3.2 Flat State Cooling
Additional breadboarding activity included designing the cooled flat state and identifying a suitable paint with high
emissivity over the 3-5 m spectral range. The TE-cooled target is enclosed in a small evacuated chamber with an ARcoated calcium fluoride window. The aluminum target is painted with Rustoleum flat black with a measured emissivity
over 3-5 m of 0.98. The assembly was designed to minimize all radiance contributions (e.g. the window reflecting
ambient energy) to minimize the effective temperature of the target and also ensure that the target cannot ice up (thus the
vacuum). The breadboard assembly is presently in the process of build and test. In the final system a cooled flat state
assembly will be used with each projector DMD.
4. NEW DESIGN FOR HIGH CONTRAST SCENE PROJECTOR
4.1 Structured Illumination Introduction
The take-away from the contrast model is that, given we can sufficiently cool the flat state, the two dominant factors that
limit the overall contrast are due to the IR source: these are the scattering and diffraction of source energy into the
projector aperture. The issue is that we are uniformly illuminating the DMD at all times with a high intensity IR source,
even when we want to project a primarily black scene (i.e. a hot object in a cold sky). A better solution to this problem
would be to have the ability to only illuminate the portions of the DMD from which we intend to project very bright
objects and to effectively not illuminate the portions we want dark. In other words, we need dynamically programmable
structured illumination of the projector DMD. We propose to implement this concept with a source conditioning DMD.
We propose to add the source-conditioning DMD that is conjugate with the projector DMD as shown in Figure 6. The
source-conditioning DMD will be operated in binary mode meaning each micromirror is either in the on or off position
for the full projected frame. By a first approximation, we do not believe we can operate two DMDs in series which are
both grayscaling via PWM because the bit projection is a function of time and is not inherently multiplicative. There
Approved for public release
may be some method for adding a phase delay and rearranging the bit projection order, however, we are not proposing to
investigate whether this can be done at this stage. The source conditioning DMD will realize some form of spatial
grayscaling via a grayscale to black and white conversion scheme such as half-tone, ordered, Stucki, Cardinal
distribution, etc, as illustrated in Figure 7. The combination of the spatial grayscaling with the point spread function
(PSF) of the relay optics which form an image of the source-conditioning DMD onto the projector DMD, coupled with
the extent to which the source conditioning DMD is off of conjugate / out of focus with the projector DMD, will result in
a low frequency illumination pattern that concentrates source energy where very bright objects are being simulated and
extinguishes source energy in regions where we want to project a very dark background (Figure 7). We have filed a
provisional patent describing this approach.
4.2
Optical Layout of Structured Illumination
In the optical configuration shown in Figure 6, both source-conditioning and projector DMDs are operated at 24º, and the
source-conditioning DMD is intentionally illuminated at normal incidence such that the tilt of the on state image is
compensated at the projector DMD where we illuminate at 24º and project normal to the DMD. The layout shown below
should be able to maintain radiometric throughput throughout the system. This layout should also allow for adequate
space for flat state cooling of the projector DMDs. At this point we are not planning for flat state cooling of the source
conditioning DMD.
The optical path is as follows. Source lens 1 (LS1) collects energy from the source at f/1.8 chosen to throughput match
the 6 mm source with the 10.5 mm DMDs which all operate at f/3. LS1 forms a magnified image of the source
(approximately 5×) at source lens 2 (LS2); i.e. LS2 is located at a pupil. The DMD lens (LDMD) then forms a telecentric
field point at the source-conditioning DMD which is located at the focal point of LDMD. The two relay lenses (LR1 and
LR2) telecentrically relay the field image at the source-conditioning DMD onto the projector DMD with a magnification
of 1×. The projector DMD is therefore telecentrically illuminated with the source-conditioning DMD image. We
propose to locate a dichroic beamsplitter (DBS) between the two relay lenses to split the two spectral channels (the
second is not shown but would be directed out of the page and then reflected back into the page with a 24º angle of
incidence by the second projector DMD. The collimating lens, LC then forms another pupil at which point we would
recombine the two colors with a dichroic beam combiner (DBC). The imaging lens (LIM) then forms another telecentric
field point at its focal length. Finally, a field lens (LF) forms the exit pupil as shown.
In practice many of these lenses can be of the same prescription which will help to control the cost – specifically, there is
no reason why LDMD, LR1, LR2, and LC could not be the same lens. With the two channels, this account for six lenses.
These will be approximately 100 mm focal length, f/3 lenses. Similarly, the DBS and the DBC should be the same. The
remaining lenses will each be different, and LIM and LF will be designed based on a 90º field of view requirement as well
as a pupil relief requirement.
Approved for public release
Figure 6: Structured Illumination Optical Layout
The optical path is as follows. LS1 forms a magnified image of the source at LS2; i.e. LS2 is located at a pupil. LDMD then forms a
telecentric field point at the source-conditioning DMD which is located at the focal point of LDMD. LR1 and LR2 telecentrically
relay the field image at the source-conditioning DMD onto the projector DMD with a magnification of 1×. The projector DMD is
therefore telecentrically illuminated with the source-conditioning DMD image. We propose to locate a DBS between LR1 and LR2
to split the two spectral channels. LC then forms another pupil at which point we would recombine the two colors with a DBC.
LIM then forms another telecentric field point at its focal length. Finally, LF forms the exit pupil as shown.
4.3
Simulation of Structured Illumination
A simulation of the image conversion and projection was generated. The simulation process is as follows.
1.) Start with a 12-bit image of the scene to be projected.
2.) Convert the square root of the original image to a binary image using a black and white conversion algorithm
such as Cardinality distribution. The square root is computed because we want to equally distribute the
transmission of the image between the source-conditioning and projector DMDs. This will become apparent
through the math. Assuming uniform illumination of the source DMD, this is effectively what will be projected
by the source DMD.
3.) Simulate the PSF of the imaging optics (LR1 and LR2 in Figure 6) that relay the image of the source conditioning
DMD onto the projector DMD.
4.) Perform a 2D convolution between the binary image and the PSF. The result is the structured illumination of
the projector DMD.
5.) Calculate the image that needs to be displayed by the projector DMD in order to produce the total projected
image, which is the product of the structured illumination and the image displayed by the projector DMD.
Logic is required to negate divide by zero.
a. As part of calculation 5, the image to be displayed by the projector DMD must be digitized to the bitdepth level of the projector DMD. For this we digitized it to 8-bits (i.e. the standard level).
Approved for public release
b.
This image represents the other half of the square root of the original image and should more or less
resemble the structured illumination.
6.) Calculate the total projected image as the product of the (convolved) structured illumination and the image
displayed by the projector DMD.
7.) Calculate an error between the projected image and the original image: error = original/max(original) - total
projected/max(total projected).
Figure 7 illustrates this process.
Figure 7: Structured Illumination Simulation
1.) 12-bit image to be projected:
3.) Lens with this PSF:
Source-Conditioning DMD
2.) Binarized square root of original image
4.) Structured (analog) illumination
of Projector DMD
Illumination Source:
(binary image displayed by 1st DMD)
7.) Error
Projector DMD
6.) Total analog projected image
5.) 8-bit image displayed by
Projector DMD
Based on the results of this model, the original image can clearly be recreated with small error that is limited by the bit
depth of the projector DMD. As will be discussed, the total projected image from this scheme is effectively analog with
no grayscale digitization. This fact releases our system of some of the traditional limits of single-DMD systems
including the restriction between frame rate and grayscale bit depth. In practice, the system will include image
processors to convert the original image stream to both the source-conditioning and projector DMD input streams prior
to the simulation. All DMDs will be precisely frame synchronized.
4.4
Projected Contrast
The contrast model summarized in Section 2 predicts the total background contributions based on an IR source
uniformly illuminating the DMD at all times. Based on our full on:full off contrast definition, with this new
Approved for public release
configuration we will be able to extinguish the IR source light to a power level equivalent to the total predicted
background power. For a 1400 K source temperature, the equivalent temperature of the background energy coming from
the source conditioning DMD when all micromirrors are off is about 450 K. If we then set the apparent source
temperature for all source-dependent background factors to 450 K, Figure 8 shows the projected contrast as a function of
actual source temperature taking into account an extra radiometric efficiency factor of 0.61 for the source-conditioning
DMD.
Figure 8: Projected FO:FO Contrast with Source-Conditioning DMD vs. Source Temperature
6000
Contrast (unitless)
5000
4000
3000
2000
1000
0
400
600
800
1000
1200
1400
1600
Source temperature (K)
As shown in Figure 8, unprecedented contrast values for a MWIR DMD-based scene projector appear to be achievable
with this configuration. As a bright source is simulated by the system, a background level will be created in proportion
to the amount of energy that is let through the system by the source conditioning DMD. To get an idea of this, we
simulated a bright source at 100% gray scale filling a varying percentage of the field and calculated the resulting
reduction in contrast. Figure 9 shows this relationship.
Figure 9: Contrast vs. Percentage (by area) of Field Filled with 100% Grayscale Object
5000
Contrast (unitless)
4000
3000
r
2000
1000
0
0
0.2
0.4
0.6
0.8
Fraction of Field Filled with 100% GS Object
Keeping in mind the types of scenes this system will be projecting, we anticipate a very small portion of the field will be
filled with a 100% grayscale object – possibly as little as a single optical resolution element. In this scenario we would
expect a contrast upwards of 4000:1. These projections are comparable to the ANSI contrast.
Approved for public release
4.5
Effective Projected Bit Depth
In addition to the improved contrast, this approach has a second benefit owing to the fact that we are effectively
illuminating the projector DMD with an analog scene with no grayscale digitization (Figure 7). The projector DMD is
operated with 8-bits of grayscale resolution, however, since it is modulating an analog structured illumination, the
resulting total projected image is analog with no grayscale digitization. This result has some extremely beneficial
ramifications – the greatest of which is a reduction in the integration time required of the UUT in order to experience the
full bit depth of the scene as is the case with a single-DMD based system. Because of the PWM grayscale scheme,
traditional projectors based on a single DMD require the test unit to integrate for the full frame in order to experience the
full bit depth. In our case, there is no restriction as such. This configuration is also free of the restriction between frame
rate and bit depth; we can in theory achieve relatively high frame rates (> 100 Hz) with a large dynamic range
(effectively > 12-bits). The bit depth of the projector DMD does impact the accuracy with which we can recreate the
original image (i.e. the error image). There are also ramifications as to the smallest optical resolution element at the
DMD since the source-conditioning DMD applies spatial grayscaling. If we can compare image accuracy to the
uniformity performance of resistive arrays, the sigma divided by the mean is on the order of 0.5%, which is better than
non-uniformity corrected resistive arrays.
5. SUMMARY AND CONCLUSIONS
In summary, we have presented the results of our analytical contrast model of the DMD-based MWIR scene projector;
presuming a cooled flat state, the largest contrast-limiting factors are IR-source related, including scattered and diffracted
source energy into the projector aperture. Two-dimensional angular scattering / diffraction measurements were
performed and generally supported the literature reporting on visible DMD projectors. Shaped aperture stop solutions
were not sufficient for achieving the required contrast improvements. We therefore generated a new concept based on
structured illumination of the projector DMD where the structure is dynamically provided by a source-conditioning
DMD operating in binary mode.
Our conclusion is that > 12-bit images with commensurate contrast can be projected by this system, even when using the
projector DMD operating in 8-bit mode. In fact, are projecting an analog image of the scene of interest. There will
ultimately be spatial resolution limits on this approach (i.e. the extent to which we will degrade high spatial frequency
information by converting to the binary image and performing the convolution), however, to within reason, the projector
DMD appears to be able to “clean up” the high frequency spatial information. There are a number of ramifications to
these results.
1.) We have engineered the ability to use a DMD to project analog grayscale images.
2.) Since the standard DMD can easily do 8-bits at 100 Hz or faster, we may be able to achieve even higher frame
rates since we may not need all of 8-bits from the projector DMD. The required bit depth of this DMD is tied to
how well we want to clean up the structured illumination image and what error we are willing to tolerate. There
may be other limits as well.
3.) On a related note, there may be ramifications with regard to how long the UUT needs to integrate to experience
the full bit depth of the scene projector if we can run the projector DMD with a lower bit depth than 8 bits. For
example, if it can be done with 7-bits, the UUT will only need to integrate half as long.
4.) This solution offers significant contrast enhancement as the source conditioning DMD is only illuminating
portions of the projector DMD which are projecting bright objects, and in turn only these portions of the
projector DMD are passing the energy through the projector to the UUT.
Acknowledgements
This research was funded under a U.S. Navy SBIR Phase II.5 contract, number N68335-11-C-0182. The Technical
Monitor is Derek Greer, ECSTIM, Naval Air Warfare Center, Aircraft Division, Patuxent River, MD.
Approved for public release
References
1
J.R. Dupuis, D.J. Mansur, R. Vaillancourt, T. Evans, D. Carlson, and E. Schundler, “Two-band DMD-Based Infrared
Scene Simulator,” Proc. SPIE Vol. 7663 (2010).
2
J.R. Dupuis and D. Mansur, “Considerations for DMDs Operating in the Infrared,” Proc. SPIE 8254 (2012).
3
D. Scott Dewald, D.J. Segler, and Steven M. Penn, “A White Paper on the Advances in Contrast Enhancement for
DLP,” June 22, 2002.
4
Chong-Min Chang and Han-Ping Shieh, “Design of illumination and projection optics for projectors with single digital
micromirror devices,” Appl. Opt. Vol. 39, No. 19 (2000).