Mixing Translucent Polygons with Volumes
Kevin Kreeger and Arie Kaufman.
Center for Visud Computing (CVC)
and Department of Computer Science
State University of New York at Stony Brook
Stony Brook, NY 11794-4400
Figure 1: Min’ng polygons with mukiple volumes: A scalpel (containing 368 polygons) cutting into a human head (from MRI) while
visualizing blood vessels (from an angiogram): (a) translucentpolygons reveal the blood vessels behind the scalpel, (b) opaque rendering
obscurespotentially dangerous movements (also in color plate).
Abstract
scalpel. Using simple opaque polygons as in Figure l b obscures
the vessels very close to the scalpel. During pre-operative surgical
practice, the doctor can use translucent polygons as in Figure l a
to see the potentially dangerous areas otherwise hidden from the
given viewpoint. Many first person LBE systems place the user behind the avatar or vehicle (e.g., F-15) that the user is controlling.
Unfortunately, this obscures a portion of the view. Translucent rendering (as in Figure 2) allows the user to see the entire field of view
right through the F-15 that hdshe is piloting. Another application,
shown in Figure 3, is a realistic image of steam emanating from
boiling tea inside a glass teapot heated on an electric range. A second medical example (Figure 5) allows an orthopedic surgeon to
pre-operatively determine which manufacturer’s prosthesis best fits
the patient. Kaufman et al. [5] showed that translucent polygons increase the surgeon’s ability to visualize prosthesis placement. Kaufman et al., however, rendered only surfaces from the volume data
into a Z-buffer and then blended the translucent polygons with the
Z-buffered volume image.
The rendering of opaque polygons with volumetric data is fairly
straightforward, using current Z-buffer methods. Therefore, we emphasize the much more difficult problem of embedding translucent
polygons within the extents of volumetric data. Translucent polygons complicate the situation because all fragments (both translucent polygon pixels and volume samples) must be drawn in topologically depth sorted order. This is required because compositing
translucent fragments with the over operator [14] is not commutative. Therefore, polygons must be re-depth-sorted whenever the
scene or viewing geometry changes. Additionally, the sorting must
be topologically correct, including the handling of depth cycles.
We present an algorithm which renders opaque andor translucent
polygons embedded within volumetric data The processing occurs
such thai all objects are composited in the correct ordet by rendering thin slabs of the translucentpolygons between volume slices
using slice-order volume rendering. We implemented our algorithm
with OpenGL on current general-purpose graphics systems. We discuss our system implementation, speed and image quality, as well
as the renderings of several mixed scenes.
Keywords: Mixing polygons and volumes, Translucent Polygon
Rendering, Volume rendering, Ray casting, Voxelization
1 Introduction
Many interactive applications such as medical systems, flight simulators, virtual environments, and location based entertainment
(LBE) systems could benefit from the improved image quality and
enhanced visual reality of mixing volumetric data with both opaque
and translucent polygons at interactive to real-time rates. For example, Figure 1 shows how a surgeon could use such a system to
interactively practice an operation. In this example, multiple volumes are rendered in one scene while still being able to embed
polygons. Since our algorithm works with standard OpenGL, many
extensions such as this multiple volume example are possible. The
Angiogram of the blood vessels show potential danger spots for the
* {klrreeger,ari}@cs.sunysb.edu
191
Figure 2: Zkanslucent rendering permits viewing the entire field
of view when the user is behind the avatadvehicle (e.g., F-15 containing 2300poIygons) (also in color plate).
Figure 4: A translucent helicopter (containing 2709 polygons)
spinning in a cloud over a textured terrain (also in color plate).
Figure 3: Steam emanating from boiling tea inside a glass teapot
(containing 3408 polygons) heated on an electric range (also in
color plate).
Figure 5: A prosthesis (containing 3758 polygons) beingfit to a
256' CT scan of a hip (also in color plate).
Presently, no hardware solutions exist to render translucent polygons correctly even without mixing them with a volume. Although
the A-buffer algorithm [2] sorts correctly, it has never been implemented in hardware because it relies on a dynamically allocated
list of depth sorted fragments for each pixel. The proposed hardware implementation of the A-buffer [6] allows only some constant
maximum depth complexity (number of overlapping polygons per
pixel) in a single pass. Hence, the sorting is typically left to the
application program on the CPU using techniques such as octrees,
k-d trees and, most popularly, Binary Space Partition (BSP)trees.
However, building the tree is a costly operation. Handling just a
few thousand polygons takes over 1 second, even when taking advantage of frame-to-frame coherence to limit the computation to the
affected subtrees [18].
An attempt to create a standard for mixing polygons and volumes
were the OpenGL extensions proposed in the HP Voxelator [ 101. In
their theoretical model, volumes and polygons were rendered by
separate pipelines and the resulting images merged in the frame
buffer using Z-depths for occlusion. This limited the proposal to
opaque polygons and opaque volume projections. More recently,
the SGI Volurnizer [3] proposed a method to handle volumes using
the polygon pipeline and 3D texture mapping hardware. Opaque
polygons could be mixed with either opaque or translucent volume
projections by utilizing Z-depth testing.
Several software systems have been used to generate high quality
images with many differing rendering modalities, including mixing
translucent polygons and volumes. However, these .systems sacrifice frame rate for image quality and flexibility by rendering completely in software. While well suited for production applications,
lack of interactivity leaves these systems unsuited for many other
applications. These software systems commonly utilize ray tracing
to render images with mixtures of translucent polygons and volumes [9, 161. Although ray-tracing produces visually pleasing images, it is very slow and requires that the volume data be rendered in
image order. Unfortunately, there are no volume rendering accelerators available to perform high quality image order ray tracing.
Vizard I1 [ 111 proposes to perform image order volume rendering,
but relies on lossy'compression of the gradients and barely achieves
interactive frame rates.
An alternative to rendering volumes by image order ray tracing
is to perform object order ray casting. Although both methods perform the correct depth sorting of volume samples, image order ray
tracing utilizes memory bandwidth very inefficiently because of the
I92
almost random access to volume data Object order algorithms,
on the other hand, traverse the input data in storage order (e.g.,
shear warp [8], splatting [21], or slice order ray casting [12, 13]),
thus accelerating the rendering process due to the memory access
coherence. One of the more common object order methods is to
process the volume in slice order most perpendicular to the ray direction [12, 131. Since all rays are processed in lock-step, memory is required to store the partially composited results. Often, as
in 3D texture map based volume rendering [3], the actual frame
buffer itself is used. Various solutions have been developed which
provide interactive to real-time frame rates for volume rendering
at various image qualities, all utilizing object order processing to
exploit regular memory access patterns. The solutions range from
low quality approximations on PCs at barely interactive rates (e.g.,
Voxar), through interactive rates on high end SGI workstations, to
the highest quality rendering at 30Hz frame rates on specialized
volume rendering accelerators (e.g., the new VolumePro PCI card
from Mitsubishi Electric/Real Time Visualization [ 121 and the UCube ultrasound visualization system from Japan Radio Co. both based on Cube-4, developed at Stony Brook [13] and to become commercially available in 1999). The algorithm we present
here also works at frame rates approaching 30Hz on the Cube-5
hybrid rendering architecture [7].
In this paper we employ a 3D texture map based volume rendering approach which allows us to render both opaque and translucent polygons with translucent volume data at interactive rates on a
high end workstation. We utilize preshaded RGBa or Luminancea textures so that we can achieve high quality images. In theory our methods will work with any texture map based volume
rendering approach (e.g., Behrens and Raterings approach to add
shadows [l]). In Section 2, we overview previous work on mixing opaque polygons with volumes on OpenGL systems and discuss some anti-aliasing issues that we discovered. In Section 3,
we describe our method to mix translucent polygons with volumes
in volume object order and describe our method of bucketing the
polygons into thin slabs which fit between the volume slices. Additionally, we show examples from our OpenGL implementation on
a high-end 3D graphics system. Section 4 touches on some of the
limitations of this method that we discovered along the way.
Rand/
I
Fragment Operations
I
Acce
Figure 6: Graphics acceleraforsolution, where volume slices compete with polygons for limited resources at the texture memory
andframe buffer memory interfaces.
2 Mixing Opaque Polygons with Volumes
Rendering mixtures of opaque polygons and volumetric data is
straightforward on any system with a Z-buffer. The use of a Zbuffer algorithm to solve the hidden surface visibility of polygons
can be easily extended to handle visibility of volume samples. The
opaque polygons are drawn first. Subsequently, the volumetric samples, either opaque or translucent, are composited over the pixels
using any volume rendering algorithm. The depth values of the
opaque polygons are checked to keep volume samples which are
hidden by opaque polygons from contributing to the final image.
On OpenGL systems which include 3D texture mapping support, the volume samples are rendered in a slice by slice method
by texture mapping polygons parallel to the image plane [3]. The
polygons represent a (usually) planar set of samples, one for each
pixel on the screen (ray cast through the volume). An overview of
this system is shown in Figure 6.
High end OpenGL implementations are optimized for frame
buffer operations. However, on lower end graphics accelerators the
Z-buffer write operation should be turned off during volume rendering with glDepthMask(). Doing this avoids unnecessary writes
to frame buffer memory.
When anti-aliased polygons are drawn on a high end workstation, it is best to utilize the glmultisampling-sgis. The alternative,
gl-polygon-smooth, creates anti-aliased edges by computing an alpha value for the percent coverage of each fragment and blending
Figure 7: An opaque polygon mixed with a gray translucent volume: (a) no anti-aliasing, (b) OpenGL G l ~ o ~ g o n m t o o t(c)
h,
glmultisamplesgis extension.
into the frame buffer. However, only one color and one depth value
is computed per fragment, and the polygons must be rendered in
front-to-back order. Since the polygons are rendered before the volume, the edges are blended with the background color instead of the
volume (see Figure 7b). On the other hand, glmultisamplingsgis,
performs rasterization to a higher resolution, storing depths, colors,
etc. for each sub-pixel. This way, the correct depth of each subpixel can be used to composite volume samples into each sub-pixel.
Before display, hardware averages the sub-pixels for a smooth antialiased image (see Figure 7c).
In Figure 7, on the right (inside the white circle), one can see
how both methods create pleasing anti-aliased edges when there
is no volume behind the polygon. However, one can see on the left
(inside the black circle) how the gl-polygon-smooth edge is blended
with only the background color. The glmultisampling-sgis triangle
has edge pixels properly blended with the volume. Infinite Reality
193
Parallel
Polygon
Slab
0
Volume
Sample
Plane
I
I
I
I
I 0 l 2 1
I
I
I
I
I
I
iI
Figure 8: Side view of dove-tailing translucentpolygons and volume data. (The gaps between polygon slabs are shownfor clarity.
I n reality, there is no overlap or gap as shown by the boundary
intervals.)
Figure 9: Clipping triangles to thin slab boundaries may create
more triangles. There are four possibilities when clipping a hiangle to a thin slab defined by two parallel planes.
frame buffer operations are optimized to perform multisampling at
no cost to the frame rate. We have verified this on four different
scenes at screen resolution up to 1280x 1024.
3
Mixing Translucent Polygons with Volumes
Rendering of volumetric datasets is a more demanding task than
rendering polygons. Additionally, the only methods to achieve interactive or real-time frame rates for volume rendering utilize object order ray casting. Therefore, in our method presented here,
we adapt polygon rendering to slice order ray casting, and synchronize the overall rendering process on a volume slice-by-slice basis
instead of a polygon-by-polygon basis or pixel-by-pixel basis.
In our method, each slice of the volume is sampled in planes
perpendicular to the image plane. The planes are drawn in depth
order from farthest from the eye to closest to the eye. Therefore,
to mix translucent polygons with volumetric data, thin slabs of the
polygons are rendered and composited in between the slices of the
volume samples, as shown in Figure 8. In this 2D side view example, the rectangles represent all of the translucent objects which lay
between two consecutive slices of the volume sample planes. The
boundaries of the slabs should be created such that the union of
all the slabs neither miss nor duplicate any region, such as less-than
the current slice and greater-than-or-equal-to the next slice (see slab
boundaries in Figure 8). The data from the volume slices and the
translucent polygonal slabs are dove-tailed together in an alternating fashion. In this way the correct depth ordering of all contributing entities is preserved and use of the over operator to composite
them creates correct colors in the final image pixels.
In our method, the opaque polygons are drawn first with Zbuffering. Before drawing any volume slices, the translucent polygons which lie behind the volume extent are drawn over the opaque
polygons using any translucent polygon rendering algorithm (e.g.,
painters algorithm). Polygons which lie depth-wise within the
volume extent, but abovehelowlleftkight of the volume, are then
drawn in slice order as if the volume slices were planes that extended to infinity cutting the translucent polygons.Finally, translucent polygons which lie in front of the volume are drawn.
OpenGL can be used to directly render the thin slabs of translucent polygonal objects. The polygons are shaded using the Gouraud
shading model included in OpenGL. A naive approach would be to
render the complete set of translucent polygons for every slab and
set the hither and yon clip planes to cut out the current thin slab of
data. Yet, for an n3 volume, there could be up to n thin slabs that
Figure 10: Bucketing triungles results in 12 triangles being sent
to the graphics pipeline.
must be rendered. Since a typical scene contains very few polygons which span all of the thin slabs, an alternative would be to clip
the polygons to the slab boundaries and only render the portions of
the polygons within each slab. This would substantially reduce the
processing load on the polygon pipeline. However, it would require
the application to clip every polygon against the two planes of each
thin slab which contains that polygon.
Figure 9 demonstrates clipping triangles to thin slab boundaries
and that we can take advantage of the fact that the two boundary
planes are parallel to only keep the portions of the polygons which
lie between the planes. The clipping process may create more triangles. There are four possibilities. The first case occurs when a
triangle intersects the thin slab, but no vertices are within the slab
boundaries. When this occurs, one vertex must be on one side of
the slab and the other two on the other side, creating a trapezoid
which is decomposed into two triangles. Next, consider when there
is one vertex within the slab. In one situation, the remaining two
vertices lay on the same side of the current slab creating only one
triangle. The second situation is when the other two triangle vertices lay on opposite sides of the slab. This is the worst condition
because it produces a pentagon, or three triangles. The final case is
when two vertices lie within the slab, and once again a trapezoid is
created resulting in two triangles. While this creates fewer polygons
than clipping against each plane separately, it still can dramatically
increase the triangle count.
Instead, we bucket sort the translucent polygons. We create a
bucket for each thin slab between two volume sample planes. We
traverse all the translucent polygons in the scene and place each
194
. ..-/' .
--
_-e-
0'
0
"
20
"
40
"
60
"
Frame Number
80
Teapot
"
100
'
1
120
01
0
I
50
Slice Count
100
150
(4
(b)
Figure 11: Normalized polygon count as a function of the: (a) fiames from the test sequences - teapot and helicopter are rotated 5
degrees perfiame, F-ISJlying through a cloud, and hip i s a prosthesis alignment search sequence, (b) number of volume slices for a
single frame in each test sequence.
grees, these polygons span many buckets and cause multiple polygons to be drawn. At 90 and 270 degrees, most polygons are perpendicular to the view direction and parallel to the slab boundaries.
This causes most polygons to be drawn only within one slab.
In the teapot scene, the viewer walks 360 degrees around the
teapot. Since volume slices are always drawn perpendicular to the
viewing direction, this has the same effect as spinning the model
360 degrees.
Finally, the F-15 sequence is a fly-over a terrain with a cloud
that the plane flys directly though during frames 42 - 81. Since the
plane does not intersect the cloud except for this area, the number
of polygons drawn is exactly the same as the number in the model.
Only when the two objects intersect do some polygons get sent to
the graphics pipeline multiple times. For our four test sequences,
we see that the number of polygons drawn is about 2-3.5 times the
original number of polygons in the models.
An attractive feature of our algorithm is that for applications
which choose to trade-off image quality to keep a certain frame
rate, the number of polygons drawn decreases as the number of
slices drawn for the volume also decreases. This occurs because
the inter-slice size increases as the number of volume slices decreases. Figure l l b shows how the number of polygons to be drawn
decreases linearly when the slice count is reduced. The rendering
rate achieved is proportional to the number of polygons drawn and
the number of volume samples drawn (which is proportional to the
number of volume slices drawn). The image quality degradation
resulting from this trade-off affects only the volume data similar
to taking fewer samples in a volume rendering algorithm. On the
other hand, the polygonal objects appear exactly the same, since the
inter-slice boundaries are invisible.
Previously, we (Silva et al. [15]) and later Westermann and
Ertl [ 191 suggested to accelerate rendering by bucketing polygons
according to which scanlines they would contribute to in the final
image. Bucketing polygons in this way allowed visibility ordering
for each scanline to be accelerated by a two pass method using standard graphics hardware. Our method differs in that we bucket the
polygons according to Z-depth.
one in a bucket for each slab it intersects. For example, in Figure 10 triangle T1 is placed in all six slab buckets since it spans
all of the slabs. Triangle T2 is placed in buckets S2 and S3, and
likewise for the rest. While many polygons are rendered multiple
times, there are still fewer polygons drawn than if they are clipped
to the slab boundaries. For example, bucketing the four triangles
from Figure 10 would result in 12 triangles being sent to the graphics pipeline. If these triangles were clipped to the slab boundaries,
as in Figure 9, 20 triangles would be sent to the graphics pipeline.
The hither and yon clipping planes automatically cut the polygons
early in the polygon processing pipeline, thus reducing fill rate requirements to be the same as if the polygons were truly clipped to
the slab boundaries. Another advantage of bucketing is that it is
a much less demanding process on the application program than
clipping.
An alternative to bucketing is to create an active triangle list
similar to the active edge list utilized in scan converting polygons.
The triangles can be placed in the active list at the first slice where
they intersect, and removed when they no longer intersect any more
slices. A data structure can be pre-computed which indicates which
slice each triangle is first encountered. This preprocessing is the
same as for bucketing. In fact, the only differences between bucketing and active triangle lists is that bucketing does not have to check
for triangle removal at each slice, but does use a larger data structure.
We analyzed the effectiveness of polygon bucketing using sequences of four test scenes: the hip prosthesis, the F-15 plane, the
teapot, and the helicopter. The scalpel example in Figure 1 was
not used since the multiple volume scenario significantly affects
the overall rendering erformance. In each of the four test cases,
our images were 300 and we rendered 100 volume slices. The
number of polygons drawn per frame was normalized to the number of polygons in the model. The normalized polygon counts are
shown in Figure 1la. The hip sequence is 92 images from an interactive search for the correct prosthesis placement. In frames 0 to
20, the model is rotated to match the alignment of the hip bones.
The number of polygons drawn has a large variation in this region
because different orientations cause the polygons to span a different number of slabs. From frames 20 through 75 the user translated
the hip around at the same orientation. The perfectly flat regions in
the graph represent the model moving in the X-Ydirections only;
therefore, the polygons remain in the same slab buckets. The regions of small variations appear when the model moves in the Zdirection and polygons enter and leave slab boundaries. In frames
75-92 the model returned to its original orientation.
In the helicopter sequence, the model is rotated 5 degrees per
frame through 360 degrees. The helicopter consists of many triangles which span the length of the helicopter body. At 0 and 180 de-
f
4 Handling Multiple Polygons Within One
Slab
There exist two options for handling multiple translucent polygons
being drawn to the same pixel within one thin slab. In the first option, the polygons could be drawn in the order they are stored in
the display list using normal back-to-front compositing. While this
method may produce the incorrect color, the amount of color error is limited since the polygons are translucent and are still sorted
I95
shows the time it takes to bucket-and-sort polygons for the test sequences. The bucketing-only times were never more than 0.03 second and the median was 0.01 second. For most cases, sorting the
polygons inside each bucket is proportional to the number of polygons in each bucket. For example, the hip has the most polygons
(polygon counts in Figure 11 are normalized to the actual model
count). Our sorting algorithm contains a quick bounding box extent pre-check. For certain frames in the teapot and hip sequences,
there are fewer polygons drawn, but the sorting still takes longer.
This is due to the percentage of polygons which fail the pre-check
and must be precisely checked for depth cycles. Udortunately, for
some cases skipping the sorting creates incorrect images. For example, for some frames of our teapot sequence, the polygons defining the tea lie within the same thin slab as the glass teapot walls.
However, this occurs very rarely. In fact, out of 37K pixels drawn
for the tea and teapot per frame, a maximum of 63 pixels blended
tea polygons with teapot polygons. Without sorting, approximately
half of these would be composited in the wrong order since the
teapot completely surrounds the tea.
In the F-15 sequence, the plane stays in the same geometrical
configuration with the viewpoint. Therefore, we take advantage of
this fact and pre-sort the polygons. Even a naive bucketing algorithm will leave the polygons sorted when the plane intersects the
cloud.
An application where polygon ordering never creates an incorrect color may take advantage of the speedup of only bucketing the
polygons. For example, in the hip application in Figure 5, the polygons which have different colors are always separated by more than
the slab thickness. Consequently, the incorrect color is never generated no matter how the polygonal model is aligned with the volume
slices. For our helicopter sequence, we also measured no incorrect
color blending operations. While this is not an argument for every
application to skip the topological sorting step, we notice that many
applications can benefit from the fact that the bucketing has done a
“good” enough sorting job to create acceptable image quality.
1
0.5
0
40
60
Frame Number
20
80
700
Figure 12: Application processing time for bucketing-and-sorting
for the test sequences. (Bucketing-only times were never greater
than 0.03 sec; the median was 0.01 sec.)
by bucketing them into the thin slabs. Additionally, the amount
of this error can be computed as follows. Consider compositing
two polygons over the previously computed ray at a given frame
buffer pixel using the OpenGL blending operations. Using back-tofront compositing, each color channel is blended with the OpenGL
glBlendFunc(gl-srcalpha,gl-onerninus-srcalpha). For each color
channel, the first polygon has color Cpl and alpha apland likewise
for the second polygon. The ray currently has color CR. There
are two cases, composite polygon 1 before polygon 2 and composite polygon 2 before polygon 1. If polygon 1 is composited first
followed by polygon 2, the resulting color for the ray after some
algebraic manipulation is:
c
= CR
+
cplapl
-CR(Ypl
+
CpZap2
+
5 Implementation
CRaplap2
- CRapZ - C p l a p l a p Z
Our algorithm has been implemented in OpenGL on existing hardware by using 3D texture map hardware to achieve interactive volume rendering. Applications begin by rendering all opaque polygons with Z-buffering and multisampling if it is available. Then,
all the translucent polygons behind the volume data are drawn using a method which creates the desired quality (e.g., painter’s algorithm). Subsequently, mixing the translucent polygons with the
volume data is dealt 8jth in the following manner. Slices of volume
data are drawn one at a time from back to front by projecting a polygon for that slice and texture mapping with 3D texture extensions.
The volume slices are Z-depth checked and blended on top of the
kame buffer. In between each volume slice, the translucent polygons in the bucket for the current thin slab are drawn, once again
with Z-buffer checking and blending. The volume slices and polygon thin slabs are alternated until the end of the volume is reached.
Finally, any translucent polygons which are in front of the volume
data are drawn.
Figures 2 , 3 , 4 , and 5 are frames from test sequences generated
on an Onyx2 with Infinite Reality graphics. We rendered the scenes
with 100 slices of the volume into a 300’ image. The helicopter
frames were rendered at 7.6Hz and the teapot images at 12.5Hz.
The F-15 sequence produced between 9 and 50Hz. Since the viewer
flys through the volume in this sequence, the size of the volume on
the screen changes dramatically and greatly affects the time it takes
to render. Finally, the hip sequence produced frames at 4.8Hz. The
hip dataset is textured using RGBa texels, each taking 32 bits to
store. This also greatly affects the rendering rate since the Infinite
Reality texture memory system is optimized for 16 bit texels as we
Similarly, the resulting color for compositing polygon 2 first is:
c
=
CR
+
cplapl
-CRapl
+
Cp2ap2
+
cRaplap2
- (?Rap2 - CpZ(Yplap2
Notice that the only difference in these two equations is the last
term. In fact if we subtract the second equation from the first, we
get the difference being:
diff = ( a b e ( C p 2
- cpi))
apiap2
Thus, we can say that the amount of error from this method depends
on the difference in color of the two polygons and the opacity of
each. So, if two translucent polygons will be drawn to the same
pixel in a thin slab and have the same color, there will be no error in
the resulting ray no matter in what order they are drawn. Also, if the
colors differ, we can compute the resulting color error by using the
opacities of each. This measure could be used for the application to
decide whether to utilize the slower, but more accurate, method we
are about to introduce.
The second option to handle drawing multiple translucent polygons to the same pixel within one thin slab is also the more accurate
one. By utilizing one of the methods mentioned earlier to depth
sort, such as a BSP tree, proper ordering of all translucent polygons within each slab is rnaintained.Unfortunately, correct topological sorting is a very expensive task. For example, Figure 12
196
discuss in Section 7. We also rendered the hip with 250 volume
slices to produce higher quality structures in the volume at about
2Hz.
Many volumes contain large portions of empty space. Techniques used in a system such as SGl Volumizercull out this empty
space, but require knowledge of the visible portions of the dataset.
For example, with the teapot scene we pre-processed the steam volume to create tighter geometric boundaries around the steam. By
using these polygons to texture map the volume into the frame
buffer, we saved on the frame buffer fill rate. Using the tight geometric boundaries, the teapot scene was rendered in 0.07 seconds
resulting in 14.3Hz instead of 12.5Hz frame rate when texturing the
entire volume into the scene including empty samples - an increase,
but still in the same order of magnitude.
I
I
I
I
I
,
Attenuation
Error
Distance
.......+
Plane of
Polygon
Volume
slices
I
6 Comparison to Voxelization
I
I
I
I
I
I
I
(a)
(b)
Figure 13: (a) Side view of error within a thin slab of polygons.
The attenuation from the volume slice directly in.front of each
polygon fragment to t h d fragment is not considered While this
is a small amount the discontinuous color resulting on theface of
the polygon is distracting to the human eye. (b) Example image
of apolygon inside a homogeneousgel.
In current voxelization methods [17], when a surface is 3D scan
converted into a 3D volume grid, the resolution of the grid is commonly chosen such that the size of a single voxel represents the
smallest area that can be discerned by the human eye when it is
rendered. In the z and y dimensions the polygons are drawn to
screen resolution. In the z dimension, assume that the volume is
being rendered with sufficient slice resolution so that each volume
sample also represents the smallest area that can be discerned by the
human eye. Therefore, each pixel bounded by two volume slices in
the z dimension also represents this small area. Considering this,
our method can be viewed as voxelizing on-the-fly by utilizing 3D
graphics hardware.
This approach has been utilized by Fang and Chen [4] to voxelize
CSG trees after they have been converted to triangle meshes. Fang
and Chen considered two surfaces occupying the same voxel as the
union of two voxelized objects. Fuzzy set theory contains differing methods for computing the union of multivalued-objects. Fang
and Chen used max blending, where each of the RGBa channels
is computed as the max of the two polygons for that channel. This
can create incorrect colors when two solid light reflecting objects
occupy the same voxel. Also, with surface modeled objects, just
because two polygons are used to model the object does not mean
that they represent two different objects. Unfortunately, while this
method is faster than software voxelization, it is still too slow to
be called interactive. This is because, to be performed correctly,
the thin slabs of polygons must be first max-blended into an auxiliary buffer. Then, this slab can be either composited into the image
or stored off as a volume slice. Therefore, for every slice of the
volume, there is an additional glClear and an image-sized glcopyPixels. This increases the pixel fill-rate since even blank pixels are
drawn into the frame buffer. Fang and Chen report 0.1 to 3 seconds to voxelize 1283volumes. Our implementation of max blending for on-the-fly voxelizatiodrendering of the F-15 scene took 0.4
seconds to render per frame, four times longer than our bucketing
version.
An alternative formulation which has been utilized for voxelization [ 171 is to take the weighted-average as follows
A last possible method is to pre-voxelize the translucent polygons and then render multiple volumes. Our method has the following advantages. Firstly, our method produces a significantly
lower fill rate into the frame buffer than rendering a complete volume for the translucent polygons. The fill rate advantage can be as
low as 50% of the voxelizdmultiple-volume solution. Even if the
translucent polygons are pre-voxelized, rendering their voxelized
slices will create many frame buffer operations with blank fragments. Secondly, if we perform depth sorting of the buckets, we can
gain viewpoint independence with correct color. With voxelization,
the max blending does not create correct color and choosing an arbitrary order ties the voxelized representation to a given viewpoint.
Lastly, our method allows polygonal objects to be dynamic without
requiring re-voxelization.
7 Discussion
As with most hardware acceleration, our method utilizes some approximations. Using the slice order mixing approach, the amount
of light absorbed by the volume material is discretized to the level
of each volume slice. Therefore, any polygon fragment which falls
between two consecutive slices of the volume data receives the exact same amount of attenuation. While this error is very small, the
discontinuity between regions of the polygon which fall between
different volume slices might be distracting to the human eye. Rgure 13 shows a side view of this error. With software ray-tracing,
the exact polygon surface intersection is computed eliminating this
error. This effect is not noticeable in any of our example images.
We experimented with different volumes at different resolutions,
and were only able to see the effect on perfectly homogeneous volumetric “gels” (see Figure 13b). Any volume which contains any
kind of heterogeneity hides this minimal error. For a perfectly homogeneous gel, it would be better to render the region using glfog
which, in addition to being faster, also creates more continuous attenuation levels.
Unfortunately, there are two drawbacks to using texture mapping
hardware for volume rendering which affect some applications.
First, the texture mapping hardware cannot accurately shaddlight
each sample of the volume such as supported in specialized volume rendering engines. Recent proposals [20] to perform shading
+ Cpzapz)
c, = (Cplcypl
(%l + %Z)
where C, is the color assigned to the voxel. We are able to
weight the colors by their alpha before blending, by using glBlendFunc(glsrcalpha, gl-one). However, OpenGL is not able to compute the weighted-average equation since it can not divide by the
sum of the alpha values after accumulating them. The OpenGL
scale and bias factors multiply every pixel with some constant
value, not by a value computed inside that pixel.
197
Acknowledgments
only work for approximating isosurface renderings and only perform diffuse, not specular, shading. While some applications can
use pre-shaded volumes or do not need highly accurate shading,
others require it. For our examples of clouds, we pre-shaded the
volumes and rendered Luminance-a textures. This approximation
is valid since most flight simulation applications are lit by the sun
which does not move quickly compared to cloud objects. For the
hip scene we pre-shaded the volume into an RGBa representation.
While in this virtual simulation environment the light may move,
high quality shading provides more intuitive understanding of the
structures inside the data. This understanding is essential for this
application and therefore can not be ignored.
Secondly, the 3D texture map approach to volume rendering does
not achieve true 3OHz frame rates except for the highest end graphics workstations. The per-vertex geometry calculations for the volume slices is easily achievable with any level graphics hardware.
However, since these systems are designed for rendering polygons,
they must support random access to both the texture memory and
frame buffer (see Figure 6). Therefore, memory bandwidth can not
be optimized for the regular order that is utilized when rendering
slices of the volume. For example, an Infinite Reality graphics engines with 4 raster manager boards is reported to place 710 million
16 bit textured, depth buffered fragments into the frame buffer per
second. However, with only one board (a common configuration
since they are the most expensive part), the fill rate quickly drops to
177 million fragments per second. In our tests we were only able
to achieve up to 90 million fragments per second fill rate, below the
published numbers since they do not include blending. Additionally, our hip example with 32 bit RGBa textures performed even
slower due to increased demand on the interpolation units in the
texture mapping subsystem. However, the increased quality and
accuracy provided by the 32 bit textures is required by the application.
We have previously presented a hybrid volume/polygon rendering architecture [7] for the Cube-5 system which utilizes objectorder ray casting to achieve 3OHz rendering rates. The algorithm
presented in this paper can be used in this system to render the
translucent polygons in between the volume slices for faster, higher
quality images.
This work was supported by the National Science Foundation under grant MIp9527694 and Office of Naval Research under grant
N000149710402. The authors would like to thank previous reviewers for pointing out some of the SGI OpenGL extensions that made
this a more complete work.
References
[I] U. Behrens and R. Ratering. Adding Shadows to a Textux-Based Volume Renderer,. In Symposium on Volume Visualization,pages 3946, Oct. 1998.
[2] L. Carpenter. The A-Buffer, an Antialiased Hidden Surface Method. In Computer Graphics, SIGGRAPH 84, pages 103-109, July 1984.
[3] G. Eckel and R. Grzeszczuk. OpenGL VolumizerProgrammer’s Guide. Silicon
Graphics, Inc. Mountain View, CA, 1998.
[4] S . Fang and H. Chen. Hardware Accelerated Voxelization. In Proceedings of
International Workshop on Volume Graphics 1999, pages 185-203, Swansea,
UK,Mar. 1999.
[5] A. Kaufman, R. Yagel, and D.Cohen. Intermixing Surface and Volume Rendering. In K. H. Hohne, H. Fuchs, and S . M. Pizer, editors, 3 0 Imaging in Medicine:
Algorithms, Systems, Applications, pages 217-227. Springer Verlag, June 1990.
[6] M. Kelley, K. Gould, B. Pease, S. Winner, and A. Yen. Hardware Accelerated
Rendering of CSG and Transparency. In Computer Graphics, SIGGRAPH 94,
pages 177-184, July 1994.
[7] K. h e g e r and A. Kaufman. Hybrid Volume and Polygon Rendering with Cube
Hardware. In Proceedings of SIGGRAPH/EurographicsWorkshopon Graphics
Hardware, Aug. 1999.
[SI P. Lacmute and M. b o y . Fast Volume Rendering using a Shear-warp Factorization ot the Viewing Transform. In Computer Graphics, SIGGRAPH 94, pages
45 1457, July 1994.
[9] M. Levoy. A Hybrid Ray Tracer for Rendering Polygon and Volume Data. IEEE
Computer Graphics &Applications, 10(2):3340, Mar. 1990.
[lo] B. Lichtenbelt. Design of A High Performance Volume Visualization System.
In Proceedings of SIGGRAPH/EurographicsWorkshopon Graphics Hadware,
pages 111-120, Aug. 1997.
[I 11 M. Meissner, U. Kanus, and W. Strasser. VIZARD II,A PCI-Card for Real-Time
Volume Rendering. In proceedings of SIGGRAPH/Eumgraphics Workshop on
Graphics Hardware. pages 61-68, Aug. 1998.
1121 H. Pfmter, J. Hardenbergh, J. Knittel, H. Lauer, and L. Seiler. The VolumePm
Real-Time Ray-Casting System. In Computer Graphics, SIGGRAPH 99, Aug.
1999.
8 Conclusions
[13] H. Pfister and A. Kaufman. Cube-4 - A Scalable Architecture for Real-Time
Volume Visualization. In Symposium on Volume Visualization, pages 47-54,
Oct. 1996.
We have presented an algorithm which allows for the correct rendering of both translucent and opaque polygons mixed with volumetric data. We have shown how this algorithm works with slice
order ray casting by rendering thin slabs of polygons between the
volume slices. We have also shown that clipping of polygons to
the thin slabs is not required. Instead, we presented a technique
for bucketing the polygons and showed its efficiency. We have presented an implementation of our algorithm on general-purpose geometry hardware. We have demonstrated our algorithm with multiple scenarios showing examples of medical applications, flight simulators, location based entertainment, and virtual environments. Finally, we have discussed the limitations of the method which affect
image quality and frame rate. In the future it would be interesting
to determine how to quickly compute and use the error from bucketing on!y versus full sorting for each slab of translucent polygons.
As we discussed in Section 4, the application program could bound
the error and allocate processing to fully depth sort the slabs where
incorrect compositing may cause significant color differences. Our
algorithm is based on standard OpenGL, allowing numerous extensions such as mixing with multiple volumes as shown in Figure 1, or
future work using stencil buffers to visualize intersections between
polygonal objects and volumetric data.
[14] T. Porter and T. Duff. Compositing Digital Images, In Computer Graphics,
SIGGRAPH 84, pages 253-259, July 1984.
[L5] C. Silva, J. Mitchell, and A. Kaufman. Fast Rendering of Irregular Grids. In
Symposium on Volume Visualization.pages 15-23, Oct. 1996.
[16] L. Sobierajskiand A. Kaufman. Volumetric ray tracing. InSymposiumon Volume
Visualization,pages 11-19, Oct. 1994.
[17] M. shriek and A. Kaufman vxt: A C++ Class Library for Object Voxelization.
In Proceedings of International Workshopon Volume Graphics 1999, pages 295306, Swansea, UK, Mar. 1999.
[18] E. Toms. Optimization of the Binary Space Partition Algorithm (BSP) for the
Visualization of Dynamic Scenes. In Eumgraphics ’90,pages 507-518, Sept.
1990.
[19] R. Westermann and T. Ertl. The VSBUFFER Visibility Ordering of Unstructured Volume Primitives by Polygon Drawing. In Proceedings of Visualization
‘97, pages 3 5 4 2 , Oct. 1997.
1201 R. Westennann and T.W. Efficiently Using Graphics Hardware in Volume
Rendering Applications. In Computer Graphics, SIGGRAPH 98, pages 169178, July 1998.
[21] L. Westover. Footprint Evaluation for Volume Rendering. In Computer Graphics, SIGGRAPH 90, pages 367-376, July 1990.
0-7803-5897-X/99/$10.00 Copyright 1999 IEEE
198
Figure 1: Mixing polygons with multiple volumes: A scalpel (containing 368 polygons) cutting into a human head (from MRI) w h i k
visualizing blood vessels (from an angwgram): (a) translucent polygons reveal the blood vessels behind the scalpel, (b) opaque rendering
obscures potentially dangerous movements.
Figure 2: Translucent rendering permits viewing the entire
field of view when the user is behind the avatarhehick (e.g.,
F-15 containing 2300 polygons).
Figure 3: Steam emanating from boiling tea inside a g h s
teapot (containing 3408 polygons) heated on an electric range.
Figure 4: A translucent helicopter (containing 2709 polygons)
spinning in a cloud over a textured terrain.
Figure 5: A prosthesis (containing 3758 polygons) being@ to
a 2563 CT scan of a hip.
Mixing Translucent Polygons with Volumes
Kevin A. Kreeger, Ane E. Kaufman
525