• Data Flow model
• Tools to automatically convert a C, C++ or
Fortran program to a module.
– provides the shared memory communication
– each “module” is it’s own process
• Can reside on it’s own CPU
• Can reside across the network
• Very similar to Corba support
• Data Push models
– aPe - Ohio State / OSC
– AVS - Advanced Visualization System
• Stellar Computer
– Khoros - Univ. of New Mexico?
– Iris Explorer – SGI / NAG
Great concepts
– Provide wrapper for remote procedure and
shared memory
– Provide GUI for parameters (args) of the
program.
Early Visual Prog. Env.
– Custom tools for custom data formats
– Custom derived data, data mappers
• Post-process of restart dumps
– Embed into simulation code
– Outputs images (plots) during run-time.
•
•
•
• Visualization libraries
Haberli’s landmark ConMan paper -1985?
Extension of Unix pipes.
Take a small specialized program
Visual Programming
Visualization History
C
• Multiple windows (paths)
• Memory requirements reduced
– Only request data when needed
– Break the data into chunks, and only request
small chunks at a time.
• Data Pull
– Push data when changed
• Data Push
Data Push vs Data Pull
B
A
C
– Much more complicate, Much more flexible
– Data Explorer (Dx) - IBM
• Data centric
– AVS/Express
– Image Vision - SGI
– Java Advanced Imaging
– Vtk
• Data Pull Models
Early Visual Prog. Env.
B
A
• Each module has a set of parameters or
state that controls its output.
• A change to Module B’s parameters only
requires re-executing B.
• A change to Module A’s parameters, will reexecute A and send new data Read
to both B and C.
• Data pushed to module A
• Data modified by A (a “mapper”) and
pushed to both modules B and C.
• Four copies of the data now exist
Read
Execution Model
Problems
[email protected]
Components
Little or no GUI
Meant to be integrated with other systems/toolkits
Programming required
– TurnTurn-key
– Minimal programming
• Applications
– Components
– GUIGUI-assisted (visual) programming
• Development Environments
–
–
–
–
• Toolkits / Libraries
Taxonomy of Visualization
Systems
Modified by
Roger Crawfis
www.kitware.com
www.vtk.org
William J. Schroeder
President, Kitware, Inc.
Rensselaer SCOREC Research Associate
VTK
The Visualization Toolkit
––
––
––
––
––
––
––
––
www.vtk.org
www.vtk.org
Portable
Portable across
across Unix,
Unix, Windows,
Windows, Mac
Mac OSX
OSX
Supports
Supports 3D/2D
3D/2D graphics,
graphics, visualization,
visualization, image
image processing,
processing, volume
volume
rendering
rendering
Focused
higher
Focused on
on scientific
scientific visualization,
visualization, some
some support
support for
for higherhigher-dimensional
dimensional visualization
visualization
Open-source (BSD-style, not
Open
(BSD
Open-source
(BSD-style,
not GPL)
GPL)
Designed
-oriented principles
object
Designed and
and implemented
implemented using
using objectobject-oriented
principles
C++
C++ class
class library
library (~700+
(~700+ classes;
classes; ~425,000
~425,000 LOC;
LOC; <225,000
<225,000
executable
executable lines)
lines)
Automated
Automated Java,
Java, TCL,
TCL, Python
Python bindings
bindings
AA visualization
visualization toolkit
toolkit
What Is VTK?
• Conclusion
• Software Process
• Interaction
• Visualization Pipeline
• Graphic Subsystem
• Architecture
• Overview
Themes
Things It Does
– Not dependent on GUI
• X, Motif, Windows, Mac, Qt, FLTK, Tk, wxWindows,
etc.
– Not dependent on rendering library
• OpenGL, others
• Adaptable
– Can pull out “pieces”
• Separable
More Things: Image-Processing
• June 1998 Kitware forms:
VTK Commercial Support
• December ‘93 work begins on VTK
text/software
– Textbook
– Sample code
• Proprietary software issues led to
open-source VTK project to create:
• Precursor: GE Legacy System LYMB (circa 1983)
• Embeddable
– Plays with other software
History
VTK Is A Toolkit, Not a
System
• Graphics / Modeling
– Projection, resampling
• Multi-dimensional
– Scalar, vector, tensor fields (100’s of filters)
– Probing, cutting, clipping, extraction
– Volume rendering
• Visualization
– 100’s of filters
– Integrated 2D / 3D
• Image Processing
• Assemble components to build applications
C++
Tcl
Python
Java
• Choose programming language
Functionality
–
–
–
–
How To Use It
Volume Rendering
– Smoothing, decimation, extrusion, Delaunay triangulation
(2D/3D), etc.
Combined Volume/Surface
Rendering
– Often interpreted (e.g. Tcl)
– Can be compiled layer
• Microsoft COM
• Wrapper layer
– Performance
– Code correctness
• Compiled C++ core
VTK Architecture
renWinÆ
renWinÆRender();
Render();
irenÆ
irenÆStart();
Start(); // being event loop
rendererÆ
rendererÆAddActor(sphereActor);
AddActor(sphereActor);
rendererÆ
rendererÆSetBackground(1,1,1);
renWinÆ
renWinÆSetSize(300,300);
Wrapper layer
generated
automatically
during build
process
C++
core
Wrappers
vtkRenderer *renderer = vtkRenderer::New();
vtkRenderer::New(); // graphics library independent
vtkRenderWindow *renWin = vtkRenderWindow::New();
vtkRenderWindow::New();
renWinÆ
renWinÆAddRenderer(renderer);
AddRenderer(renderer);
vtkRenderWindowInteractor *iren = vtkRenderWindowInteractor::New();
vtkRenderWindowInteractor::New();
irenÆ
irenÆSetRenderWindow(renWin);
SetRenderWindow(renWin);
vtkPolyDataMapper *sphereMapper = vtkPolyDataMapper::New();
vtkPolyDataMapper::New();
sphereMapperÆ
sphereMapperÆSetInput(sphereÆ
SetInput(sphereÆGetOutput());
GetOutput());
vtkActor *sphereActor = vtkActor::New();
vtkActor::New();
sphereActorÆ
sphereActorÆSetMapper(sphereMapper);
SetMapper(sphereMapper); // mapper connects the actor with pipeline
vtkSphereSource *sphere = vtkSphereSource()::New();
vtkSphereSource()::New(); // create data pipeline
Typical Application (in C++)
• source vtkInt.tcl (define interpreter GUI)
Tcl Interpreter
– Graphics
– Visualization pipeline
• Image processing
• 3D data processing
– Interaction
• Subsystems
– C++ core
– Tcl, Python, Java wrappers
• Compiled / Interpreted
Architecture
m SetInput [ sf GetOutput ]
vtkPolyDataMapper m
sf SetShrinkFactor 0.8
vtkShrinkFilter sf
• Tcl
• vtkTransform - position actors
– vtkPolyDataGeometry,
vtkPolyDataGeometry, vtkDataSetMapper are subclasses
• vtkLights - illuminate actors
• vtkCamera - renders the scene
• vtkMapper - represents geometry
– vtkProp,
vtkProp, vtkProp3D are superclasses
– vtkProperty
A VTK scene consists of:
• vtkRenderWindow - contains the final image
• vtkRenderer - draws into the render window
• vtkActor - combines properties / geometry
The VTK Graphics Subsystem
m=
vtkPolyDataMapper::Ne
()
vtkPolyDataMapper *m;
sf>SetShrinkFactor(0.8);
sf =
vtkShrinkFilter::New();
vtkShrinkFilter *sf;
• C++
Code Comparison
Lights
Cameras
Actors
Mappers
Transforms
Properties
Render Window
Renderer
Actor instances
(vtkActor)
Renderer instances
(vtkRenderer)
Instances of render window (vtkRenderWindow)
Graphics Model
•
•
•
•
•
•
•
•
Graphics Subsystem
Data
Mapper
Actor
to graphics
system
Actor
– Ingest, transform, and output data objects
– represent visualization algorithms
• Process Objects
– represent data
– provide access to data
– compute information particular to data
(e.g., bounding box, derivatives)
contain instance of vtkFieldData
an array of arrays
no geometric/topological structure
Superclass of all VTK data objects
– Consists of geometry (points) and topology (cells)
– Has associated “attribute data” (e.g., scalars, vectors) as well
as field data
– Convert data object to data set with
vtkDataObjectToDataSetFilter
• vtkDataSet has geometric/topological structure
–
–
–
–
• vtkDataObject represents a “blob” of data
Filter
Data
Mapper
• Data Objects
Data
Data
Filter
vtkDataObject / vtkDataSet
Source
A sequence of process objects that operate on data objects
to generate geometry that can be rendered by the
graphics engine or written to a file
The Visualization Pipeline
Visualization Model
• Managing execution
• Pipelines
• Process objects
• Data objects
Visualization Pipeline
Associate
d
Data
vtkPolyData
vtkStructuredGrid
vtkImageData
vtkUnstructuredGrid
vtkRectilinearGrid
Point Data
Cell Data
Points
& Cells
Geometry
& Topology
Attribute
Data
Implementation
Concept
Structure
Field Data
vtkDataSet Subclasses
vtkDataSet
vtkDataObject
Array of
arrays
vtkDataObject / vtkDataSet
Cell
–
–
–
–
–
–
Scalars – (multi-component)
Vectors - 3-vector
Tensors - 3x3 symmetric matrix
Normals - unit vector
Texture Coordinates 1-3D
Array of arrays (I.e. FieldData)
• vtkDataSet also has point and cell attribute
data:
Data Set Attributes
Points
– points (x(x-y-z coordinates)
– cells (e.g., polygons, lines, voxels)
voxels) that are defined by
connectivity list referring to points ids
– Access is via integer ID
– implicit representations
– explicit representations
• Structure consists of
• A dataset is a data object with structure
Dataset Model
Data
Source
Mapper
direction of update (via Update())
Data
Filter
direction of data flow (via Execute())
Pipeline Execution Model
– mapped through lookup table
– if unsigned char Æ direct color specification
• Scalars can be used to generate colors
I (intensity)
IA (intensity(intensity-alpha: alpha is opacity)
RGB (red(red-greengreen-blue)
RGBA (RGB + alpha)
Render()
1 or more outputs
Filter
1 or more inputs
Mapper
1 or more inputs
–
–
–
SetInput() accepts dataset type or subclass
C++ compile-time checking
Interpreter run-time checking
• The Role of Type-Checking
• aFilter->SetInput( bFilter->GetOutput());
Creating Pipeline Topology
1 or more outputs
Source
• Process objects operate on data objects
• Scalars are represented by a vtkDataArray
• Scalars are typically single valued
• Scalars can also represent color
–
–
–
–
Process Objects
Scalars (An Aside)
vtkPolyDataMapper
vtkPolyDataNormals
vtkSmoothPolyDataFilter
vtkDecimatePro
vtkActor *cyberActor = vtkActor::New();
vtkActor::New();
cyberActorcyberActor->SetMapper (cyberMapper);
cyberMapper);
cyberActorcyberActor->GetProperty()
GetProperty()-->SetColor ( 1.0, 0.49, 0.25 );
cyberActorcyberActor->GetProperty()
GetProperty()-->SetRepresentationToWireframe();
SetRepresentationToWireframe();
vtkPolyDataMapper *cyberMapper = vtkPolyDataMapper::New();
vtkPolyDataMapper::New();
cyberMappercyberMapper->SetInput(
SetInput( normalsnormals->GetOutput()
GetOutput() );
vtkPolyDataNormals *normals = vtkPolyDataNormals::New();
vtkPolyDataNormals::New();
normalsnormals->SetInput(
SetInput( smoothsmooth->GetOutput()
GetOutput() );
vtkSmoothPolyDataFilter *smooth = vtkSmoothPolyDataFilter::New();
vtkSmoothPolyDataFilter::New();
smoothsmooth->SetInput(deciSetInput(deci->GetOutput());
GetOutput());
smoothsmooth->SetNumberOfIterations(
SetNumberOfIterations( 20 );
smoothsmooth->SetRelaxationFactor(
SetRelaxationFactor( 0.05 );
Smoother & Graphics Objects
Note: data objects are not
shown Æ they are implied
from the output type of the
filter
vtkBYUReader
• Decimation, smoothing, normals
• Implemented in C++
Example Pipeline
ireniren->Start();
ren1ren1->AddActor(
AddActor( cyberActor );
ren1ren1->SetBackground(
SetBackground( 1, 1, 1 );
renWinrenWin->SetSize(
SetSize( 500, 500 );
vtkRenderWindowInteractor *iren =
vtkRenderWindowInteractor ::New();
ireniren->SetRenderWindow(
SetRenderWindow( renWin );
vtkRenderWindow *renWin = vtkRenderWindow::New();
vtkRenderWindow::New();
renWinrenWin->AddRenderer(
AddRenderer( ren1 );
vtkRenderer *ren1 = vtkRenderer::New();
vtkRenderer::New();
More Graphics Objects
vtkDecimatePro *deci = vtkDecimatePro::New();
vtkDecimatePro::New();
decideci->SetInput(
SetInput( byubyu->GetOutput()
GetOutput() );
decideci->SetTargetReduction(
SetTargetReduction( 0.9 );
decideci->PreserveTopologyOn();
PreserveTopologyOn();
decideci->SetMaximumError(
SetMaximumError( 0.0002 );
vtkBYUReader *byu = vtkBYUReader::New();
vtkBYUReader::New();
byubyu->SetGeometryFileName("../../vtkdata/fran_cut.g”);
SetGeometryFileName("../../vtkdata/fran_cut.g”);
Create Reader & Decimator
After Decimation
and Smoothing
(7,477 triangles)
– 8.5 million tetrahedra (per time step)
– 150 million tetrahedra (soon)
• Modeling turbulence (Ken Jansen Rensselaer)
Examples
Before
(52,260 triangles)
Results
140 Gbytes
• Bell-Boeing V-2 2 tiltrotor
870
870 MBytes
MBytes 1734
1734 Slices
Slices at
at 512x512x2
512x512x2
• Visible Woman CT Data
Large Data Management
– Petabyte example on 2048 LANL machine (Martin
(Martin et al.
IEEE CG&A July 2001)
2001)
(A petabyte is 2 to the 50th power
(1,125,899,906,842,624) bytes. A petabyte is equal to
1,024 terabytes. )
• Scalable
– Task
– Data
– Pipeline
• Parallel Mechanisms
– MPI
– Abstracted to support other communication
• Why ? – we have to
• Shared memory
• Distributed execution
Parallel Processing
• Interactor Styles
• Picking interaction
• Light Follow Camera (a headlight)
• Key and mouse bindings (Interactor Style)
xMax
• 3D Widgets
xMin
xMax
Key features:
• SetRenderWindow – the single render
window to interact with
yMin
yMin
yMin
xMin
zMax
Piece = 0
NumberOfPieces = 2
• Command / Observer callback mechanism
Filter
yMin
zMin
data->SetUpdateExtent(0, 2);
data->Update();
image->SetUpdateExtent(0, 128, 0, 128, 0, 90);
image->Update();
vtkRenderWindowInteractor
Piece size based on memory limits
Can avoid system swap
Supports parallel processing
Issues
• How to create pieces
• Mapping output from input
• Results invariance
Graphics
Imaging: i,j,k extents.
Pieces
Interaction
–
–
–
–
• Data is broken into pieces, and pieces processed one
at a time through pipeline
• Basic idea: handle any size data on any size computer
Streaming Pipeline
(Note: defaults are automatically created, you rarely
ever need to do this)
iren->SetInteractorStyle(flight);
iren->SetPicker(picker);
• Observers observe events on a particular VTK
instance
vtkRenderWindowInteractor *iren =
vtkRenderWindowInteractor::New();
vtkInteractorStyleFlight *style =
vtkInteractorStyleFlight::New();
vtkCellPicker *picker = vtkCellPicker::New();
– ThisThis->InvokeEvent(vtkCommand::ProgressEvent,NULL);
InvokeEvent(vtkCommand::ProgressEvent,NULL);
• The events originate from instances that invoke
events on themselves (and may pass call data)
• When an observer sees the event it is interested in,
it invokes a command via the command’s Execute()
method
– Use AddObserver()
AddObserver() to watch for events
– The observer is implicit; you actually add the command
Command / Observer Callbacks
• vtkWorldPointPicker - get x-y-z coordinate; does not
pick prop (hardware assisted)
• vtkCellPicker - pick cells (software ray cast)
• vtkPointPicker - pick points (closest point to camera
within tolerance - software ray cast)
Example: Picking and Style
Switch styles: Keypress j – joystick; t - trackball style
• vtkPropPicker - hardware-assisted picking of vtkProps
• Button 1 – rotate
• Button 2 – translate (<Shift> Button 1 on
PC)
• Button 3 – zoom
• Keypress e or q – exit
• Keypress f – “fly-to” point under mouse
• Keypress s/w – surface/wireframe
• Keypress p – pick
• Keypress r – reset camera
• Keypress 3 – toggle stereo
• vtkPicker - pick based on prop3D’s bounding box
(software ray cast)
Picking
Interactor Style(s)
vtkCommand
vtkCommand
vtkPointWidget
vtkLineWidget
vtkPlaneWidget
vtkBoxWidget
vtkSphereWidget
• More than one widget at a time can be used
• Widgets often provide auxiliary functionality (e.g.,
obtaining transforms, polydata, implicit functions,
etc.)
•
•
•
•
•
Registered observers
vtkCommand
Some 3D Widgets
this->InvokeEvent()
vtkObject
Command / Observer
vtkBoxWidget
– Interactor observers watches events invoked on
vtkRenderWindowInteractor
– Events are caught and acted on
– Events can be prioritized and ordered
– The handling of a particular event can be
aborted
• Subclass of vtkInteractorObserver
– Requires nightly / VTK 4.2 release
• Added since VTK 4.0 release
3D Widgets
vtkLineWidget
vtkImplicitPlaneWidget
vtkPlaneWidget
vtkPointWidget
Example Usage (Tcl)
vtkSphereWidget
lineWidget SetInteractor iren
lineWidget AddObserver StartInteractionEvent BeginInteraction
lineWidget AddObserver InteractionEvent GenerateStreamlines
…….
………
vtkLineWidget lineWidget
vtkPolyData seeds
lineWidget SetInput [pl3d GetOutput]
GetOutput]
lineWidget AlignWithYAxisOn
lineWidget PlaceWidget
lineWidget GetPolyData seeds
vtkScalarBarWidget
vtkImagePlaneWidget
– Makefiles on Unix/Linux
– Projects/workspaces on MSVC
Cross-Platform Make (build environment)
Open source, runs everywhere
Simple “makefiles” CMakeLists.txt
Generates native build files
CVS
CMake
Doxygen
DART
• www.cmake.org
•
•
•
•
–
–
–
–
• Tools
Registration
Segmentation
Medical Image
Processing
C++ Toolkit
Insight Toolkit (ITK)
• Automatic wrapper generator (look at
SWIG)
• Daily regression testing
– Allows simultaneous edits
– Merges files, flags conflicts
• Standard C++ Style / Methodology
• Documentation embedded in code (use
Doxygen to generated HTML)
• Use CVS source code control system
• The Quality Dashboard
– Managing open-source development
– The importance of feedback
Development Process
Software Process
Uses Generic Programming Concepts (i.e. heavily templated)
templated)
N-Dimensional Images / Meshes
Complex Pixel Types
BSDBSD-style openopen-source license
Streaming
CMake Build Environment
CVS Source Code Control
DART Testing System
Doxygen Documentation System
GNATS Bug Tracking
Web (itk.org
), Mailing lists (mailman), etc.
(itk.org),
• www.itk.org
–
–
–
–
–
–
• CABLE Interpreted Language Bindings
• Development Process
–
• Data Flow Architecture
–
–
–
–
• C++ Library
ITK