Volume Stylizer:
Tomography-based Volume Painting
Oliver Klehm1, Ivo Ihrke1,2, Hans-Peter Seidel1, Elmar Eisemann3
1
Max-Planck-Institut für Informatik
2 Saarland University
3 TU Delft
I3D 2013
23/03/2012
Oliver Klehm, 2
Motivation
Oliver Klehm, 3
 Want to influence the appearance of volumes (voxels)
• Editing on a fine-grained level is non-intuitive
• No suitable tools available
o A Programmable System For Artistic Volumetric Lighting [Nowrouzezahrai2011]
 Approach:
• Inverse rendering: editing by painting
Our Goal
Oliver Klehm, 4
 What do we modify
• Volume parameters (per voxel)
o Emission
o Albedo
o Extinction
Our Goal
Oliver Klehm, 5
 What do we modify
• Volume parameters (per voxel)
 From what do we edit the parameters
• Arbitrary input images (target views)
• In practice: rendered images that were painted
Our Goal: Volume Painting
Oliver Klehm, 6
Our Goal: Volume Painting
Oliver Klehm, 7
Our Goal
Oliver Klehm, 8
 What do we modify
• Volume parameters (per voxel)
 From what do we edit the parameters
• Arbitrary input images (target views)
• In practice: rendered images that were painted
 Additional required information
• Per image: camera parameters (viewing position, direction)
• Lighting, scene
 How do we optimize
• Per-pixel difference to input image
• Linear problem
• Inconsistent input -> least-squares solution
Specific Goal
Oliver Klehm, 9
 Given: each pixel k of target view as a constraint pixel
 Data associated:
ray origin, direction, target radiance
 Try to reconstruct volume properties, such that
computed radiance matches target radiance
(per pixel k)
Problem overview
Oliver Klehm, 10
 A single pixel may influence many voxels
 And vice versa…
Volume Rendering
Oliver Klehm, 11
Volume Reconstruction
Oliver Klehm, 12
 Represent the target field (e.g.
basis functions
) as linear combination of
• Common for stored volumes (== voxels: box/triangle basis fct.)
 Reconstruct coefficient vector
, such that
Includes target
Includes volume rendering
Emission Reconstruction
Oliver Klehm, 13
Optimization
Oliver Klehm, 14
 W is huge, ill-conditioned, …!
=> iterative conjugate gradient method
 We need to compute:
• Objective Function:
• Derivative:
 The trick:
=
=> this is rendering / back-projection!
Implementation
Oliver Klehm, 15
 Pre-computation
• Cache incoming light in volume per voxel
• Sub-sample environment map, deep shadow maps
 Optimization
• 1. objective function evaluation
o Voxel traversal, gathering pre-cached incoming light
Implementation
Oliver Klehm, 16
 Pre-computation
• Cache incoming light in volume per voxel
• Sub-sample environment map, deep shadow maps
 Optimization
• 1. objective function evaluation
• 2. gradient of objective function
Implementation
Oliver Klehm, 17
 Pre-computation
• Cache incoming light in volume per voxel
• Sub-sample environment map, deep shadow maps
 Optimization
• 1. objective function evaluation
• 2. gradient of objective function
• Additional vector arithmetic (addition, norm2, …)
Results - Video
Oliver Klehm, 18
Limitations / Future Work
Oliver Klehm, 19
 How to handle inconsistent inputs?
 How to define intermediate views?
 How to generalize? – requires different optimization
 Optimize for volume and lighting
Conclusion
Oliver Klehm, 20
 3D volume properties by 2D painting
 Standard Rendering Model
 Fast GPU Optimization for interactive sessions
Thank you
Fast Implementation
Oliver Klehm, 21
 All OpenGL
 Vectors are actually 3D Textures
 Operations as shaders
• Matrix multiplication is volume rendering
• Arithmetic operations are “simple” shaders
 => Reconstruction in order of seconds
Results
Oliver Klehm, 22
Results
Oliver Klehm, 23
Results - Extinction
Oliver Klehm, 24
Related Work
Oliver Klehm, 25
 Light Editing
• Lighting with paint [Pellacini2007]
o Non-linear light optimization
• BendyLights [Kerr2010]
o Editing of spot lights
• Optimizing Environment Maps for Material Depiction [Bousseau2011]
o Optimization with adhoc metrics
 Volume editing
• A Programmable System For Artistic Volumetric Lighting
[Nowrouzezahrai2011]
o
o
o
o
Derivation of volume properties for single light beam only
Replacing physically-motivated parts of rendering with shaders
Homogeneous media only
Treat light beam as string of light (e.g. flow of particle)
• Automatically Adjusting Parameters for Rendering Clouds using Photographs
[Dobashi2012]
Related Work
Oliver Klehm, 26
 Tomography
• Ihrke
• Wetzstein
• Lanmann
 Fabrication
• Shadow Art [Pauly2009]
• Manufactoring Layered Attenuators for Multiple Prescribed Shadow Images
[Baran2012]
Volume Properties
Oliver Klehm, 27
 Properties to reconstruct:
• Extinction (overall density of volume)
• Albedo (amount of scattering)
• Emission (self glow)
 Assumption: Single Scattering!
• Light scatters ONCE in volume
Emission reconstruction
Oliver Klehm, 28
Background / first surface, attenuated by volume
Integration along view ray
Attenuation by volume
Number of particles (extinction)
In-scattered light