Course Schedule
Mathematical Optimization
in Graphics and Vision
Part 1 - Optimization Problems in Graphics
Luiz Velho (45 minutes)
Why optimization is important for graphics?
Luiz Velho
Paulo Cezar Pinto Carvalho
Part 2 - Overview of Optimization Techniques
Paulo Cezar Pinto Carvalho (45 minutes)
IMPA - Instituto de Matematica Pura e Aplicada
How optimization can be used in graphics?
End - Questions and Answers (10 minutes)
July 2002
Course 51
2
Website
Optimization Problems
in Graphics
http://www.visgraf.impa.br/otim-02
• Presentation Slides
Luiz Velho
• Additional Material
July 2002
IMPA - Instituto de Matematica Pura e Aplicada
Course 51
3
Outline
Motivation
• Concepts
• Optimization is a Basic Tool for Graphics
• Graphical Objects
• Widely Used
• Operators
• Flexible
• Graphics Problems
• What Makes Optimization Important ?
• Direct / Inverse
• Well-Posed / Ill-Posed
• Need to Understand Graphics Problems
• Optimization
• Conceptual View
• Representation Operator
• Examples of Applications
July 2002
Course 51
5
July 2002
Course 51
6
1
Graphics and Vision
Graphics
Data
Graphical Object
o = ( U, f ),
Image
Vision
f : U ⊂ Rn → Rm
• Geometric Support: U
(shape)
• Relation with Physical Universe
• Attribute Function: f
• Photography: (2D representation)
(properties)
• Human Visual System: (3D reconstruction)
• Concepts
* Dimension
• What are the Models ?
* Object: dim(U)
• What are the Problems ?
July 2002
* Space: n
Course 51
7
July 2002
Course 51
Simple Example
8
More Examples
• 2D Drawing
• Images:
• Grayscale, Color
(simple shape, complex attributes)
• Models (Data)
Shape
• Surfaces, Solids
Texture
(complex shape, simple attributes)
Graphical Object
Color
* Graphical Object :- Comprehensive Concept
Attributes
July 2002
Course 51
9
July 2002
Course 51
10
Processing Graphical Objects
Operators on Graphical Objects
Geometric Modeling
Operators
Models
Image Analysis
Image Synthesis
Operators
Operators
Images
Image Processing
July 2002
Course 51
Operators
11
July 2002
Course 51
12
2
Graphical Problems
Types of Problems
• Spaces of Graphical Objects
T(x) = y
( function spaces )
• Direct Problems
• Given T and x , find y
x ∈Ο
• Operators on Spaces of Graphical Objects
• Inverse Problems
T : O1 → O2
• Given T and
• Given x and
T(x) = y
July 2002
Course 51
13
y , find x such that T (x) = y
y , find T
July 2002
Course 51
Example: Visualization
14
Problem Characterization
• x is the scene
* Hadamard (1902)
• Geometry
• Well-Posed Problem
• Illumination
• Existence of Solution
• Camera
• T is the rendering operator
• Uniqueness of Solution
• Continuous Dependence on Initial Conditions
Tx=y
• y is the rendered image
• Ill-Posed Problem
(doesn’t satisfy at least one of the above conditions)
* Direct Problem (Vision - Inverse Problem)
July 2002
Course 51
15
July 2002
Ill-Posed Problems
16
Example of Ill-Posed Problem
• Inherent in some Applications
• Linear System: T x = y,
• Inverse Problems are usually Ill-Posed
T=
• Sources of Ill-Posedness
• Multiple Solutions
 1 3  , x = (u, v) , y = (a, 2a) , and a ∈ R
 2 6 


• Solution y must lie on
• Numerical Errors
2u - v = 0
⇒Need to get around Ill-Posedness
• Ill-Conditioning / Perturbations
• Optimization Methods
T(x)
y
• Well-Posed Solution
• Best Solution (unique)
July 2002
Course 51
y = argmin |y, Tx|
Course 51
17
July 2002
Course 51
18
3
Describing Graphical Objects
Ill-Posed Problems in Graphics
• Representation Operator
• Computer Representation from Mathematical Model
R : O → Od
• Discretization of Graphical Objects
• Direct Problem
Continuous
Reconstruction
• Reconstruction of Graphical Objetcs
Discretization
• Inverse Problem
Discrete
July 2002
Course 51
19
July 2002
Reconstruction Issues
Course 51
20
Ambiguity in Models
• Wireframe Model
• Why do we need to reconstruct ?
• Discretization gives incomplete information
• Working in continuous domain avoid numerical errors
• Semantics
• Many Interpretations
• Reconstruction
(Invertibility of the Representation Operator)
• Exact
• Non-Exact
• Ill-Posed Reconstruction
• Ambiguous Representation
July 2002
Course 51
21
Ambiguity in Images
July 2002
Course 51
22
Graphical Problems and Optimization
• Two possible interpretations
* Typical problems from different areas
• Selected Problems
• Modeling
• Visualization
• Vision / Image Procesing
• Not Discussed
• Animation
Foreground / Background
2D / 3D
July 2002
Course 51
23
July 2002
Course 51
24
4
Variational Modeling
Internal Energy
• Energy Minimization Model
• Fair Shape (Physical Model)
γ(t ) = arg min Etotal (γ )
Einternal (γ ) = λEbend + (1 − λ) Estretch
• Thin Plate
Ebend (γ ) = ∫ k 2 (t )dt
• Energy Functional
• Shape Quality
γ
• Model Control
• Membrane
Estretch(γ ) = ∫ || γ ' (t ) || dt
Εtotal (γ ) = Einternal (γ ) + Eexternal (γ )
γ
* Spline Curves and Surfaces
July 2002
Course 51
25
July 2002
Course 51
External Energy
26
Applications
• Modeling Controls
• Reconstruction from Points
• Attraction / Repulsion Forces
• Interactive Modeling
• Punctual
Epunctual (γ ) = min t || γ(t ) − p ||2
• Directional
Edirectional (γ ) = min t || γ ' (t ) × p ||
July 2002
Course 51
27
July 2002
Camera Calibration
28
Differential Camera Control
• Projective Transformation
• Camera Motion with Constrains
T (x ) = p
minimize
• Inverse Problems
E=
• Compute Camera Transformation T
• Compute Scene Points x
Pinhole Camera
(7 parameters)
Course 51
1 dc dc0
−
2 dt dt
2
subject to
dp dp0
−
=0
dt dt
* Best Parameter Estimation
July 2002
Course 51
29
July 2002
Course 51
30
5
Image Analysis
Wavelets
• Primal Sketch
• Frequency Methods for Computation of Edges
• Edges
• Image Reconstruction and Compression
• David Marr’s Conjecture
( invertibility of boundary operator )
• Image is Characterized by its Edges
• Boundary Operator
( Computation of Edges )
• Frequency Methods
• Geometric Methods
July 2002
Course 51
31
July 2002
Snakes
Course 51
32
Conclusions
• Geometric Computation of the Boundary
* Optimization has many uses in Graphics
• Representation
• Energy Minimization Approach
• Solves Ambiguity
• Incorporates User-defined Criteria
• Algorithms
• Efficient
• Robust
• User Interface
• Intuitive Controls
July 2002
Course 51
33
July 2002
Course 51
34
6