Image Analogies
Aaron Hertzmann1,2
Chuck Jacobs2
Nuria Oliver2
Brian Curless
3
David Salesin2,3
1New
York University
2Microsoft
Research
3University
of Washington
1
Motivation
Goal: Transform a image by example
One mechanism, many filters
2
Problems
No unique transformation
What aspects of the images are
important?
3
Idea
Difficult or impossible to find the
filter/Transformation!
Find proper patches/pixels from the
source image directly and combine
them the get target image!
4
Analogies
We will find:
Many analogies with the former
paper presented by Rui!
Find right patches and combine
them into a good result for your
target image!
5
Contribution
1. New framework for transforming
image by example
2. Generalized the texture
synthesis, VISTA … to image
analogy.
6
Applications
Texture synthesis
Super-resolution
Texture transfer
Artistic filters
Texture-by-numbers
7
Previous work
Graphics: Texture Synthesis
1.
2.
3.
Ashikmin [2001]
Wei and Levoy [2000]
Efros and Leung [1999]
Vision:
1.
2.
Learning low-level Vision (VISTA)
Example based Super Resolution
(SinglePass)
8
Related work
Machine learning for graphics
Texture synthesis
Non-photorealistic rendering
Example-based NPR
9
Algorithm
10
Texture synthesis (Efros 99)
Input texture
Output texture
11
Multiscale synthesis (Wei 00)
12
Multiscale synthesis
Input texture
Output texture (Wei 2000)
13
Coherent synthesis
Input texture
(Ashikhmin 01)
Output texture
14
Coherent synthesis
Input texture
Output texture (Ashikhmin 2000)
15
Learning Low-level Vision
Vision by Image/Scene TrAining –
VISTA
16
New, hybrid algorithm
17
Combined algorithm
Input texture
Output texture (our algorithm)
18
Hybrid synthesis
19
Feature vectors
20
Feature vectors
21
Color
22
Feature vectors
23
Analogies with VISTA
Traning (Data Collection)
Y
Y
X
X
Inference
24
Analogies with VISTA
Belief Propagation vs. Single Pass
Algorithm
25
Applications
26
::
:
B
B’
28
Texture transfer
:
A
A’
29
::
:
B
B’
30
Pen-and-ink
:
A
A’
31
Pen-and-ink
::
:
B
B’
32
Stippling
:
A
A’
33
Stippling
::
:
B
B’
34
Pastel
:
A
A’
35
Pastel
::
:
B
B’
36
Oil
:
A
A’
From Still life with Melon and Peaches, Edouard Manet, 1866
37
Oil
::
:
B
B’
38
Watercolor
:
A
From (Curtis 97)
A’
39
Watercolor
::
:
B
B’
40
Texture-by-numbers
:
A
A’
41
Texture-by-numbers
::
:
B
B’
42
Texture-by-numbers
:
A
A’
43
Texture-by-numbers
::
:
B
B’
44
Thanks!
45