Project by
Arie Kozak


Given photograph with sheet of paper with
text only, infer shape of the surface and plot it
in 3d.
Single (infinite) light source from above, using
reflectance map (paper is nearly Lambertian
1
surface): 𝑅 𝑝, 𝑞 =
2
2
𝑝 +𝑞 +1

The surface is assumed to be constant in one
direction.

Mark it using personal biological visual
system.

Divide the image into two connected subimages divided by red border.

Use thresholding twice: after high pass and
original image. Text found in the intersection.


Constant albido assumption for ink, doesn’t
work, use (cubic) interpolation.
Smooth image with Gaussian kernel before to
reduce “sharpening effect” (lateral inhibition),
and also after.

Maximum intensity point in image => p = q = 0. Use
parabolic approximation according to B.K.P. Horn's
chapter 11:
𝐻 𝑥, 𝑦 = 𝐻0 + 0.5(𝑎𝑥 2 + 𝑏𝑥𝑦 + 𝑐𝑦 2 )
𝜕𝐻
𝜕𝐻
𝑝=
= 𝑎𝑥 + 𝑏𝑦, 𝑞 =
= 𝑐𝑦 + 𝑏𝑥
𝜕𝑥
𝜕𝑦
1
2 + 𝑞2 + 1
𝐸 𝑥, 𝑦 =
=
0.5
𝑝
2𝐼(𝑥, 𝑦)2
= 0.5 𝑎2 + 𝑏 2 𝑥 2 + 𝑎 + 𝑐 𝑏𝑥𝑦 + 0.5 𝑐 2 + 𝑏 2 𝑦 2 + 1
𝐸𝑥𝑥 = 𝑎2 + 𝑏 2
𝐸𝑦𝑦 = 𝑐 2 + 𝑏 2
𝐸𝑥𝑦 = 𝑎 + 𝑐 𝑏


Solution
𝑏 4 𝐸𝑦𝑦 2 − 2𝐸𝑦𝑦 𝐸𝑥𝑥 + 𝐸𝑥𝑥 2 + 4𝐸𝑥𝑦 2
+ 𝑏 2 −2𝐸𝑦𝑦 𝐸𝑥𝑦 2 − 2𝐸𝑥𝑦 2 𝐸𝑥𝑥 + 𝐸𝑥𝑦 4 = 0
Only solutions with a<0,c<0 are relevant.

Identify “clusters” – areas of local
maxima/minima. All points within certain %
of highest intensity values.

Start with H = 0, perform for each cluster
separately.



Find closest clusters A and B; B with known
height.
For points in A close to B, calculate expected
height according to B.
Find closest points using Voronoi diagram.

Find relative height between A and B. If 𝑎𝑖 , 𝑏𝑖
is current and expected height of point i
accordingly, find relative height x, such that
error will be minimal:
𝑁
(𝑎𝑖 + 𝑥 − 𝑏𝑖 )2 → 𝑚𝑖𝑛
𝑒 𝑥 =
𝑖
1
𝑥=
𝑁
𝑁
(𝑏𝑖 − 𝑎𝑖 )
𝑖


Find direction v, in which H is constant =>
derivative is 0.
∀𝑖: 𝑔𝑖 ∗ 𝑣 = 0, 𝑔𝑖 = (𝑝𝑖 , 𝑞𝑖 )
Find least square line, its directions is
perpendicular to v.

If v is new x-axis, calculate projection of all
points to YZ plane.


Use polyline approximation. Given number of
desired points = number of clusters + 2, the
desired error can be approximated using
binary search.
Example – 5 points:

Finally, use spline, on polyline edge points.

Not perfect, usually works sufficiently.





Detect sheet of paper automatically.
Relax assumptions (light direction, H is
constant in one direction).
Improve clusters search.
Replace/improve polyline approximation.
Use this for text recognition.