A plane-plus-parallax algorithm
Basic Model:

When FOV is not very large and the camera motion has a small
rotation, the 2D displacement (u,v) of an image point (x,y) in the
image plane can be expressed:

where
General Framework



Estimate the 2D parameters of a single motion
Register two successive frames according to the computed
2D parameters. This alignment was proved to cancel the
rotation motion for the entire scene, resulting in a new
sequence which only contains 3D translation, and looks as
if taken from a stabilized platform with no jitter.
Further option: Compute FOE (translation) from the residual
epipolar displacement field between the two registered
frames and recover the 3D rotation parameters from the 2D
parameters and 3D translation parameters.
2D parameters estimation:
Overview

For small motions, assume gray level constancy:
- For translation: 2 parameters, u(x,y) = a, v(x,y) = b
- For Affine: 6 parameters, u(x,y) = a+bx+cy, v(x,y) = d+ex+fy.
- For a pseudo projective transformation: u(x,y)=a+bx+cy+gx^2+hxy, v(x,y)=c+dx+fy+gxy+hy^

First order Taylor expansion and neglecting all nonlinear terms:
I ( x  u, y  v, t  1)  I ( x, y, t )  uI x  vI y  I t
where

Ix 
I ( x, y, t )
I ( x, y, t )
I ( x, y, t )
, Iy 
, It 
x
y
t
Minimize the error function at Frame t in the region of R:
Err (t ) (u, v) 

( x , y )R
(uI x  vI y  I t ) 2
(Its derivatives with respect to the parameters are set to zero.)
2D parameters estimation:
Details





Multiresolution iterative framework
Construct a Gaussian pyramid
Start at the lowest resolution level, motion parameters are estimated by solving
the set of linear equation to minimize the error function according to the
appropriate model.
Incremental motion estimation can be performed at each resolution level.
Coarse-fine alignment: at each successive iteration, the shift and estimates
steps are performed on the next higher resolution pyramid level, that is, the shift
estimated at level k+1 of frame t is applied to level k of frame t to form a new
level k of frame t, then the residual can be computed between the new level k of
frame t and the level k of frame t+1.
Motion Correction

Warping two successive frames according to the
computed 2D parameters.
 Theoretically, this step should be able to cancel
the rotation motion for the entire scene, resulting
in a new sequence which only contains 3D
translation, and looks as if taken from a stabilized
platform with no jitter.
 Stabilized GoCart (not so good)
 Problems: Didn’t work for other test videos ( Got
zero gradient, thus NaN …. Need debug and
further improvements)