Easy Optimization
Problems,
Relaxation,
Local Processing
for a single variable
Multiscale solvers
Coarsening: create a hierarchy of problems
graphs, equations, systems of particles, etc.
Original system
1st coarsening
2nd coarsening
3rd coarsening
Multiscale solvers
Coarsening: create a hierarchy of problems
graphs, equations, systems of particles, etc.
Solve the coarsest level
Coarsest level solution
Multiscale solvers
Coarsening: create a hierarchy of problems
graphs, equations, systems of particles, etc.
Solve the coarsest level
Uncoarsening:
 Initialize the solution on a finer level from the
coarser level by interpolation
 Improve the initial solution by local processing
Local processing
Main assumption
The solution of the larger scales has
been obtained by the coarser levels
At each level apply only local changes
Since done iteratively, need not solve to
the optimum, just approach it
Variable by variable strict
unconstrained minimization
 Discrete (combinatorial) case : Ising model
2D Ising spins
 Minimize

s s h s
i , j
i
j
i i
i
 Periodic boundary condition
 Initialize randomly: si  1 with probability .5
Exc#1: 2D Ising spins exercise
 Minimize 
s s h s
i , j
i
j
i i
i
 Periodic boundary condition
 Initialize randomly: si  1 with probability .5
1. Go over the grid in lexicographic order, for each spin
choose 1 or -1 whichever minimizes the energy
(choose with probability ½ when the two possibilities have the
same energy) until no changes are observed.
2. Repeat 3 times for each of the 4 possibilities of (h1,h2).
3. Is the global minimum achievable?
4. What local minima do you observe?
Variable by variable strict
unconstrained minimization
 Discrete (combinatorial) case : Ising model
 Quadratic case : P=2
Necessary optimality conditions
Let x * be a local minimum of E ( x ) and assume
Eis continuously differentiable in some domain D,
then the 1st order Necessary Condition is
E ( x*)  0
If in addition Eis twice continuously differentiable
within D, then the 2nd order Necessary Condition is
2
 E ( x*) positive semidefinite
Sufficient optimality conditions
Let E be twice continuously differentiable in
domain Dand let x*  D satisfy the conditions
E ( x*)  0,  E ( x*) positive definite
2
then
x * is a strict unconstrained local minimum of E
.
If, in addition, E ( x ) is quadratic, the local minimum is
also the global unique minimum.
Pointwise relaxation for P=2
 Minimize
( x )   aij( xi  xj )
2
ij
 Pick a variable
 Minimize
xi , fix all xj j  i at
~xj
2
~
( xi )   aij(xi  x j)
j
Quadratic functional in one variable – easy to solve!
d( xi )
0 
d ( xi )
~
xi   aij x j /  aij
j
j
Pointwise relaxation for P=2
(cont.)
 Check the 2nd derivative:
d ( xi )
  aij  0
2
j i
d ( xi )
2
=> Unique minimum!
Put
xi
at the weighted average location
of its graph neighbors
 Go over all variables in lexicographic order
Problem: Does not preserve the volume demands!
Reinforce volume demands at the end of each sweep
Variable by variable strict
unconstrained minimization
 Discrete (combinatorial) case : Ising model
 Quadratic case : P=2
 General functional : P=1 , P>2
Exc#2: Pointwise relaxation for P=1
 Minimize
( x )   aij | xi  xj |
ij
 Pick a variable
 Minimize
xi , fix all xj j  i at
~
( xi )   aij | xi  x j |
j
 Find the optimal location for
xi
~xj