Combinatorial Dominance Analysis
The Knapsack Problem
Presented by:
Keywords:
Combinatorial Dominance (CD)
Domination number/ratio (domn, domr)
Yochai Twitto
Knapsack (KP)
Incremental Insertion (II)
Local Exchange (LE)
PTAS
Optimal Head - Greedy Tail (GRT)
1
Overview

Background


Combinatorial Dominance




What is it ?
Definitions & Notations.
The Knapsack Problem
simple Algorithms & Analysis





On approximations and approximation ratio.
Incremental Insertion
Local Exchange
PTASing
“Optimal head - greedy tail” algorithm
Summary
2
Overview

Background


Combinatorial Dominance




What is it ?
Definitions & Notations.
The Knapsack Problem
simple Algorithms & Analysis





On approximations and approximation ratio.
Incremental Insertion
Local Exchange
PTASing
“Optimal head - greedy tail” algorithm
Summary
3
Background

NP complexity class.

AA and quality of approximations.

The classical approximation ratio analysis.
4
NP

If P ≠ NP, then finding the optimum of
NP-hard problem is difficult.
If P = NP, P would encompass the NP and NP-Complete areas.
5
Approximations


So we are satisfied
with an approximate
solution.
OPT
Near optimal
Question:

How can we measure
the solution quality ?
Infeasible
Solutions
quality line
6
Solution Quality


Most of the time, naturally derived from
the problem definition.
If not, it should be given as external
information.
7
The classical
Approximation Ratio
(For maximization problem)


Assume 0 ≤ β ≤ 1.
A.r. ≥ β if

OPT
Near optimal
½ OPT
the solution quality is
greater than β·OPT
Infeasible
Solutions
quality line
8
Overview

Background


Combinatorial Dominance




What is it ?
Definitions & Notations.
The Knapsack Problem
simple Algorithms & Analysis





On approximations and approximation ratio.
Incremental Insertion
Local Exchange
PTASing
“Optimal head - greedy tail” algorithm
Summary
9
Combinatorial Dominance

What is a “combinatorial dominance
guarantee” ?
Why do we need such guarantees ?

Definitions and notations.

10
What is a
“combinatorial dominance
guarantee” ?

A letter of reference:




“She is half as good as I am, but I am the best in
the world…”
“she finished first in my class of 75 students…”
The former is akin to an approximation ratio.
The latter to combinatorial dominance
guarantee.
11
What is a
“combinatorial dominance
guarantee” ? (cont.)

We can ask:
Is the returned solution
guaranteed to be always
in the top O(n) best
solutions ?
OPT
top O(n)
Near optimal
Infeasible
Solutions
quality line
12
Why do we need that ?


Assume an problem for which all
solutions are at least a half as good as
optimal solution.
Then, 2-factor approximating the
problem is meaningless.
13
Corollary


The approximation ratio analysis gives
us only a partial insight of the
performance of the algorithm.
Dominance analysis makes the picture
fuller.
14
Definitions & Notations

Domination number: domn

Domination ratio: domr
15
Domination Number: domn



Let P be a CO problem.
Let A be an approximation for P .
For an instance I of P, the domination
number domn(I, A) of A on I is the
number of feasible solutions of I that
are not better than the solution found
by A.
16
domn (example)

STSP on 5 vertices.


There exist 12 tours
If A returns a tour of length 7
then domn(I, A) = 8
4, 5, 5, 6, 7, 9, 9, 11, 11, 12, 14, 14
(tours lengths)
17
Domination Number: domn



Let P be a CO problem.
Let A be an approximation for P .
For any size n of P, the domination
number domn(P, n, A) of an
approximation A for P is the minimum
of domn(I, A) over all instances I of P
of size n.
18
Domination Ratio: domr




Let P be a CO problem.
Let A be an approximation for P .
Denote by sol(I ) the number of all feasible
solutions of I.
For any size n of P, the domination ratio
domn(P, n, A) of an approximation A for P is
the minimum of domn(I, A) / sol(I ) taken
over all instances I of P of size n.
19
Overview

Background


Combinatorial Dominance




What is it ?
Definitions & Notations.
The Knapsack Problem
simple Algorithms & Analysis





On approximations and approximation ratio.
Incremental Insertion
Local Exchange
PTASing
“Optimal head - greedy tail” algorithm
Summary
20
The Knapsack Problem

Instance:



Multiset of integers S  {w1 , w2 ,, wn }
Capacity T
Find:
arg max W ( S ' )
S ' S
W ( S ') T
where W ( S ' )  wS ' w
21
Simple
Algorithms & Analysis

Incremental Insertion (II)



Arbitrary order
Increasing order
Decreasing order (Greedy)

Local Exchange (LE)

PTASing

“Optimal Head – Greedy Tail” (GRT)
22
II – Arbitrary Order



Go over the elements (arbitrary order)
Insert an element if the capacity not
exceeded
Theorem:
domn( KP, II , n)  2
n 1
1
23
domn( KP, II , n)  2
n 1
1
Proof


Suppose the weights are w1  w2    wn
Let S1 be any locally optimal solution
n

wi  T
We may assume 
i 1


Otherwise,
S1  S
S1 is optimal
24
domn( KP, II , n)  2
Proof


S1 : 1
(cont.)
S1C :
45
23
n 1
1
10 11 12
6789
Let k be the largest index of a weight
not belonging to S1
W ( S1 )  T  wk
Since
S1 is locally optimal
25
domn( KP, II , n)  2
Proof


1
(cont.)
Denote by L the interval W (S1 ), W (S1 )  wk 
For any solution S 2 not containing wk
 Either W ( S 2 )  L


n 1
Or W (S2 {wk })  L
That is, the number of solutions with
n 1
total weight in L is at most 2  1
26
domn( KP, II , n)  2
Proof



n 1
1
(cont.)
Solutions of weight at least W ( S1 )  wk
are infeasible.
Solution weighted not more than W (S1 )
are not better than S1
domn( KP, II , n)  2
n 1
1
27
domn( KP, II , n)  2
Proof

n 1
1
(cont.)
Blackball instance:
w1  n
w2  w3    wn  1

T  2n  2
II can lead to {w2 , w3 , , wn }

Which is locally optimal
28
domn( KP, II , n)  2
Proof

1
(cont.)
Taking the first item and omitting at
least one of the rest is better.


n 1
Hence
domn( KP, II , n)  2
n 1
1
And we finished...
29
II – Increasing Order

No Gain!

That was our blackball…

In the previous proof.
30
II - Decreasing Order (Greedy)


No drastic gain!
Blackball instance B:
w1  3n
w2  w3  w4  2n
w5  w6    wn  1
T  5n  4
31
II - Decreasing Order (Greedy)

Greedy(B)  {w1 , w5 , w6 ,, wn }


Weight:
4n  4
Any solution taking


Exactly two elements from w2 , w3 , w4
Any of the last n  4 elements
is better!
32
II - Decreasing Order (Greedy)
domn( KP, Greey, n)  2  3  2
n
n4
33
Simple
Algorithms & Analysis

Incremental Insertion (II)



Arbitrary order
Increasing order
Decreasing order (Greedy)

Local Exchange (LE)

PTASing

“Optimal Head – Greedy Tail” (GRT)
34
Local Exchange (LE)


Assume S ' is a solution
Allowed operations:


Insert a new element x to
Exchange x by y
 x belongs to S '
 y not belongs to S '

S'
x<y
35
Local Exchange

Theorem:
domn( KP, LE, n)  2
n 1
n
36
domn( KP, LE, n)  2
n 1
n
Proof


Suppose the weights are w1  w2    wn
Let S1 be any locally optimal solution
n

wi  T
We may assume 
i 1


Otherwise,
S1  S
S1 is optimal
37
domn( KP, LE, n)  2
Proof


S1 : 1
(cont.)
S1C :
45
23
n 1
n
10 11 12
6789
Let k be the largest index of a weight
not belonging to S1
W ( S1 )  T  wk
Since
S1 is locally optimal
38
domn( KP, LE, n)  2
Proof


(cont.)
S1C :
45
23
n
10 11 12
6789
Denote by L the interval W (S1 ), W (S1 )  wk 
For any solution S 2 not containing wk
 Either W ( S )  L
2


S1 : 1
n 1
Or W (S2 {wk })  L
That is, the number of solutions with
n 1
total weight in L is at most 2  1

And there are at least
n 1
2 1
outside
39
domn( KP, LE, n)  2
Proof



Let
S1 : 1
(cont.)
S1C :
45
23
n 1
n
10 11 12
6789
be the number of items belonging to
S1 among the first k -1 items
Let l be the number of items not belonging
to S1 among the first k -1 items
h
How many solution pairs are of weight not
belonging to L ?
40
domn( KP, LE, n)  2
Proof


S1 : 1
(cont.)
S1C :
45
23
We saw that domn  2
n 1
n 1
n
10 11 12
6789
1
All 2  1 solutions obtained by
dispensing of some items from h
And the one obtained from them by
adjoining the k’th item
not belong to the interval
h
41
domn( KP, LE, n)  2
Proof


So
S1 : 1
(cont.)
domn  2
S1C :
n 1
45
23
n 1
n
10 11 12
6789
 1  2 1
h
For each of the l solutions obtained from S1
by adjoining one of the items
of l


Both the obtained solution
And the one obtain by adjoining it the k ’th item
not belong to the interval
42
domn( KP, LE, n)  2
Proof


So
S1 : 1
(cont.)
domn  2
S1C :
n 1
45
23
n 1
n
10 11 12
6789
1  2 1  l
h
Since our solution can not be improved by
local exchange


Each of the n-k solutions obtained by removing
one of the last n-k items not belong to the interval
Adding each of them the k’th item we get
infeasible solutions
43
domn( KP, LE, n)  2
Proof

S1 : 1
(cont.)
S1C :
45
23
n 1
n
10 11 12
6789
So domn( KP, LE , n) 
2
n 1
1  2 1 l  n  k 
2
n 1
1  h 11 l  n  k 
2
n 1
n
h
44
nn1
1
domn(
domn(KP
KP
, LE
, II,,nn))22 n1
Proof

(cont.)
Blackball instance:
w1  n
w2  w3    wn  1

T  2n  3
LE can lead to {w2 , w3 , , wn }

Which is locally optimal
45
nn1
1
domn(
domn(KP
KP
, LE
, II,,nn))22 n1
Proof


(cont.)
Taking the first item and omitting at
least two of the rest is better.
b(n)
Hence:
n
n 1
domn( KP, LE, n)  2  [2  1  (n  1)] 
2

n 1
n
And we finished...
46
Simple
Algorithms & Analysis

Incremental Insertion (II)



Arbitrary order
Increasing order
Decreasing order (Greedy)

Local Exchange (LE)

PTASing

“Optimal Head – Greedy Tail” (GRT)
47
PTASing

There exist a PTAS for Knapsack



That is, it is possible to approximate the
optimal solution to within any factor c >1
In time polynomial in n and 1/(c -1)
We’ll see
n


c

1


n /(c 1)
2
 1  domn( KP, PTASc , n)    c /(c 1)  / n 
 c




48
Theorem 1




Let S be an instance of KP
Denote the weight of optimal solution
by OPT (S )
Assume H is a factor-c approximation
for KP
n /(c 1)
 1  domn( KP, PTASc , n)
Then 2
49
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof


Assume that the elements of optimal
solution S opt  {w1 , w2 , , wk } are
labeled such that wi  wi 1
Let k ’ be the smallest integer such that
k ' 1
k'
OPT ( S )
wi 
  wi

c
i 1
i 1
50
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof

(cont.)
Let C1  {w1 , w2 , , wk ' }
C2  OPT ( S ) \ C1
C3  S \ OPT ( S )

Observe that
n  C1  C2  C3
51
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof


(cont.)
Also note that C2  (c  1) C1
Since


The weight of every element of C1 is not
more than the weight of any element of C2
C1 is a c -approximated solution to
S opt  C1  C2
52
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof


(cont.)


Let D  max C1 , C3
D is minimized for C1  n /(c  1)
Since
C3  n  C1  C2  n  c C1
53
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof

(cont.)
Note that our solution dominate
S opt  C1  C2 united with any of the
C3
2  1 non-empty subsets of C3

Since they are not feasible

Since
S opt is optimal
54
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof

(cont.)
Note that our solution dominate all
C1 1
subset of {w1 , w2 ,, wk '1}
2

Since the weight of each is not more than
OPT ( S ) / c

And our solution weight is at least
OPT ( S ) / c
55
2 n /(c 1)  1  domn( KP, PTASc , n)
Proof

(cont.)
Summing both terms, the number of
solution dominated is
2

C3
1  2
C1 1
Minimizing the left-hand term we get
the result.
56
Theorem 2

For every c >1 there exist a KP
instance S and a solution thereof S1 of
total weight W (S1 )  OPT (S ) / c
dominating only
  c  1 n

  c /(c 1)  / n 
 c




solution.
57
Proof

  c  1 n


domn( KP, PTASc , n)    c /(c 1)  / n 
 c




Blackball instance:
w1  w2  w3    wn  1
T  cn /( c  1)

PTASc can return a solution consisting
of n /(c  1) items
58
Proof


(cont.)
Such solution dominates all solutions
consisting of up to n /(c  1) item
It also dominates all infeasible solution


  c  1 n


domn( KP, PTASc , n)    c /(c 1)  / n 
 c




i.e: solution consisting of more than
items.
T
Those are the only solution it dominates
59
Proof

  c  1 n


domn( KP, PTASc , n)    c /(c 1)  / n 
 c




(cont.)
Hence
domn( KP, PTASc , n) 

n /(c 1)   n 

j 0
n
  
 

 j  j 1 cn /(c 1)   j 
n
Employing Stirling’s formula we obtain
the theorem
60
Simple
Algorithms & Analysis

Incremental Insertion (II)



Arbitrary order
Increasing order
Decreasing order (Greedy)

Local Exchange (LE)

PTASing

“Optimal Head – Greedy Tail” (GRT)
61
Optimal Head – Greedy Tail
62
Optimal Head – Greedy Tail
63
Optimal Head – Greedy Tail
64
Optimal Head – Greedy Tail
65
Optimal Head – Greedy Tail
66
Optimal Head – Greedy Tail
67
Optimal Head – Greedy Tail
68
Optimal Head – Greedy Tail
69
Optimal Head – Greedy Tail
70
Summary
domn( KP, II , n)  2
n 1
1
domn( KP, Greey, n)  2  3  2
n
domn( KP, LE, n)  2
n 1
n4
n
n


c

1


n /(c 1)
2
 1  domn( KP, PTASc , n)    c /(c 1)  / n 
 c






domn( KP, GRT , n)  2 1 1 /  log n
n

71
Combinatorial Dominance Analysis
The Knapsack Problem
72