Quality Indicators
(Binary ε-Indicator)
Santosh Tiwari
Background





Multi-objective optimization ̶ Outcome is an
approximation set.
In a real scenario ̶ Actual pareto-optimal set often
unknown.
Our motive is to compare approximation sets, not
algorithms.
In case of algorithms ̶ multiple runs ̶ distribution
of indicator values need to be considered.
Basic idea ̶ x1 is preferable to x2 if x1 dominates x2.
Performance Evaluation of an Outcome


Quality of an outcome ̶ Quantitative
description of the result (approximation
set) e.g. Convergence, Diversity etc.
Computational resources required ̶
Measured in terms of number of function
evaluations required, running time of
algorithm etc.
Quality Indicators
Three basic types
Unary performance indicators ̶ require only
one approximation set.
 Binary performance indicators ̶ require more
than one approximation set.
 Attainment function approach (conceptually
different) ̶ Estimating the probability of
attaining arbitrary goals in objective space
from multiple approximation sets.

Unary Quality Indicators
(Few Examples)



Convergence metric ̶ average distance of
the approximation set from the efficient
frontier – Actual efficient frontier required.
Hyper-volume measure ̶ volume of the
objective space dominated by an
approximation set.
Diversity metric ̶ chi-square-like deviation
measure.
Limitations of Unary Performance
Indicators





Cannot indicate whether an approximation set A
is better than an approximation set B.
Above statement holds even if a finite
combination of unary indicators are used.
Most unary indicators only infer that an
approximation set A is not worse than B.
Unary measures that can detect A is better than
B are in general restricted in their use.
Binary quality measures overcome all such
limitations.
Binary Quality Indicators
Few Examples




Coverage indicator – fraction of solutions in B
dominated by one or more solutions in A.
Binary ε-indicator (detailed description ahead).
Binary hyper-volume indicator – hyper-volume of
the subspace that is weakly dominated by A but
not by B.
Other indicators e.g. Utility function indicator,
Lines of intersection (uses attainment surface)
etc.
Domination Relation for Objective
Vectors

Weak Domination
a ± a, a ± bb, a ± cc, a ± d , b ± b, b ± d , c ± c, c ± dd , d ± dd



Domination
a b, a c, a
d, b
d, c
d
Strict Domination
a
d
Non-dominated (Incomparable)
b || c
a
d a
d  a ± dd
Approximation set is a set of
incomparable solutions
Domination Relation in
Approximation Sets
A ± B Every objective vector in B is weakly dominated
by at least one member in A.
A B A weakly dominates B but B does not weakly
dominate A.
A B Every objective vector in B is dominated by at
least one member in A.
A
B Every objective vector in B is strictly dominated
by at least one member in A.
Binary ε-Indicator (Definition)

ε-domination (multiplicative)
z ±  z , iff 1  i  n : z   z for a given   0
1

2
1
i
2
i
Binary ε-indicator Iε(A,B)
I ( A, B)  inf z  Bz  A : z ±  z
2

1
1
2

Minimum value of ε (>0) for which every member of an approximation set B is
weakly ε-dominated by at least one member of approximation set A.
Computation of Iε(A,B)
z , z
1
2
1
i
2
i
z
 max
1 i  n z
 z  min  z , z
2
z A
1
1
2
z1  A, z 2  B
z 2  B
I  ( A, B )  max

z2
2
z B
Time Complexity O(n.|A|.|B|)
Algorithm to compute Iε(A,B)



Step 1: Find the ideal point of the combined
sets (A & B).
Step 2: Translate both the approximation sets
such that ideal point is situated at (1, 1, …, 1)
in n-dimensional hyper-space.
Compute
 i, j

 zki 
 max  j  i  1, ..., | A | & j  1, ..., | B |
1 k  n z
 k


1 j | B| 1i | A|

Finally, I ( A, B)  max  min  i , j 