ECE531 Screencast 8.5: Randomized Decision Rules
ECE531 Screencast 8.5: Randomized Decision Rules
D. Richard Brown III
Worcester Polytechnic Institute
Worcester Polytechnic Institute
D. Richard Brown III
1/7
ECE531 Screencast 8.5: Randomized Decision Rules
Randomized Decision Rules
◮
◮
◮
◮
◮
So far, we have focused on deterministic decision rules. Given an
observation y ∈ Y, a deterministic decision rule is a map from Y
directly to Z (the indices of the hypotheses).
A generalization of this idea is a randomized decision rule. Given
an observation y ∈ Y, a randomized decision rule is a mapping from
Y to a distribution (a pmf) on Z. The set of valid pmfs on Z is
denoted as PM .
Examples of random decision matrices:
0.9 0.9 0.1 0.1
0.5 0.5 0.5 0.5
D=
or D =
0.1 0.1 0.9 0.9
0.5 0.5 0.5 0.5
Note that the elements of D must be non-negative and the columns
must sum to one.
Also note that the deterministic decision rules are special cases in the
family of randomized decision rules D.
Worcester Polytechnic Institute
D. Richard Brown III
2/7
ECE531 Screencast 8.5: Randomized Decision Rules
Achievable Conditional Risk Vectors
1
As D ranges over all possible
decision rules in D, R(D)
traces out a set Q of
achievable conditional risk
vectors.
0.9
0.8
0.7
R1
0.6
Q is the convex hull of the
conditional risk vectors of the
deterministic decision rules.
0.5
0.4
0.3
If |Y | is finite, then Q is a
closed and compact polytope
in RN .
0.2
0.1
0
0
0.1
0.2
0.3
0.4
Worcester Polytechnic Institute
0.5
R0
0.6
0.7
0.8
0.9
1
D. Richard Brown III
3/7
ECE531 Screencast 8.5: Randomized Decision Rules
Working Example: Risk Vectors [q0 = 0.5 and q1 = 0.8]
D15
1
◮
Can we now balance the
risk R0 = R1 = 0.4?
◮
What does the line
R0 + R1 = 1 represent?
Random guessing.
◮
Where are the “good”
decision rules?
Southwest of the
random guess line.
◮
What point on the
Southwest boundary of
Q corresponds to the
best decision rule?
0.9
0.8
0.7
R1
0.6
D14
0.5
0.4
0.3
0.2
D12
0.1
D8
D0
0
0
0.1
0.2
0.3
0.4
Worcester Polytechnic Institute
0.5
R0
0.6
0.7
0.8
0.9
1
D. Richard Brown III
4/7
ECE531 Screencast 8.5: Randomized Decision Rules
Optimal Tradeoff Surface of Q
The optimal tradeoff surface of Q is the set of all R(D) for D Pareto optimal.
Any “best” decision rule must have a CRV on this optimal tradeoff surface.
D15
1
0.9
0.8
0.7
R1
0.6
D14
0.5
0.4
0.3
0.2
D12
0.1
D8
D0
0
0
Worcester Polytechnic Institute
0.1
0.2
0.3
0.4
0.5
R0
0.6
D. Richard Brown III
0.7
0.8
0.9
1
5/7
ECE531 Screencast 8.5: Randomized Decision Rules
Specifying a Unique Decision Rule
Note that the optimal tradeoff surface does not specify a unique best
decision rule. An additional criterion is needed.
1. Neyman Pearson criterion: Find D that minimizes R1 (D) subject
to an upper bound on R0 (D).
2. Bayes criterion: Fix some λ ∈ [0, 1] and define the weighted Bayes
risk r(D, λ) = (1 − λ)R0 (D) + λ(R1 (D)). Find D that minimizes
r(D, λ).
3. Minimax criterion: Find D that minimizes max{R0 (D), R1 (D)}.
Worcester Polytechnic Institute
D. Richard Brown III
6/7
ECE531 Screencast 8.5: Randomized Decision Rules
Working Example: Risk Vectors [q0 = 0.5 and q1 = 0.8]
D15
1
0.9
0.8
0.7
0.6
R1
NP CRV R0<=0.1
D14
0.5
0.4
minimax CRV
0.3
0.2
D12
Bayes CRV λ=0.6
0.1
D8
D0
0
0
Worcester Polytechnic Institute
0.1
0.2
0.3
0.4
0.5
R0
0.6
D. Richard Brown III
0.7
0.8
0.9
1
7/7