Artificial Intelligence
1
1
Practice Exercises.
decision making
Probability, utility and
Exercise 1. Consider a set of n = 2 weather stations. Your prior belief that
the i-th station is correct is P (Hi ) = 1/n. You assume that only one station is
the correct one, i.e. that P (Hi ∩ Hj ) = 0 for any two different stations i 6= j.
Now assume that the stations are making the following predictions for the
next two days. Let A denote the event of rain on Saturday. Let B denote the
event of rain on Sunday. The first station predicts that there will be rain on
Saturday with probability 10%, i.e. P (A | H1 ) = 0.1 and rain on Sunday with
probability 50%, .e. P (B | H1 ) = 0.5. The second station that there will be rain
with probability on Saturday with probability 20% and on Sunday with probability
20% again, i.e. P (A | H2 ) = 0.2 and P (B | H2 ) = 0.2.
1. What is the marginal probability of rain on Saturday, P (A)?
2. What about on Sunday, P (B)?
Solution to exercise 1. We use the definition of the marginal probability, and
the fact that H1 , H2 are disjoint, i.e. that either one of the two stations will be
the right one, but not both at once.
1. For this case we need to calculate:
P (A) =
n
X
i=1
P (A ∩ Hi ) =
n
X
P (A | Hi )P (Hi )
i=1
Replacing n = 2, P (Hi ) = 1/n and the values for P (A | Hi ), we get P (A) =
(0.1 + 0.2)/2 = 0.15.
2. We can repeat the same procedure for B, where we get that P (B) =
(0.5 + 0.2)/2 = 0.35.
Exercise 2. Assume that you need to travel over the weekend. You wish to
decide whether to take the train or take the car. Assume that the train and car
trip cost exactly the same amount of money. The train trip takes 2 hours. If
it does not rain, then the car trip takes 1.5 hour. However, if it rains the road
becomes both more slippery and more crowded and so the average trip time is
2.5 hours. Assume that your utility function is equal to the negative amount of
time spent travelling: U (t) = −t.
1. Let it be Friday. What is the expected utility of taking the car on Sunday?
What is the expected utility of taking the train on Sunday? What is the
Bayes-optimal decision, assuming you will travel on Sunday?
2. Let it be a rainy Saturday, i.e. that A holds. What is your posterior
probability over the two weather stations, given that it has rained, i.e.
P (Hi | A)? What is the new marginal probability of rain on Sunday, i.e.
P (B | A)? What is now the expected utility of taking the car versus taking
the train on Sunday? What is the Bayes-optimal decision?
Solution to exercise 2. The time in this case depends on two variables: whether
we take the car, and whether it rains. Consequently, the probability of the
trip taking t, for the decision set D = {Car, Train} is given by the following
equations
2
Practice exercises
The distribution for taking the train is:
(
1, t = 2
PTrain (t) =
0, otherwise.
(1.1)
This means that the train time will two hours, with probability 1. So no other
time is possible.


1 − P (B), t = 1.5
PCar (t) = P (B),
t = 2.5


0,
otherwise.
(1.2)
This is because the time taken when we drive the car only depends on B, whether
it rains on Sunday.
Now we can directly calculate the expected utility of taking the train, which
is
ETrain U = U (−2) × 1 = −2.
(1.3)
The expected utility for taking the car is
ECar U = U (−2.5) × P (B) + U (−1.5) × [1 − P (B)]
(1.4)
= −2.5 × 0.35 − 1.5 × 0.65 = −1.85.
(1.5)
Consequently, taking the car is the Bayes-optimal decision.
If A is true, then we need to calculate P (Hi | A). From Bayes theorem,
P (Hi | A) =
P (A | Hi )P (Hi )
.
P (A)
(1.6)
We have already calculated P (A), and the other terms are known, so
P (H1 | A) = 0.1 × 0.5/0.15 = 1/3
(1.7)
P (H2 | A) = 0.2 × 0.5/0.15 = 2/3
(1.8)
Now the marginal probability of rain on Sunday given rain on Saturday is
X
X
P (B | A) =
P (B | Hi ∩ A)P (Hi | A) =
P (B | Hi )P (Hi | A)
(1.9)
i
i
where P (B | Hi ∩ A) = P (B | Hi ) because the stations made a fixed prediction,
which did not depend on whether or not it rained on Saturday. Replacing, gives
us
P (B | A) = 0.5 × 1/3 + 0.2 × 2/3 = 0.3.
(1.10)
Correspondingly the probability of no rain is P (B { | A) = 1 − P (B | A) = 0.7.
The expected utility of taking the car is therefore
ECar (U | A) = ECar (U | B)P (B | A) + ECar (U | B { )[1 − P (B | A)]
= −2.5 × 0.3 − 1.5 × 0.7 = −1.8.
(1.11)
(1.12)
The expected utility of taking the train is of course unchanged, so taking the
car is still the best option.
Artificial Intelligence
3
Exercise 3. It is possible for the utility function to be nonlinear.
1. One example is U (t) = 1/t, which is a convex utility function. How would
you interpret the utility in that case? Without performing the calculations,
can you tell in advance whether your optimal decision can change? Verify
your answer by calculating the expected utility of the two possible choices.
2. How would you model a problem where the objective involves arriving in
time for a particular appointment?
Solution to exercise 3.
1. The utility can be intepreted as proportional to
the average speed of travel. Since the utility is convex, this implies risk
taking. Consequently, the attractiveness of taking the car would generally
increase. To verify this, the expected utility of taking the train is
ETrain U = U (2) = 1/2.
The expected utility of taking the car is
ECar U = P (B)U (2.5) + [1 − P (B)]U (1.5)
= 0.35/2.5 + 0.65/1.5 = 0.573.
(1.13)
2. This can be done in two ways. One idea is to just define an event for begin
late, and have the utility function depend on that event. But, since all
is relative, and you really don’t want to be too late, but you don’t really
want to wait too long at the rendez-vous point, perhaps you need a utility
function which reaches a maximum at around 0 delay, and sharply drops
off afterwards.
Exercise 4. Consider the general case of n stations. At the end of each day t,
you observe the i-th station’s predictions for the probability of rain the next day:
xt+1,i , P (yt+1 | Hi , y1 , . . . , yt ). Let xt = (xt,1 , . . . , xt,n ) denote the vector of
probabilities. You also observe yt , whether it rained or not that day.
Finally, at time t = 0, you have a prior belief p0,i = P (Hi ) that the i-th
station is correct.
1. Write a program, such that, at each time-step t and given weather history
y1 , . . . , yt and the predictions of all stations until that time: x1 , . . . , xt , xt+1 ,
it returns:
ˆ The posterior probability that each station is correct:
pt,i , P (Hi | y1 , . . . , yt )
P (yt | Hi , y1 , . . . , yt−1 )P (Hi | y1 , . . . , yt−1 )
P (yt+1 | y1 , . . . , yt )
xt,i pt−1,i
= >
,
xt pt−1
=
(1.14)
(1.15)
(1.16)
where pt−1 = (pt−1,1 , . . . , pt−1,n ) is the vector of priro probabilities.
4
Practice exercises
ˆ The marginal probability of rain the following day:
X
P (yt+1 | y1 , . . . , yt ) =
P (yt+1 | Hi , y1 , . . . , yt )P (Hi | y1 , . . . , yt )
i
(1.17)
=
x>
t+1 pt
(1.18)
ˆ Generate weather data so that it agrees with the predictions of one of
the stations, i.e. so that the probability of rain is what that station
says. How fast does your posterior converge?
ˆ Now generate weather data arbitrarily, so that there is no “right”
weather station. How does the posterior behave over time?