EP550: Utility Assessment
The general term that is used to refer to outcom e values is "utility."
The im m ediate surgical m ortality is very low, which produces the
following survival curves.
The decision tree that follows describes the decision faced by the
som e patients with newly diagnosed cancer of the larynx, because both
surgery and radiation therapy are reasonable alternatives.
For patients with this stage of laryngeal cancer, surgery leads to
better short and long-term survival. If patients were to decide on survival
alone, all would choose surgery over radiotherapy. However, surgery
leaves the patient with a tracheostom y, which is disfiguring and lim its
som e activities such as swim m ing and bathing. More im portantly, after
surgery the patient m ust learn to speak without a larynx. Even for those
who are able to learn how to do this, the result is not always satisfactory.
Because survival is not the only outcom e that m ight influence the
patient's choice of therapy, the decision tree probably should be revised.
2
Different patients will have different opinions about the trade-off
between longer survival, which requires a tracheostom y and artificial
speech, and slightly shorter survival with norm al anatom y and norm al
speech. In the decision tree that describes this problem , therefore, som e
way m ust be found to describe the values of the three outcom es--death,
norm al anatom y with norm al speech, and abnorm al anatom y with artificial
speech--on a com m on scale so the treatm ent choices can be com pared
with each other. Deriving utilities for each outcom e is the solution to this
problem .
The following hypothetical survival curves illustrate the second
problem that requires utilities.
3
The im m ediate surgical m ortality is high, which explains the lower
survival for surgically-treated patients in the first few years. Surgery is
m ore effective than radiation therapy, however, which explains the lower
survival for patients treated with radiation therapy in later years. This
differential survival exactly balances out at three years.
Even though there is no difference in three-year survival, patients
m ay have strong preferences for one therapy or the other depending on
what they think of the trade-off between early and late survival. For
exam ple, m any patients will place a greater value on early survival and
thus will choose radiation therapy. Other patients who are willing to
accept the increased risk of early death for a greater chance of longer life
will choose surgical therapy. These differences illustrate a com m on
phenom enon--patients change their value for a year of life depending on
4
when the year will occur. Moreover, even though m ost patients place
greater value on early years, som e place greater value on later years,
and m ost im portantly, each patient has a different value for this trade-off.
The problem here is to m easure each patient's change in value for early
versus late years of life. Solving this problem requires that the patient’s
utilities for survival be m easured for different periods.
These utility values can be used to plot curves describing the changes in
utilities over tim e.
M ethods for Deriving Utilities
The Rank-and-Scale M ethod
The sim plest way to derive utilities is to ask patients for these values
directly. The rank-and-scale m ethod is the m ethod that is used m ost
com m only.
To use this m ethod, the analyst first m ust identify the outcom es,
usually after drawing the decision tree. Second, the patient ranks the
outcom es from best to worst. Third, the analyst defines the scale range,
which typically is from zero to one or zero to 100, but any scale can be
used. Fourth, the patient assigns the best outcom e to the best scale
value, and the worst outcom e to the worst scale value. The patient then
assigns scale values to the interm ediately-ranked outcom es.
The values that follow on this slide represent one set of possible
responses from a patient with newly diagnosed laryngeal cancer.
Rank
1
Value
100
Outcom e
25-year survival with norm al anatom y and norm al
speech
2
65
25-year survival with tracheostom y and artificial speech
3
58
10-year survival with norm al anatom y and norm al
speech
4
50
5
0
10-year survival with tracheostom y and artificial speech
Death is given the value of zero. 25-year survival with norm al
anatom y is given the value of 100. 25-year survival with abnorm al
anatom y is given the value of 70. Points corresponding to 10-year
survival have been graphed at 50 and 70. Note that longer survival is
better than shorter survival regardless of the health state. Norm al speech
is better than artificial speech, regardless of whether survival is long or
short. Moreover, the values are not proportional to the num ber of years
of survival.
Death
5
6
W e now have all the inform ation needed to recom m end surgery or
radiation therapy for the patient with stage T3 carcinom a of the larynx.
W hat rem ains is to calculate the expected utilities of the two choices,
which requires only the probability of survival and the utility of survival
each year for each therapy.
patient to choose between two outcom es, which are described in the
following decision tree.
For each therapy, we can m easure the probability of survival each
year from the survival curves that were presented earlier (after extending
the graph out to 25 years). Also for each therapy, we can m easure the
patient's utility for each year from the graph of utilities that was just
constructed. To calculate the expected utility for each therapy, m ultiply
the probability of survival for each year tim es the corresponding utility of
survival for each year and sum over all years. The therapy with the
greater cum ulative expected utility would be the preferred therapy.
SUMMARY OF THE RANK-AND-SCALE METHOD
1.
Identify the outcom es
2.
Rank the outcom es
3.
Define the scale range and units
4.
Anchor each end of the scale with an outcom e
5.
Assign scale values to the interm ediate outcom es
6.
Check to m ake sure the ranks and values are com patible
The Standard-Gamble M ethod
If the patient chooses the upper outcom e, he gets it for certain. If he
chooses the lower outcom e, he m ust participate in a gam ble with an
equal chance of getting either of two options. The patient m ust choose
either the certain outcom e or the gam ble.
The rank-and-scale m ethod is often used because it is easy to do. It
does not, however, m eet the theoretical standards for utility assessm ent
because there is no risk in the options, as there is in real life. The
standard-gam ble m ethod does m eet theoretical standards, because it
involves a gam ble and thus incorporates the patient's value for
uncertainty in the utility m easurem ent. This m ethod involves asking the
In the following exam ple, patients were asked to im agine a situation
that involved an interm ediate period of certain survival (on the top) versus
a gam ble (on the bottom ) with a 50:50 chance of longer survival and a
50:50 chance of early death. In the gam ble, patients were asked to
im agine a situation in which a coin would be flipped; "heads" would m ean
they would have long-term survival, while "tails" would m ean their survival
7
8
would be greatly shortened. The patient was asked to choose between
taking the gam ble, with a chance for long-term survival, or settling for an
interm ediate period of guaranteed survival. Specifically, the patient was
asked to specify the period of certain survival that would be equivalent to
the gam ble.
The patient was asked for the num ber of years of certain survival that
would m ake him indifferent to the choice between the gam ble and the
certain outcom e. The question m ight be phrased som ething like this: “To
avoid the gam ble in which you have an equal chance of living a few
m onths or 25 years, what is the fewest years of certain survival you
would accept?"
9
Assum e the answer is seven years. This patient would settle for a
certain survival of seven years rather than accept the gam ble, which has
an average survival of (25 + 0) / 2, or 12.5 years. Because the gam ble is
worth an average of 12.5 years, this patient would be willing to trade the
difference between an uncertain 12.5 years and a certain 7 years to avoid
the risk of gam bling. W ith this inform ation, we can calculate the utility of
the 7 years.
Assum e that death in a few m onths is the worst outcom e with a utility
of zero and that 25-year survival is the best outcom e with a utility of 100.
The first step is to calculate the "expected" utility of the gam ble, which is
50. Rem em ber that because of the way the question was posed, the
patient is indifferent between the upper branch, with its certain outcom e,
and the lower branch, with its gam ble. As far as he is concerned, one is
10
SUMMARY OF THE STANDARD GAMBLE METHOD
as bad as the other. This m eans that the utilities of the two branches are
equal, and thus that the utility of the certain outcom e also is 50.
1.
Explain that the choice is between a certain outcom e and a gam ble
2.
Define the probabilities of the gam ble
3.
Define the best outcom e, and m ake it part of the gam ble
4.
Define the worst outcom e, and m ake it part of the gam ble
5.
Ask for a certain interm ediate outcom e that is equivalent to the
gam ble
Alternatively, the analyst can specify the certain outcom e and ask the
patient for the probabilities of the gam ble that would m ake the choices
equivalent. For exam ple, the analyst m ight specify, as before, that the
gam ble is between im m ediate death and 25 years of life in norm al health
but add that the certain outcom e is 25 years with a tracheostom y. The
analyst then could specify a sequence of different probabilities until the
patient identified the ones that would m ake the gam ble equivalent in
value to the certain outcom e.
Once the utility is revealed with the standard gam ble, it can be used
to exam ine the value for early versus later years of life.
11
12
therapy that would guarantee short-term survival, even if it were less
effective in the long run.
The patient described on the following graph has just the opposite
utilities.
The diagonal line describes a hypothetical patient who has exactly
the sam e value for early and later years of life. The utility increases at
the sam e rate for each additional year of survival throughout the entire 25
years. For the patient whose points are plotted, however, the utilities for
early years of life are greater than the utilities for later years of life. This
patient's curve rises rapidly in early years but levels off in later years. In
general, when the curve lies above the horizontal line, the patient places
a higher value on early than on later years of life. Such patients are "risk
averse"- they prefer to avoid the risk of early death in the gam ble and are
willing to settle for a certain, but shorter, survival. Such a patient would
tend to avoid risky therapy because of the gam ble with early death, even
if it were m ore effective in the long run. This patient would settle for a
13
This patient's curve rises slowly in early years, but m ore rapidly in later
years. In general, when the curve lies below the diagonal line, the patient
places a higher value on later than on early years. Such patients are "risk
seeking"- they prefer to gam ble. They prefer to accept the risk of early
death in exchange for a greater chance of long-term survival.
Time-Tradeoff M ethod
14
In the tim e-tradeoff m ethod, the patient is asked to im agine that he
can choose between survival with norm al anatom y and survival with a
tracheostom y. Further, he is asked to estim ate the num ber of years in
one state that would equal a given num ber years in the other state. For
exam ple, the patient is asked to estim ate the num ber of years with
norm al anatom y that would equal 25 years with a tracheostom y.
Because m ost people prefer norm al anatom y over a tracheostom y, m ost
people would be willing to live a shorter period of tim e if it m eant they
would have norm al anatom y instead of a tracheostom y.
SUMMARY OF THE TIME-TRADEOFF METHOD
1. The analyst defines the outcom es.
2. The analyst selects a tim e period in worse health for com parison
3. For each tim e period, the patient determ ines the duration of better
health that is equivalent to the tim e period in worse health.
Assum e a patient is willing to live 12.5 years with norm al anatom y in
exchange for 25 years with a tracheostom y. Also, assum e he is willing to
live 7 years with norm al anatom y in exchange for 10 years with a
tracheostom y.
Because this patient has the sam e utility for 7 years with norm al anatom y
as he does for 10 years with tracheostom y, we can look on the horizontal
axis for 7 years with norm al anatom y (arrow) and read from the vertical
axis his utility for 10 years with a tracheostom y, which is about 50.
Sim ilarly, because he has the sam e utility for 12.5 years with norm al
anatom y as he does for 25 years with a tracheostom y., we can look on
the horizontal axis for 12.5 years with norm al anatom y and read on the
vertical axis his utility for 25 years with a tracheostom y, which is about
65. This gives us two new points that describe his utilities for survival
with a tracheostom y. Using these points, we can plot his utility curve for
survival with a tracheostom y, as follows.
15
16
graph of utilities that was just created. To calculate the expected utility
for each therapy, m ultiply the probability of survival for each year tim es
the corresponding utility of survival for each year and sum over all years.
The therapy with the greater cum ulative expected utility would be the
preferred therapy.
The standard-gam ble and the tim e-tradeoff m ethods were used by
other researchers to derive utilities from 25 executives and 12 firefighters.
These utilities were used to calculate the preferred therapy for each of
the 37 people. The preferred therapy was radiation for 16 of them and
surgery for the rem aining 21. The authors concluded that, "For m ost
patients, a willingness to trade off at least 15 to 30 per cent of their full life
expectancy to preserve speech should m ake the patient and physician
consider the possibility of radiation therapy instead of surgery."
To sum m arize, we have used the rank-and-scale m ethod alone to
derive utilities for the choice between radiation and surgical therapy for
patients with cancer of the larynx. W e also have used a com bination of
the standard gam ble and the tim e-tradeoff m ethod to derive the sam e
utilities. In this exam ple, the utilities derived using the different m ethods
were alm ost identical, although this is not always the case.
QALYs and QALE
On this graph the lower line represents the values for tracheostom y
with abnorm al speech. 10 years with a tracheostom y has the sam e utility
as 7 years with norm al speech. Also, 25 years with a tracheostom y has
the sam e utility as 12.5 years with norm al speech
W e now have all the inform ation needed to recom m end surgery or
radiation therapy for this patient with stage T3 carcinom a of the larynx.
W hat rem ains is to calculate the expected utilities of the two choices,
which requires only the probability of survival and the utility of survival
each year for each therapy For each therapy, we can m easure the
probability of survival each year from the sam e survival curves that were
presented earlier (after extending the graph out to 25 years). Also for
each therapy, we can m easure the patient's utility for each year from the
17
In other articles you read, you will find that utilities are described as
"Quality-Adjusted Life Years" or QALYs. Also, you will find descriptions
of "Quality-Adjusted Life Expectancy" or QALE. These term s refer sim ply
to the process of adjusting norm al survival to represent the lower value
that patients assign to other health states. In the exam ples we have just
com pleted, we have adjusted the value of norm al survival to represent
the lower value that m ost patients assign to tracheostom y.
Problems with Utility Assessment
As you m ay have concluded already, utilities are the m ost
troublesom e part of decision analysis. W e do not know enough about
them . How reproducible are they when derived by different analysts?
Can the analyst lead the patient to give utilities which reflect the analyst's
bias? How sim ilar are utilities when derived by different m ethods? Can
18
patients understand what it m eans to experience tracheostom y, or
com plicated surgery or im paired health states well enough to assign
values to them without actually experiencing them ? How stable are
patients' utilities over tim e? If utilities are not stable, should we use them
to advise patients about the im portant decisions they are facing?
preferred to avoid anesthesia, but as labor progressed their values
changed drastically. A sim ilar, but less drastic, change occurred in the
wom en who had undergone a previous delivery. Note also that the
utilities of both groups of wom en returned to approxim ately the original
values one m onth after the delivery.
The authors of this study point out the problem that these changes
pose for the physician who wants to help a wom an intelligently choose
whether or not to use anesthesia during labor. Should the physician use
the utilities that the wom an gives before labor begins, which is the utility
she likely will give after labor is over? Or should the physician use the
utilities that she gives during labor, which are different than the ones she
nave before labor and likely will be different from the ones she will give
after labor?
Other exam ples can be used to show that changes in the language
used to describe outcom es can influence how the patient values the
outcom es. This is troublesom e because it shows that the language
chosen by the analyst can chance the utilities that are derived from the
patient, no m atter which m ethod is used to derive utilities. W e do not
know which descriptions are better suited to deriving "true" utilities, and
therefore, there is little guidance about to phrase these descriptions.
In sum m ary, there are m any unanswered questions about the
use of utilities in decision analysis and evidence that utilities provide
m isleading inform ation even when derived by the unbiased expert who is
trying hard. Nevertheless, for m ost problem s that are suitable for
decision analysis, utilities m ust be used.
Com pensation Strategies
Slide 30 describes the utilities for avoiding pain with anesthesia of 10
wom en undergoing their first pregnancy and of 8 wom en undergoing a
later pregnancy. The horizontal axis m easures the stage of labor and
includes points one m onth before and one m onth after labor. The vertical
axis m easures utilities on a scale from m inus 100, which represents the
lowest value, to plus 100, the highest value. A m odification of the rankand-scale m ethod was used to derive each patient's utilities. As you can
see, before labor began wom en undergoing their first pregnancy strongly
19
Recognizing that utilities are problem atic, there nevertheless are
ways to m inim ize those problem s.
First, in som e situations decision analysis should not be attem pted
because it is not feasible to m easure the patient's utilities convincingly-for exam ple, when the patient is m entally im paired from pain or drugs or
the underlying disease. In other situations, the patient m ay be com petent
but incapable or unwilling to participate in the unusual and abstract tasks
20
that are required. In a few of these situations, it m ay be appropriate to
substitute the physician's or the fam ily's utilities, but only if the substitute
knows the patient's preferences in unusual detail.
The second option involves creating a realistic decision tree that
does not defend on utilities. Perhaps the m ost com m on exam ple is when
all the relevant outcom es are different rates survival rates, which can be
com pared without converting them to utilities.
The third option involves identifying and resolving inconsistencies as
the utilities are being derived. For exam ple, in the rank-and-scale
m ethod it is im portant to check that the patient's scale values for the
outcom es are in the sam e order as his rank values for the sam e
outcom es. Also, it is appropriate for the analyst to challenge the patient's
utilities when there is a possibility of m isunderstanding, for exam ple when
the possibility says he would prefer to die quickly after a m ajor stroke but
would possibility a possibility life after a m inor stroke. In this case, the
possibility m ay understand that paralysis and loss of higher brain
functions are potential consequences of a m ajor stroke and actually m ay
possibility to die early rather than suffering these outcom es. The
possibility also m ay prefer to live with these disabilities and m ay have
m isunderstood the analyst.
References
1. McNeil BJ, W eischelbaum RI Pauker SG. Speech and survival:
tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J
Med. 1981; 305:982-987.
2. Christensen-Szalanski JJJ. Discount functions and the m easurem ent
of patients' values: wom en's decisions during childbirth. Med Decis
Making. 1984; 4:47-58.
The fourth option uses m ore than one m ethod to derive the patient's
utilities. W hen different m ethods yield approxim ately the sam e values, as
they did in the exam ple, one can be m ore confident that the values are
correct. Another exam ple of this strategy is to use different descriptions
to explain the sam e outcom e to the patient, regardless of the m ethod
used. This would detect any bias introduced by the choice of language.
Still another strategy would be to have m ore than one analyst derive the
patient's utilities, which would detect any changes introduced by either
analyst's underlying bias.
The m ost useful com pensation strategy is to use sensitivity analyses.
The basic idea is to assum e that the derived utilities m ay not be precise
and then to determ ine if the decision would be changed when different
utilities are used. If the decision is not changed when m any different
utilities are used, one can be m ore confident about the results of decision
analysis.
21
22