Behavioral
Ecology
The official journal of the
ISBE
International Society for Behavioral Ecology
Behavioral Ecology (2016), 27(5), 1296–1303. doi:10.1093/beheco/arw044
Original Article
A general expression for the reproductive
value of information
Rebecca K. Pike,a John M. McNamara,b and Alasdair I. Houstona
aSchool of Biological Sciences, University of Bristol, Tyndall Avenue, Bristol BS8 1TQ, UK and bSchool
of Mathematics, University of Bristol, University Walk, Bristol BS8 1TW, UK
Received 22 September 2015; revised 2 March 2016; accepted 7 March 2016; Advance Access publication 1 April 2016.
Information transfer and utilization is ubiquitous in nature. Animals can increase their reproductive value by changing their behavior in
light of new information. Previous work has shown that the reproductive value of information can never be negative given an animal
behaves optimally. Statistical decision theory uses Bayes’ theorem as a mathematical tool to model how animals process information
gained from their environment. We use this technique with an optimality model to establish a new expression for the value of information when behavior is chosen from a continuous range of possibilities. Our expression highlights that the value of information is
proportional to the rate of change of behavior with information. We illustrate our approach using the cooperative behavior between a
male and a female raising their common young. We show that the value of knowing about one’s partner can be quantified and establish
the value of information to a member of the pair when the continuous trait is how long to spend caring for their young. However, the
applications of this expression are wider reaching than parental care decisions and can be used to analyze the behavior of individuals
across a variety of species and contexts.
Key words: Bayes’ theorem, flexibility, information, optimality model, parental care.
INTRODUCTION
Animals are constantly bombarded with cues and signals, and
throughout, we will refer to these cues and signals as observations.
We do this without prejudice that observations may or may not be
actively sought by the animal. When we refer to information, we
use this in an informal sense as the receipt of an observation.
An observation can be thought of as a random variable. If there
is no variance in possible observations, then an animal would not
need to take an observation; the animal would know the information they were about to receive. Before an observation is made, how
valuable is the observation?
Stephens (1989) emphasizes to biologists that an observation is
valuable to the individual if it has the possibility of changing the
behavior of the individual. The value of an observation is not
determined by variance in the possible states of the environment
but rather by the variance in behavior as a result of making the
observation (Gould 1974).
Animals are known to make informed decisions based on the
observations of their environment (Maynard Smith and Harper
1995, 2003), and many organisms show remarkable flexibility in
life-history traits on receipt of some observation that correlates
with their environment.
Address correspondence to R.K. Pike. E-mail: [email protected].
© The Author 2016. Published by Oxford University Press on behalf of
the International Society for Behavioral Ecology. All rights reserved. For
permissions, please e-mail: [email protected]
For instance, Daphnia pulex switch from asexual to sexual reproduction depending on population density (Berg et al. 2001).
Empirical studies (Collins et al. 1994; Luttbeg 1996) and theoretical studies (Getty 1996; Mazalov et al. 1996; Collins et al. 2006)
of female mating tactics suggest that assessing potential mates can
improve estimates of the quality distribution and an individual
male’s quality, improving the female’s choice of whether she should
accept or reject him as a mate. In each example, the individual
changes its behavior in response to making an observation.
Animals gather imperfect information to update their opinion about the true state of nature, a process that can be modeled using the quantitative framework of statistical decision
theory (McNamara and Houston 1980; Dall et al. 2005). At the
heart of the approach is Bayes’ theorem that provides a method
of determining probabilities based on observations. A Bayesian
animal behaves as if it has prior knowledge of the state of the
world (from personal experience or genetic information), which
it uses coupled with a new observation to arrive at a revised,
posterior opinion concerning the state of the world. We are not
suggesting animals consciously complete these calculations or
reach their optimal strategy in this way. Instead, observed behavior may approximate theoretically derived optimal solutions. We
use statistical decision theory as a framework to provide a mathematically rigorous way to model the information gain in animals
(McNamara and Houston 1980).
Pike et al. • The value of information
1297
Behavior can be chosen from a continuous (e.g., how long to
spend being vigilant as opposed to foraging) or discrete (e.g., accept
or reject a mate) range of possibilities. In our model, we make the
assumption that an individual can choose from a continuous range
of behaviors. We argue that there are many cases in nature where
it is more realistic to view behavior as a continuous rather than
discrete trait. Vampire bats, Desmodus rotundus, vary the quantity of
a blood meal shared depending on the degree of relatedness and
expected future reciprocation (Wilkinson 1984). Males and females
in various species of birds adjust the amount of time devoted to
incubation or the amount of food brought to nestlings in response
to the efforts of their mates (Nice 1937; Kendeigh 1952; Skutch
1976; Harrison et al. 2009). Allowing varying degrees of cooperation has been an important assumption in the theoretical study of
the evolution of cooperation (Killingback et al. 1999; Killingback
and Doebeli 2002; McNamara and Doodson 2015).
We derive a new expression to quantify the value of information. Following McNamara and Dall (2010), we define the value of
information as the average increase of an individual’s reproductive value if they behaved optimally after an observation compared
with before. They consider a discrete set of behavioral options and
thus the curve of the resulting payoff function contains a “kink.”
We consider behavior chosen from a continuous range of possibilities where the subsequent optimal payoff function is differentiable
everywhere and is thus smooth. A general expression for the value
of information is given for an animal in a world that can be in one
of 2 states. We then consider an animal in a world that can be in
any of n possible states. The value of gaining information about a
partner is explored with a case study involving a male and female
raising their young. The female must make the decision of how
much parental effort to exert given her information on the type of
male she is paired with.
MODEL
Consider a world that can be in one of 2 states. The world is in
state 0 with probability 1 − p and in state 1 with probability p.
Now consider an individual in this world with a continuous trait T.
We assume that the individual is uncertain of the true state of the
world but as a result of selection acting in the past or from personal
experience, the individual behaves as if it knows p (Dall et al. 2005;
McNamara et al. 2006). Let W0(T) and W1(T) denote the payoff to
an individual with trait T when the world is in state 0 and state 1,
respectively. The expected payoff W(T, p) to an individual depends
on the trait T and the prior p and is given by
W (T , p ) = (1 − p )W0 (T ) + pW1 (T ).
(1)
The optimal trait T*(p) that maximizes the expected payoff in
Equation 1 is a function of p and satisfies the derivative condition
∂W *
(T ( p ), p ) = 0.
∂T
(2)
(
(3)
The payoff to an individual that behaves optimally is then
)
W * ( p ) = W T * ( p ), p .
We assume the individual gains information about the true state of
nature from observation or otherwise. This information is coupled
with the individual’s prior knowledge to give an updated posterior
probability p̂ on the state of nature, which is a random variable
because the animal gains information by observing the random
variable X. Allowing X to take this form prevents the animal from
knowing what information the observation will give. If an animal
knew the information, they were about to receive they would not
need to take an observation. In our analysis below, we assume that
whatever the value of pˆ , the individual then behaves optimally
given this value.
We build on the result of McNamara and Dall (2010) who
describe the value of information as the average increase in reproductive value to the individual after gaining information about the
state of the world. This difference is
I (p )=E[W * ( pˆ ) ] − W * ( p ).
(4)
* ˆ
The first term on the right hand side, E[W ( p )], is the expected
payoff to the individual if they gain information by observation
before choosing their optimal action given this information. The
second term on the right hand side, W*(p), is the payoff given the
individual follows the optimal strategy given that the prior knowledge p is available when the individual makes its choice of action.
McNamara and Dall (2010) show that E[ pˆ ] = p ; that is, the
mean of the posterior probability that state 1 is the true state
equals the prior probability that state 1 is the true state. They
also show W* is a convex function of p. Because E[ pˆ ] = p and
due to the convexity of the optimal payoff function W*(p), then
E[W * ( pˆ )] ≥ W * ( p ). Therefore, I(p) is non-negative. We exploit the
fact that the trait is continuous to extend the result from Equation 4
*
by approximating the term E[W ( pˆ )] using the Taylor expansion
to give the equation
E[W * ( pˆ )] = E[W * ( p )] + E[( pˆ − p ) ⋅W * ′ ( p )]
1

+ E  ( pˆ − p )2W * ′′ ( p ) + o(( pˆ − p )2 )
2


(5)
where the last term on the right hand side of Equation 5 represents
terms of order ( pˆ − p )3 and above. The second term on the right
hand side in Equation 5 disappears because E[ pˆ ] = p. Because W*
is a constant, using routine laws of expectations, we can rewrite
Equation 5 as
1
E[W * ( pˆ )] = W * ( p ) + Var( pˆ ) ⋅W * ′′ ( p ),
(6)
2
disregarding terms of order 3 or higher. Therefore, by Equations 4
and 6, we have
1
I ( p ) = Var( pˆ ) ⋅W * ′′ ( p ).
(7)
2
Expression 7 represents the value of information for small variance.
As we can see, 2 terms are important in quantifying the value of
information, the variance of the posterior probability and the curvature of W*(p). The 2nd derivative of the optimal payoff function
can be calculated directly using Equations 1–3 as
W * ′′ ( p ) = W1′ (T * ( p )) − W0′ (T * ( p )) T * ′ ( p )


(8)
(see Appendix 1 for proof). Substituting Equation 8 into Equation
7, we have
1
I ( p ) = Var( pˆ ) W1′ (T * ( p )) − W0′ (T * ( p )) T * ′ ( p ).


2
(9)
Expression 9 is the new result that the value of information is the
product of the variance of the posterior, the difference between the
rate of change of the payoff with trait T for each state and the rate
of change of a trait with information. If the rate of change of T*(p)
is 0, then there is no change to behavior after a gain in information
and there is no value to information. Only when the term T*(p) is
nonzero, is there a change in behavior as a result of information
gain and therefore value to information.
Behavioral Ecology
1298
GENERAL CASE
The 2D model can be generalized to derive an expression for the
value of information when we consider a world that can be in one
of any n distinct states. Let pi be the probability that the true state
of the world is θi (where i = 1,…, n). We assume that an individual
who is uncertain about the true state of the world has the prior
n
knowledge vector p = ( p1 ,…, pn ) where ∑ i =1 pi = 1. Suppose the
individual gains information by observing the random variable X.
The conditional probability that the observed value of X is x given
the true state of the world is θi is denoted by fi(x). Averaging over all
possible values of the world, the unconditional probability that the
observed value of X is x is f ( x ) = p1 f1 ( x ) + ... + pn f n ( x ) . Applying
Bayes’ rule, if the observation is x, the posterior probability that the
true state of the world is θi is
p f (x )
(10)
qi ( x ) = i i
f (x )
and the updated knowledge vector is defined as
q( x ) = (q1 ( x ), …, q n ( x )).
(11)
Observing the random variable X is seen as determining the value
of the random knowledge vector Q = (Q 1 ,…,Q n ) where Q = q(X).
It can be shown that E(Q) = p where E {Q i } = pi (McNamara
and Dall 2010). That is, the mean of the posterior probability for
any possible state of the world θi being the true state of the world
equals the prior probability that it is the true state. We assume an
individual possessing a continuous trait T has reproductive value
Wi(T) when the world is in state θi. Following from Equation 1, the
payoff function of an individual with trait T and knowledge vector
p, representing the individual’s expected reproductive value, is
(12)
W (T , p ) = pW
1 1 (T ) +  + pnWn (T ).
T*(p) denotes the trait that maximizes payoff given that the knowledge vector is p. The expected reproductive value under this optimal trait is
W ( p ) =W (T ( p ), p ) = p1W1 (T ( p )) +… + pnWn (T ( p )). (13)
To proceed, for mathematical convenience, we do not restrict p to
a probability and so we do not assume that ∑ i pi necessarily sums
to 1. The results derived (Expression 21) will therefore hold for all p.
n
More specifically, they will also hold for the case where ∑ i =1 pi = 1 .
*
*
*
*
Given X is a random variable and W*(p) is a constant, E[W*(Q)]
can be approximated by the second order Taylor expansion of W
about p,
*
n ∂W
E[W * (Q )] = W * ( p ) + ∑ i =1
( p )E[(Q i − pi )]
∂pi
n
1 n
∂2W *
+ ∑ i =1 ∑ j =1
( p )E[(Q i − pi )(Q j − p j )] (14)
2
∂pi ∂p j
+o
(∑
n
i =1
∑
n
j =1
)
(Q i − pi )(Q j − p j ) .
Because E {Q i } = pi , then E [(Q i − pi )] = 0 . Thus, the second
term on the right hand side in Equation 14 is equal to 0. Also,
because this is true, we have
∑ ∑
n
n
i =0
j =0
E[(Q i − pi )(Q j − p j )]
= ∑ i = 0 ∑ j = 0 E (Q i − E{Q i })(Q j − E{Q j })
= Cov(Q i ,Q j ),
n
n
(15)
where Cov(Q i ,Q j ) is the covariance between the estimated probabilities of states of the world i and j. Thus, by Equations 14 and
15, we have
E[W * (Q )] = W * ( p ) +
n
∂2W *
1 n
Cov (Q i ,Q j )
( p ) (16)
∑
i =1 ∑ j =1
∂pi ∂p j
2
when terms of order higher than 3 are disregarded. Because the
value of information is the difference between E[W*(Q)] and W*(p)
from Equation 4, we have
n
∂2W *
1 n
I ( X ) = ∑ i =1 ∑ j =1 Cov (Q i ,Q j )
( p ).
(17)
∂pi ∂p j
2
Calculating the 2nd partial derivative in Expression 17 as
∂2W *
∂T *
( p ) = Wi ′ (T * ( p ))
( p)
∂pi ∂p j
∂p j
(18)
gives I(X) in the form
n
∂T *
1 n
I ( X ) = ∑ i =1 ∑ j =1 Cov (Q i ,Q j )Wi ′ (T * ( p ))
( p ).
(19)
∂p j
2
Expression 19 is the new result for the value of information when
the world can be in any one of n distinct states. As in Expression 9,
∂T *
( p ) in Expression 19 is nonzero is there a
only when the term
∂p j
change in behavior as a result of information gain and thus a value
to information. We can write Expression 19 schematically as
1
I ( X ) = W ′T CT * ′
2
(20)
where W ′T is the transpose of the vector W ′ = (W1′ ,…,Wn ′ ), C is
*
*
*
the covariance matrix and T * ′ is the vector T ′ = (T1 ′ ,…,Tn ′ ).
The value of information is therefore the inner product of the rate
of change of the payoff function with the optimal trait T*(p), the
covariance matrix of the random knowledge vector Q and the rate
of change of a trait with information.
Further manipulation (see Appendix 2) gives I (X) in the form
(
′ *
1 Var ∑ i =1Wi (T ( p ))Q i
I (X ) = −
2 ∑ n pW ′′ (T * ( p ))
n
i =1 i
)
(21)
i
Expression 21 is equivalent to Expression 9 when the world can be
in one of 2 distinct states (see Appendix 3). When n = 2, then the
covariance term in Expression 19 is perfectly negatively correlated,
that is − Var( pˆ ) = Cov (1 − pˆ , pˆ ) . Increasing the probability that the
state of the world is in state 1 decreases the probability the world is
in state 2 by exactly the same amount. This is not always the case
when n > 2 because there is more than 1 degree of freedom.
CASE STUDY: OPTIMAL WAIT TIME AT THE
NEST WHEN THE TYPE OF PARTNER IS
UNKNOWN
Parent birds raising young constantly change their behavior in
response to information gained about the effort of their mate
(Skutch 1976; Harrison et al. 2009). With the life history of long
lived, monogamous sea birds (e.g., some albatross species) in mind,
we apply our expression to find the value of information to an individual about their partner.
Consider a male and female pair of a socially monogamous bird
species where incubation and feeding of the young is shared by both
parents. During the breeding season, one parent must be present
with the young at a nest site at all times from laying to fledgling.
Although one parent stays with the young, the other leaves to forage for themselves and their offspring. We assume that the female
Pike et al. • The value of information
1299
takes the first incubation shift and the male leaves to forage. We also
assume a constant high probability of survival. There are 2 types of
male in the population. Type 1 will return to the nest after foraging
whereas type 0 will not. The female is uncertain which type of male
she is paired with. The probability that the female is paired with a
type 1 male is p, and the probability that the female is paired with
a type 0 male is 1 − p. It is assumed the female behaves as if she
knows p. The female possesses a continuous trait T that determines
how long she waits at the nest for the male to return from foraging before abandoning the nest herself. Waiting carries a cost and
so waiting too long could be detrimental to future breeding success.
The female may face increased vulnerability to predators or suffer a reduction of body condition that may affect her reproductive
success in future years. However, abandoning the offspring leads to
complete breeding failure for the current season. The female follows
strategy T ≥ 0 if she abandons the nest at time T. If the male is of
type 1, the female receives a reward value of V if the male returns
before time T and receives a reward value of L if the male returns
after time T. If the male is of type 0, the female receives reward L if
she leaves at time T. If the male fails to return and the female does
not leave, the costs incurred to the female result in her death and the
payoff to the female is 0. We assume that the reward values V and
L satisfy the inequality V > L. We further assume that the female
incurs a cost c per unit of time she waits for the male to return. The
return time of a male of type 1 is described as a continuous random
variable X supported on the interval (0, ∞) and is modeled by the
exponential distribution, X ~ exp(λ), with rate parameter λ > 0.
When the female’s strategy is T, the expected payoff to the
female paired with a type 0 male is
W0 (T ) = L − cT
(22)
and the payoff to a female paired with a type 1 male averaging over
his possible return times is
T
∞
0
T
W1 (T ) = ∫ f X ( x )[V − cx ]dx + ∫ f X ( x )[ L − cT ]dx
(23)
where f X ( x ) = λe − λx . The first term on the right hand side of
Expression 23 represents the payoff to the female if X < T and
the second term represents the payoff to the female if X > T.
Expression 23 can be solved to give
c
c

W1 (T ) = e − λT  L − V +  + V − .
(24)


λ
λ
From Equation 1, the expected payoff to the female who adopts
strategy T given p is L − cT

W (T , p ) = (1 − p )( L − cT ) + p e − λT

c
c

 L − V +  + V −  (25)
λ
λ
There exists a critical value of p that we define as
c
pcrit =
.
(26)
λ (V − L )
The strategy that satisfies the derivative condition of Equation 2
and maximizes Equation 25 is
T * ( p) =
 p   λ (V − L ) − c )  
1
Ln 

  .
c
λ  1 − p  

(27)
This is the optimal wait time of the female given p. Only when p >
pcrit, is T*(p) > 0. Otherwise T*(p) = 0 and the female should abandon the nest immediately. Figure 1a shows how the optimal wait
time depends on p. If the female follows the optimal strategy then
by definition (Equation 3), she receives the optimal payoff W*(p)
(see Figure 1b). From Equations 25 and 27, the 2nd differential of
the optimal payoff function is
c
W * ′′ ( p ) = 2
.
(28)
p (1 − p )λ
As stated above, we assume that the female behaves as if she knows
the prior probability. If the female gains further information about
the type of male she is paired with by observation, the value of this
information to the female can be calculated directly using Equation
28 as
I ( p ) ∝ W * ′′ ( p ),
(29)
because the behavior of the value of information function I(p)
is proportional to the function W * ′′ ( p ) for a given variance.
Figure 1c illustrates the value of information as a function of p.
Unlike models that reward the focal individual for simply “knowing
the truth” (Grafen 1990), the function I(p) values information only
if a gain in knowledge changes the female’s behavior. The value in
information is the reproductive value gained when this information
is used to change behavior. There is no change to the female’s optimal behavior for a small gain in information about her partner’s
type below pcrit and so information is not valuable. When p is strictly
greater than pcrit, the function I(p) is positive, and information is
valuable because the consequence of gained information always
changes the female’s optimal wait time. The function I(p) becomes
most valuable for values of p closest to 1; thus, there is value in
gaining information about your partner even if you know with a
high likelihood which type they are. When the value of information
is dependent on variance in behavior, a greater change to behavior
will render information more valuable (see Figure 1a).
In some contexts, p can be taken as given (when the prior p is
determined from courtship with a specific male) and in others a
self-consistent solution would be required.
Consider a population where females pair with a randomly chosen male at the beginning of each breeding season. Let 40% of the
males in the population be of type 1. Through a period of observation (e.g., courtship), the female gains information about the type
of male she is paired with to give a prior p. We are interested in
how valuable further observation would be given p. In this case, the
reward value L is a constant because the expected future reproductive success of the female will always be the same despite the type
of male the female is paired with in the current breeding season.
For more complex cases, for example, when a female retains her
mate across breeding seasons, her expected future reproductive
success (L) will depend on p. There are also complications when a
female can choose a male from one of multiple populations where
the proportion of type 1 males differ in each population. If p varies, then so will the costs and benefits. In this scenario, the life history of each population would need to be considered to determine
parameter values. In such cases, one would need to consider selfconsistency (Houston and McNamara 2005). A self-consistent version of this model would be a useful direction for future work.
DISCUSSION
Using statistical decision theory techniques and an optimality
model, we establish a new expression to quantify the reproductive
value of information when behavior is continuously variable. Our
formulae highlight that the value of information is determined by
Behavioral Ecology
1300
(a)
18
16
14
c = 0.1
c = 0.3
c = 0.6
c = 0.9
12
T*(p)
10
8
6
4
2
0
0
0.2
0.4
p
0.6
0.8
1
0.6
0.8
1
(b) 4
3.5
c = 0.1
c = 0.3
c = 0.6
c = 0.9
W*(p)
3
2.5
2
1.5
1
0.5
0
0.2
0.4
p
(c) 45
c = 0.1
c = 0.3
c = 0.6
c = 0.9
40
35
W*"(p)
30
25
20
15
10
5
0
0
0.2
0.4
p
0.6
0.8
1
Figure 1.
(a) The optimal wait time against p. The optimal wait time of the female increases with the probability that the male is of type 1. For p less than or equal to
pcrit, the female should abandon the nest immediately. For p above the critical value of pcrit, it is optimal to wait an intermediate amount of time until p = 1
when the female should wait indefinitely. c is the cost incurred per unit of time the female waits for the male to return. (b) Reproductive value given the
optimal strategy against p. Reproductive value, characterized by the optimal payoff function, increases monotonically with p. W*(p) is a constant in the region
0 < p < pcrit as the optimal wait time for the female is always 0. (c) W*(p) against p, which is proportional to the value of information function I(p). W*(p) is
constant and 0 in the region 0 < p < pcrit, and we can say information is not valuable here. W*(p) is positive for p strictly greater than pcrit and information is
valuable. W*(p) tends to infinity as p tends to 1; thus, information is most valuable for high values of p. There is a discontinuity at p = pcrit. Parameter values
for (a), (b), and (c) are λ = 0.4, V = 4, and L = 1. pcrit = 0.833, 0.25, 0.5, and 0.75 for c = 0.1, 0.3, 0.6, and 0.9, respectively.
Pike et al. • The value of information
a possible change in behavior as a result of an observation. We first
derive an expression for the value of information for a 2-state world
then generalize the model for a world that can be in multiple states.
We illustrate our approach with a case study based on the cooperative behavior between a male and female raising their common
young where an incubating female must decide how long to wait
for her foraging partner to return before abandoning the nest.
Our formulae highlight that the value of information is determined by its potential to change behavior, a point made by
Stephens (1989). In our formula, the variance of the posterior plays
a crucial role as opposed to Stephens’ (1989) criterion that is in
terms of the variance of behavior.
In our model, because ∑ i pi = 1, the set of all possible knowledge vectors p = ( p1 ,…, pk ) is an n − 1 dimensional space Sn − 1.
The optimal payoff function W*(p) is a defining surface over Sn − 1,
and this surface is the reproductive landscape. Following McNamara
and Dall (2010), we use the fact that E[Q] = p (with Q being the
random knowledge vector) and due to the convexity of the reproductive landscape W*(p), then E[W*(Q)] ≥ W*(p). That is, the
height of the reproductive landscape at the prior p is less than or
equal to its average height at the posterior probabilities whose mean
is p. The variance in the posterior determines the average increase
in reproductive value of the individual after gaining information
about the state of the world, which is the value of information.
Stephens quantifies the value of information as the difference
between a perfectly informed individual and an individual with no
information beyond the prior. The value of attending to a potential observation that gives partial information about the state of
the world is solved by calculating the value of perfect information
about the observable states of the cue. In our model, we assume,
perhaps more realistically, that an individual gains partial information about the state of the world and an observation will update the
prior distribution.
For an animal’s behavior to be influenced by a gain in information, the animal must be flexible with their choice of action, always
taking the best action given their current knowledge. In more technical terms, assuming an animal behaves optimally, it is flexible in
its choice of action if it can change its behavior based on the random variable pˆ.
As with other theoretical models of flexible effort adjustment (Axelrod and Hamilton 1981; Sherratt and Roberts 1998;
Killingback and Doebeli 2002), we consider an adjustable trait in
response to the state of the world. The alternative is for the trait to be
genetically determined and unchangeable by the animal (McNamara
et al. 2008). Dall et al. (2005) argue, however, that animals do not
possess complete flexibility but instead follow mechanistic rules which
limit this flexibility, an example being a period of learning in which
behavioral flexibility occurs being followed by a lack of learning for
the rest of the animal’s life (Lotem and Halpern 2010). The value of
information can still be defined in this case but we would expect it to
be less than if behavior were completely flexible.
Behavioral flexibility is encapsulated by the multiplier term T*(p)
in Expression 9 and the vector T * ′ in Expression 19. Both terms
represent the rate of change of the optimal trait with information.
If there is no change to behavior after a gain in information, then
both terms are equal to 0 and there is no value to the information.
Only when there is flexibility in behavior is the rate of change of
the optimal trait nonzero and information is valuable. We conclude
that there is only value to information if the animal possesses flexibility to change its behavior in response to a gain in information.
Our new expression to quantify the value of information uses
Bayesian statistical theory to model the process of an animal
1301
updating its knowledge. An alternative measure of information is
mutual information (Shannon 1948; Shannon and Weaver 1949),
which describes uncertainty reduction measured in terms of
entropy (Cover and Thomas 1991).
Donaldson-Matasci et al. (2010) show the connection between
mutual information and the decision theoretic measure of information within the context of evolution in an uncertain environment.
They show that the fitness value of a developmental cue, when
measured as the increase in long-term growth rate of a lineage,
equals reduction in uncertainty as described by mutual information.
The analysis of Donaldson-Matasci et al. (2010) is concerned
with the case in which the environment as a whole fluctuates, so
that all individuals in a population are subject to the same environmental conditions. In this setting, a cue provides potential information on the current environment and the value of a cue is the
resultant expected increase in geometric mean fitness. In contrast,
we are concerned with the situation in which there are no fluctuations in the environment as a whole, but different individuals are
in different situations, for example, some have partners of type 1,
whereas others do not. In our analysis, a cue provides potential
information about a certain individual in the environment largely
independent of other individuals in the population, and the value
of observing a cue is the resultant expected increase in reproductive
value. Thus, Donaldson-Matasci et al. (2010) are concerned with
environmental stochasticity (aggregate risk), whereas our analysis is
concerned with demographic stochasticity (idiosyncratic risk).
Although Donaldson-Matasci et al. (2010) establish a link
between mutual information and the decision theoretic measure of
information, it is not clear that our context would provide the same
connection. For our context, reproductive value is the appropriate
measure of information.
It is important to note the continuous nature of the trait in our
model. We argue that using a continuous trait in the derivation of
the expression for the value of information allows for more realistic biological applications. For example, males vary the continuous
level of effort invested in a clutch dependent on paternity (Møller
and Cuervo 2000) such as in the mating system of the dunnock
(Prunella modularis) (Davies 1992). In some species, males allocate
sperm according to the expected reproductive value of a copulation
based on the level of sperm competition associated with a female
(Wedell et al. 2002) and on female mate quality (Hunter et al. 2000;
Pizzari et al. 2003). In principle, we can use our expression to find
the value of information about paternity to a male or the value of
information about different types of female.
To apply a biological context to our proof, we illustrated our
expression for the value of information using the cooperative
behavior between a male and a female bird raising their common
young. It is assumed that the female is in possession of a continuous,
flexible trait that allows her to vary the amount of time devoted to
incubation of the young. We quantified the value of information to
the female about the type of male she is paired with and the effect
of information on her behavior. We showed that as the probability the male was of the type that returns to the nest after foraging
increased, the female’s optimal wait time for his return was longer.
We showed that information was valuable to the female if a gain of
information changed her behavior.
In our illustration, we assumed a fixed cost to the female per unit
of time. More realistically for some contexts, a mortality rate parameter could be used. However, the use of such a parameter would
not affect the shape of the value of information curve and therefore
the interpretation of results. The Taylor series is used to approximate the payoff function W*(p). Stephens (1989, p. 137) discusses
Behavioral Ecology
1302
the limitations of using such an approximation method in a similar
context.
Our illustration has shown that there is value in knowing your
partner. If information is valuable to an animal, we would expect an
animal to pay a cost of gathering it. Through courtship, for example,
a female can gather information about a desired quality in a mate.
Our approach allows us to quantify the value of this information.
The general case of our expression is also applicable to a variety
of foraging problems where we can consider an animal in a world
of n distinct habitats with a different value of a parameter in each.
For example, we could consider an animal with a choice of n patches,
each with a different level of prey abundance, where the animal is
flexible in the amount of time it dedicates to a patch (McNamara and
Houston 1985). The animal may gain information about the level of
prey in an individual patch simply by spending time in the patch. We
can determine how valuable this information is to the animal.
There are a wide variety of species and contexts in which an individual possesses a flexible, continuous trait where information would
be useful to them. Our results can be extended by rederiving our
expression for the value of information with the assumption that
the state of nature is a continuum. The probabilities would become
density functions and the summation, integrals. For example, the
optimal clutch size of a female given the state of the environment
could be predicted, where the environment is modeled with a continuous distribution. This would render further opportunities for our
expression to be applied to even more biological scenarios.
APPENDIX 2
Deriving Equation 21
The definition of T*(p) is extended to this setting, still defined as
the value of T that maximizes W(T, p). Differentiating W*(p) partially with respect to pi gives
∂W *
∂W *
∂T *
∂W *
( p) =
(T ( p ), p )
( p) +
(T ( p ), p ).
∂pi
∂T
∂pi
∂pi
(2.1)
The first term on the right hand side in Equation 2.1 is equal to 0
from Equation 2. The remaining term on the right hand side can
be simplified because from Equation 13, we have
n
*
W (T * ( p ), p ) = ∑ pW
i
i (T ( p ))
(2.2)
i =1
and when partially differentiating by pi then all pj when j ≠ i are
held constant. Thus, we resolve
∂W *
(T ( p ), p ) = Wi (T * ( p ))
(2.3)
∂pi
Differentiating Equation 2.3 partially with respect to pj gives
∂2W *
∂T *
( p ) = Wi ′(T * ( p ))
( p ).
∂pi ∂p j
∂p j
n
∑ pW ′(T
( p )) = 0
(2.5)
and differentiating this equation partially with respect to pj gives
i =1
FUNDING
This work was supported by the Engineering and Physical Sciences
Research Council (Ep/1032622/1 to R.K.P. and J.M.M.) and the
European Research Council Advanced Grant (250209 to A.I.H.).
n
(1.1)
∂2W
dT *
∂2W
(T * ( p ), p )
( p ) + 2 (T * ( p ), p ).
∂T ∂p
dp
∂p
(1.3)
The second term on the right hand side of Equation 1.2 is equal
to 0 because the 2nd partial derivative of W from Equation 1
with respect to p evaluated at the optimal strategy T*(p) is equal
to 0. The remaining 2nd partial derivative in Equation 1.2 can be
calculated directly from Equation 1 to give
∂2W
∂
(T * ( p ), p ) =
(W1 (T ) − W0T ).
(1.4)
∂T ∂p
∂T
Thus, by Equations 1.2 and 1.3, we have
∂T *
( p ) = 0,
∂p j
k
k
Thus, by Equations 2.4 and 2.7, we have
k =1
Differentiating W*(p) partially with respect to p gives
∂W *
∂W *
dT *
W *′ ( p ) =
(T ( p ), p ) +
(T ( p ), p )
( p ).
(1.2)
∂p
∂T
dp
The second term on the right hand side of Equation 1.1 is equal to
0 by Equation 2. Differentiating Equation 1.1 again partially with
respect to p gives
k =1
which rearranges to
(1.5)
(2.6)
(2.7)
Wi (T * ( p ))W j’ (T * ( p ))
∂2W *
( p) = −
.
n
∂pi ∂p j
∑ k =1 pkWk ″(T * ( p ))
From Equation 3, the optimal payoff function is defined as
W * ′′ ( p ) = [W1′ (T ) − W0′ (T )]T * ′ ( p ).
*
i
W j ′ (T * ( p ))
∂T *
( p) = − n
.
∂p j
∑ p W ′′ (T * ( p ))
APPENDIX 1
Calculating W*”(p) directly from W*(p)
W * ′′ ( p ) =
i
W j ′ (T * ( p )) + ∑ pkWk ′′ (T * ( p ))
Handling editor: Shinichi Nakagawa
W * ( p ) = W (T ∗ ( p ), p ).
(2.4)
Because W(T, p) is at its maximum at T = T*(p), we have
(2.8)
Combining Equations 2.8 and 17, we get
*
*
n Wi ′ (T ( p ))W j ′ (T ( p ))
1 n
Cov (Q i ,Q j ).
∑
∑
n
(2.9)
2 i =1 j =1 ∑ pW ′′ (T * ( p ))
i
i
i
=
1
Using Cov(X + Y, Z) = Cov(X, Z) + Cov(Y, Z) and because the
covariance of a random variable with itself is the variance of the
random variable, we have
I (X ) = −
∑ ∑
n
n
i =1
j =1
Wi ′ (T * ( p ))W j ′ (T * ( p ))Cov (Q i ,Q j )
(∑
= Var (∑
= Cov
Wi ′ (T * ( p ))Q i , ∑ j =1W j ′ (T * ( p ))Q
n
n
i =1
n
)
Wi ′ (T * ( p ))Q i .
i =1
By Equations 2.9 and 2.10,
(
which gives the result in Expression 21.
i =1
) (2.10)
)
′ *
1 Var ∑ i =1Wi (T ( p ))Q i
I (X ) = −
,
*
n
2
∑ piWi ′′(T ( p ))
n
j
(2.11)
Pike et al. • The value of information
1303
APPENDIX 3
Expression 9 is equivalent to Expression 21
when the world can be in 2 distinct states
Setting i = 2 in Expression 21 gives
I (X ) = −
(
)
′ *
′ *
1 Var Q 0W0 (T ( p )) + Q 1W1 (T ( p ))
.
2
′′ *
p0W0′′ (T * ( p )) + pW
1 1 (T ( p ))
(3.1)
Because ∑ i Q i = 1, then Q0 can be set as 1 − Q and Q1 can be set
as Q. Therefore, the numerator in Expression 3.1 becomes
[W1′ (T * ( p )) − W0′ (T * ( p ))]2 Var (Q )
(3.2)
because Wi ′ (T * ( p )) is a constant. As ∑ i pi = 1, then p0 can be set
as 1 − p and p1 can be set as p and the payoff function W(T, p) has
the same form as Equation 1. Differentiating W(T, p) partially with
respect to T gives
W0′ (T ) + p[W1′ (T ) − W0′ (T )].
(3.3)
Evaluating Equation 3.3 at T = T*(p) and making use of Equation
2 gives
(3.4)
W0’ (T * ( p )) + p W1′ (T * ( p )) − W0′ (T * ( p )) = 0


Differentiating Equation 3.4 totally with respect to p gives
−[(1 − p )W0′′ (T * ( p )) + pW1′′ (T * ( p ))] T * ′ ( p )
= W1′ (T * ( p )) − W0′ (T * ( p)))
Combining Equations 3.1, 3.2, and 3.5,
(3.5)
1 [W1′ (T * ( p )) − W0′ (T * ( p ))]2 Var(Q )T * ′ ( p )
2
W1′ (T * ( p )) − W0′ (T * ( p ))
1
= VarQ [W1′ (T * ( p )) − W0′ (T * ( p ))]T * ′ ( p ). 2
I (X ) =
(3.6)
which matches the original derivation of I(p) in Expression 9
when X = p and Q = pˆ .
REFERENCES
Axelrod R, Hamilton WD. 1981. The evolution of cooperation. Science.
211:1390–1396.
Berg LM, Pálsson S, Lascoux M. 2001. Fitness and sexual response to population density in Daphnia pulex. Freshw Biol. 46:667–677.
Collins SA, Hubbard C, Houtman AM. 1994. Female mate choice in the
zebra finch: the effect of male beak colour and male song. Behav Ecol
Sociobiol. 35:21–25.
Collins EJ, McNamara JM, Ramsey DM. 2006. Learning rules for optimal selection in a varying environment: mate choice revisited. Behav Ecol. 17:799–809.
Cover TM, Thomas JA. 1991. Elements of information theory. New York:
Wiley.
Dall SR, Giraldeau LA, Olsson O, McNamara JM, Stephens DW. 2005.
Information and its use by animals in evolutionary ecology. Trends Ecol
Evol. 20:187–193.
Davies NB. 1992. Dunnock behaviour and social evolution. Oxford series in
ecology and behaviour. Oxford: Oxford University Press.
Donaldson-Matasci MC, Bergstrom CT, Lachmann M. 2010. The fitness
value of information. Oikos. 11:219–230.
Getty T. 1996. Mate selection by repeated inspection: more on pied flycatchers. Anim Behav. 51:739–745.
Gould JP. 1974. Risk, stochastic preference and the value of information. J
Econ Theory. 8:64–84.
Grafen A. 1990. Biological signals as handicaps. J Theor Biol.
144:517–546.
Harrison F, Barta Z, Cuthill I, Szekely T. 2009. How is sexual conflict over parental care resolved? A meta-analysis. J Theor Biol.
22:1800–1812.
Houston and McNamara. 2005. John Maynard Smith and the importance
of consistency in evolutionary game theory. Biol Philos. 20:933–950.
Hunter FM, Harcourt R, Wright M, Davis LS. 2000. Strategic allocation of
ejaculates by male Adelie penguins. Proc Biol Sci. 267:1541–1545.
Kendeigh SC. 1952. Parental care and its evolution in birds. Ill Biol
Monogr. 22:1–356.
Killingback T, Doebeli M. 2002. The continuous prisoner’s dilemma and
the evolution of cooperation through reciprocal altruism with variable
investment. Am Nat. 160:421–438.
Killingback TM, Doebeli M, Knowlton N. 1999. Variable investment, the
continuous prisoner’s dilemma, and the origin of cooperation. Proc Biol
Sci. 266:1723–1728.
Lotem A, Halpern JY. 2010. Co-evolution of learning and data acquisition
mechanisms: a model for cognitive evolution. Philos Trans R Soc Lond B
Biol Sci. 367:2686–2694.
Luttbeg B. 1996. A comparative Bayes tactic for mate assessment and
choice. Behav Ecol. 7:451–460.
Maynard Smith J, Harper DGC. 1995. Animal signals: models and terminology. J Theor Biol. 177:305–311.
Maynard Smith J, Harper DGC. 2003. Animal signals. Oxford: Oxford
University Press.
Mazalov V, Perrin N, Dombrovsky Y. 1996. Adaptive search and information updating in sequential mate choice. Am Nat. 148:123–137.
McNamara JM, Barta Z, Fromhage L, Houston AI. 2008. The coevolution
of choosiness and cooperation. Nature. 451:189–192.
McNamara JM, Dall SRX. 2010. Information is a fitness enhancing
resource. Oikos. 119:231–236.
McNamara JM, Doodson P. 2015. Reputation can enhance or suppress
cooperation through positive feedback. Nat Commun. 6:6134.
McNamara JM, Green RF, Olsson O. 2006. Bayes theorem and its application in animal behaviour. Oikos. 112:243–251.
McNamara JM, Houston AI. 1980. The application of statistical decision
theory to animal behaviour. J Theor Biol. 85:673–690.
McNamara JM, Houston AI. 1985. A simple model of information use in
the exploitation of patchily distributed food. Anim Behav. 33:553–560.
Møller AP, Cuervo JJ. 2000. The evolution of paternity and paternal care in
birds. Behav Ecol. 11:472–485.
Nice MM. 1937. Studies in the life history of the song sparrow I. A population study of the song sparrow. Linn Soc NY Trans. 4:1–247.
Pizzari T, Cornwallis CK, Lovlie H, Jakobsson S, Birkhead TR. 2003.
Sophisticated sperm allocation in male fowl. Nature. 426:70–74.
Shannon CE. 1948. A mathematical theory of communication. Bell Syst
Tech J. 27:379–423, 623–656.
Shannon CE, Weaver W. 1949. The mathematical theory of communication. Urbana (IL): The University of Illinois Press.
Sherratt TN, Roberts G. 1998. The evolution of generosity and choosiness
in cooperative exchanges. J Theor Biol. 193:167–177.
Stephens DW, Krebs JR. 1986. Foraging theory. Princeton (NJ): Princeton
University Press.
Skutch AF. 1976. Parent birds and their young. Austin (TX): University of
Texas Press.
Wedell N, Gage MJG, Parker GA. 2002. Sperm competition, male prudence and sperm-limited females. Trends Ecol Evol. 17:313–320.
Wilkinson GS. 1984. Reciprocal food sharing in the vampire bat. Nature.
308:181–184.