Case: two days
Assume that there is a constant incidence rate, with daily probability of the
start of a new episode of θ. It is assumed that these events are independent.
For a given individual it is assumed that the episode length (in days), D,
is a random variable, whose distribution does not change over time, with
support the real numbers 1,2,..., and expectation ∆. We want to estimate
∆. In the following we need to make some approximations, and a key one is
that the probability of two episodes overlapping, or being on adjacent days,
is small. Given the assumptions above, the probability of this happening is
approximately θ∆. It is difficult to say for all settings how small this should
be, but in the simulation studies and in the example in this paper, it is at
most about 0.1, and typically much smaller than this.
Suppose that an individual provides responses Y1 and Y2 on two consecutive
days. These each take the values one or zero indicating presence or absence
of the disease. We need the joint probabilities of the four possible outcomes
(Y1 , Y2 ), that is (0, 0), (0, 1), (1, 0), and (1, 1). The event Y1 = 1 corresponds
to the first observation having been made during a disease episode. From the
expected proportion of days with infection it follows directly that P(Y1 = 1)
= θ∆. Given that Y1 = 1 the probability that the second observation Y2 = 1,
i.e, it is also part of the episode, depends on the length of the episode. Given
that this can happen only if Y1 does not occur at the end of the episode we
have P(Y2 = 1 | Y1 = 1, D = d) = (d − 1)/d. Using a first order Taylor series
approximation to take expectations over the distribution of D we have, approximately, P(Y2 = 1 | Y1 = 1) = (∆ − 1)/∆. It follows that, approximately,
P(Y1 = 1, Y2 = 1) = P(Y2 = 1 | Y1 = 1)P(Y1 = 1) = θ(∆ − 1).
Hence
P(Y1 = 1, Y2 = 0) = P(Y1 = 1) − P(Y1 = 1, Y2 = 1) = θ.
By symmetry P(Y1 = 0, Y2 = 1) = P(Y1 = 0, Y2 = 1), and P(Y1 = 0, y2 = 0) =
1 − P(Y1 = 0, Y2 = 1) − 2P(Y1 = 1, Y2 = 1).
Suppose that there is a sample of N subjects, for each of which there is one of
three possible outcomes: both responses 0, one response 0 and one 1, or both
1. Label these outcomes 0, 1 and 2 respectively, with associated probabilities
π0 , π1 and π3 and counts from the whole sample n0 , n1 and n2 . These follow a
multinomial distribution with index N where, from the results above,
π0 = 1 − θ(1 + ∆)
π1 = 2θ
π2 = θ(∆ − 1).
1
From this the log likelihood kernel is
`(θ, ∆) = n0 ln{1 − θ(∆ + 1)} + n1 ln(2θ) + n2 ln{θ(∆ − 1)}.
Differentiating with respect to θ and ∆, and solving the resulting score equations, leads to the maximum likelihood estimators
θ̂ =
n1
2N
ˆ = 1 + 2 n2 .
∆
n1
(1)
ˆ can be obtained via the information matrix, or
The asymptotic variance of ∆
directly using a conventional second order Taylor series expansion, as follows.
Approximately,
2 2 π1 π 2
n2
1 π2
π1 (1 − π1 ) π2 (1 − π2 )
1 π2
1
1
+
+2
V
≈
=
+
.
n1
N π1
π12
π22
π1 π 2
N π1
π 1 π2
Replacing these probabilities by their obvious estimators, gives the variance
ˆ
estimator for ∆:
2 n
n
+
n
2
1
2
ˆ =4
V̂(∆)
.
(2)
n1
n1 n2
ˆ it is probably
Because of the asymmetry of the sampling distribution of ∆
better to work on the log scale for the calculation of the confidence interval,
i.e. use,
ˆ =∆
ˆ −2 V̂(∆).
ˆ
V̂{ln(∆)}
and then re-transform.
Case: three days
There are now eight possible outcomes on the three days: (0,0,0), (0,0,1),
(0,1,0), (1,0,0), (0,1,1), (1,0,1), (1,1,0) and (1,1,1). First we make the assumption that θ∆ is sufficiently small for P(1, 0, 1) to be negligible. This is a small
extension of the sparseness assumption underlying the derivation in the two
day case above. Using a similar derivation to that used above, it can be shown
approximately, that
P(Y1 = 1, Y2 = 1, Y3 = 1) = θ(∆ + α − 2),
for α = P(D = 1). Note the introduction an additional unknown, α.
2
As before we have P(Yt = 1, Yt+1 = 1) = θ(∆ − 1), P(Yt = 0, Yt+1 = 1) =
P(Yt = 1, Yt+1 = 0) = θ and P(Yt = 1) = θ∆. Combining these with the
requirement that P(Y1 = 1, Y2 = 0, Y3 = 1) = 0, the probabilities for all eight
outcomes can be obtained as follows:
(0,0,0):
(1,0,0):
(0,0,1):
(0,1,0):
(1,0,1):
(0,1,1):
(1,1,0):
(1,1,1):
1 − θ(∆ + 2)
θ
θ
θα
0
θ(1 − α)
θ(1 − α)
θ(∆ + α − 2)
The relevant data can be expressed as five counts, that sum to N :
n0 :
n1a :
n1b :
n2 :
n3 :
the
the
the
the
the
number of (0,0,0) outcomes
total number of (1,0,0) and (0,0,1) outcomes
number of (0,1,0) outcomes
total number of (1,1,0) and (0,1,1) outcomes
number of (1,1,1) outcomes.
The kernel of the multinomial log likelihood for this setup is then
n0 ln{1−θ(∆+2)}+n1a ln(θ)+n1b ln(θα)+n2 ln{θ(1−α)}+n3 ln{θ(∆+α−2)}.
This does not have analytic solutions for the maximum in terms of the parameters, but is easily optimized using standard numerical techniques. Estimates
of precision can be obtained in the usual way using second derivatives of the
log likelihood at the maximum.
As a very simple but much less efficient alternative, a moments based estimator can be derived from the (0,0,1), (1,0,0) and (0,0,0) outcomes. Using the
expectations
E(n1a ) = 2N θ,
E(n0 ) = N {1 − θ(∆ + 2)}
and solving for the two parameters gives
n1a
,
θ̂ =
2N
N − n0 − n1a
ˆ
∆1 = 2
,
n1a
3
which can also be written
ˆ 1 = 3 − p + 2 n∗
∆
n1a
(3)
for n∗ = N − n0 − n1a and p the number of days recorded. From this it can
be seen that this estimator has the same form of the one obtained earlier for
p = 2 (1), and so the variance estimator from the earlier case (2) can be used
here, replacing n1 by n1a and n2 by n∗ .
A second, less inefficient, moments estimator can be obtained by combining,
in addition, the information from the (1,1,0), (0,1,1) and (0,1,0) outcomes.
Using
n1a + 2n1b + n2
= θ,
E
4N
gives the second moments estimator
ˆ2 =
∆
4(N − n0 )
− 2.
n1a + 2n1b + n2
This has efficiency that is reasonably close to that of the maximum likelihood
estimator.
Standard delta methods can be used to obtain an asymptotic variance formula
for this, which can be estimated using
16(N
−
n
)
2n
(N
−
n
)
0
1b
0
ˆ 2) =
V̂ (∆
1+
−1
(n1a + 2n1b + n2 )2 (n1a + 2n1b + n2 )
(n1a + 2n1b + n2 )
Case: Four days
There are now sixteen possible outcomes (0,0,0,0) to (1,1,1,1). To proceed we
need an extension of the same sparceness requirement used before, it is now
assumed that the probability of two separate episodes occuring within the four
day interval is sufficiently small. Using exactly the same arguments as earlier,
it can be shown that, approximately
P(Y1 = 1, Y2 = 1, Y3 = 1, Y4 = 1) = θ(∆ + 2α + β − 3),
for β = P(D = 2). Combining this with the earlier results, we get the following
probabilities (and associated counts) for the 16 outcomes:
4
n0
n1a
n1b
n2a
n2b
n3
n4
(0,0,0,0):
(1,0,0,0)+(0,0,0,1):
(0,1,0,0)+(0,0,1,0):
(1,1,0,0)+(0,0,1,1):
(0,1,1,0):
(1,1,1,0)+(0,1,1,1):
(1,1,1,1):
1 − θ(∆ + 3)
2θ
2θα
2θ(1 − α)
θβ
2θ(1 − α − β)
θ(∆ + 2α + β − 3)
for β = P(D = 2), and the remaining outcomes having zero probability. The
kernel of the multinomial log likelihood is then
n0 ln{1 − θ(∆ + 3)} + n1a ln(θ) + n1b ln(θβ) + n2a ln{θ(1 − α)} + n2b ln(θβ)
+ n3 ln{θ(1 − α − β)} + n4 ln(∆ + 2α + β − 3)
and as before this can be used in the conventional way to obtain maximum
likelihood estimators and estimates of precision.
Two less efficient moments estimators can also be derived, using the same
arguments as above. The least efficient takes the form (3) as in the earlier
cases
ˆ 1 = 3 − p + 2 n∗
∆
n1a
for n∗ = N − n0 − n1a .
The second moments estimator can be written:
ˆ2 =
∆
6(N − n0 )
+ 1 − p.
n1a + 2n1b + n2a + 2n2b + n3
5