Supplementary Materials to
Development of a diagnostic test based on multiple
continuous-biomarkers based on an imperfect reference-test.
Leandro García Barrado a, Els Coart b, Tomasz Burzykowski a,b, for the Alzheimer’s
Disease Neuroimaging Initiative*
a
Interuniversity Institute for Biostatistics and statistical Bioinformatics (I-BioStat), Hasselt
University, Agoralaan Building D, 3590 Diepenbeek, Belgium
b
International Drug Development Institute (IDDI), Avenue Provinciale 30, 1340 Louvain-laNeuve, Belgium
*
Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging
Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to
the design and implementation of ADNI and/or provided data but did not participate in analysis or
writing of this report. A complete listing of ADNI investigators can be found at:
http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.
Appendix A: The 𝑆𝑒 + 𝑆𝑝 > 1 restriction
To implement the 𝑆𝑒 + 𝑆𝑝 > 1 restriction, a joint distribution for 𝑆𝑒 and 𝑆𝑝 has to be
assumed. The chosen form of the joint distribution will have consequences for the marginal 𝑆𝑒
and 𝑆𝑝 priors because of the potential dependence induced by the joint distribution.
One might attempt to implement the restriction while retaining the flat standard uniform
marginal priors for 𝑆𝑒 and 𝑆𝑝. This is, however, not possible. A formal argumentation could be
built considering the copula representation of the joint distribution function of two standard
uniform marginal distribution functions. From the representation it follows that the joint
1
distribution function has to be bounded by the Fréchet lower (𝐶𝐿 ) and upper (𝐶𝑈 ) bound
copulas. These are expressed as follows:
𝐶𝐿 (𝑢1 , 𝑢2 ) = max{0, 𝑢1 + 𝑢2 − 1} , 𝒖 ∈ [0,1]2 , and
𝐶𝑈 (𝑢1 , 𝑢2 ) = min{𝑢1 , 𝑢2 } , 𝒖 ∈ [0,1]2 .
It can be shown that, for 𝐶𝐿 we get 𝑝(𝑢1 + 𝑢2 = 1) = 1, while for 𝐶𝑈 we have 𝑝(𝑢1 + 𝑢2 >
1) = 0.5. In other words, no joint distribution with standard uniform margins will lead to
𝑝(𝑢1 + 𝑢2 > 1) = 1 for the desired restriction.
For the independent 𝑆𝑒 and 𝑆𝑝 case, one could think of two uniform marginal distributions of
𝑆𝑒 and 𝑆𝑝 restricted to be strictly higher than 0.5 to ensure that their sum exceeds 1.
To allow 𝑆𝑒 or 𝑆𝑝 assuming values smaller than 0.5, a possible solution is to use a standard
uniform marginal distribution for 𝑆𝑒 or 𝑆𝑝 and define a suitable conditional distribution of the
other parameter. For instance,
𝑆𝑒 ~𝑈(0,1)
𝑆𝑝|𝑆𝑒 ~𝑈(1.001 − 𝑆𝑒, 1).
Note that this solution is ‘asymmetric’ in that one of the parameters is selected to be uniformly
distributed on the (0,1) interval. Obviously, other implementations of the 𝑆𝑒 + 𝑆𝑝 > 1
restriction are also possible. They will differ in terms of informativeness of the marginal 𝑆𝑒 and
𝑆𝑝 distributions and of the amount of dependence between 𝑆𝑒 and 𝑆𝑝. The choice of the
2
implementation may require a careful consideration of the characteristics of the particular
problem at hand.
3
Appendix B: The 𝐴𝑈𝐶𝑎 -prior approximation
Under the reparameterized 𝐴𝑈𝐶𝑎 -prior definition it can be shown that when assuming that:
𝜹~𝑵𝒌 (𝜿, 𝜳),
the implied 𝐴𝑈𝐶𝑎 prior distribution can be approximated based on quadratic forms theory [B1,
B2]. In particular, the distribution can be represented by an expansion in non-central chi-square
distributions:
∞
2
𝑓(𝑥) = ∑ 𝑐𝑘 𝜒(𝑝+2𝑘;𝜁)
(𝜙 −1 (𝑥)2 ) × |
𝑘=0
2𝜙 −1 (𝑥)
|;
𝜙(𝜙 −1 (𝑥))
𝑓𝑜𝑟 𝑥 ∈ [0.5,1]
With
𝑝
𝜁 = √∑ 𝑏𝑗2
𝑗=0
1
𝑝
1 2
𝑐0 = ∏ ( )
𝜆𝑗
𝑗=1
𝑘−1
𝑐𝑘 =
1
∑ 𝑑𝑘−𝑟 𝑐𝑟
2𝑘
𝑟=0
𝑝
𝑑1 = ∑(1 − 𝑏𝑗2 )(1 −
𝑗=1
𝑝
𝑘
𝑃
1
)
𝜆𝑗
𝑘−1
𝑏𝑗2
1
1
𝑑𝑘 = ∑ (1 − ) + 𝑘 ∑ ( ) (1 − )
𝜆𝑗
𝜆𝑗
𝜆𝑗
𝑗=1
𝑗=1
1
′
where λ1 , … , λp are the eigenvalues of Ψ and the vector b′ = (b1 , … , bp ) = (P ′ Ψ −2 Κ) , which is
a by-product of diagonalizing Ψ by 𝑃, the 𝑝 × 𝑝 orthonormal matrix of eigenvectors of Ψ. By
4
using a finite number of terms in the series expansion, it is possible to approximate the
distribution of 𝐴𝑈𝐶𝑎 for different choices of 𝛋 and Ψ, with arbitrary precision.
For instance, assume that 𝜿 = (0,0,0)′ and that standard deviations and correlation coefficients
resulting from the variance-covariance matrix 𝜳 vary between 0.1 and 1, and 0 and 0.9,
respectively. Figure B1 presents the histograms of 100,000 simulated values of the 𝐴𝑈𝐶𝑎 for the
90 resulting combinations of standard deviations and correlations. Additionally, the figure
presents approximations computed from the appropriate expansion of 200 non-central chisquare distributions. From the figure it can be seen that the approximations correspond quite
closely to the histograms when standard deviations are larger than 0.4 and correlation
coefficients are smaller than 0.7. For more extreme values, the approximation tends to break
down at the higher tail of the distribution. Despite the issues for some of the cases, the seriesexpansion can be used to explore the specified prior distribution for the 𝐴𝑈𝐶𝑎 .
5
Density
Standard deviation
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Correlation
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
AUC
Figure B1: Histogram (grey) and approximated densities (solid black line) of the simulated AUC
values.
[B1] Mathai AM, Provost SB. Quadratic forms in random variables. Marcel Dekker: Inc., New York, USA,
1992.
[B2] Ruben H. Probability content of regions under spherical normal distributions, IV: the distribution of
homogeneous and non-homogeneous quadratic functions of normal variables. The Annals of
Mathematical Statistics 1962; 33:542-570.
6
Appendix C: Implied 𝜇1 -prior distribution
Figure C1 shows the implied prior distribution for the components of parameter vector 𝝁1 . The
histograms shown in the figure are derived by simulating 100,000 values of 𝝁1 = 𝑸−1 𝜹 + 𝝁0
based on the proposed prior distributions for 𝜇0 , 𝛴0 , 𝛴1 , and 𝛿. The solid lines in the panels
shown in Figure C1 represent the assumed flat priors for the components of 𝝁0 (note that the xaxis ranges from -15,000 to 15,000).
-15000
-5000
0
5000
10000
15000
-15000
0.00012
Density
0
1
0.00008
1
0.00000
0.00004
0
0.00000
0.00004
Density
0.00008
1
0.00004
0
Prior-distributions
mean components biomarker 3
0.00008
0.00012
Prior-distributions
mean components biomarker 2
0.00000
Density
0.00012
Prior-distributions
mean components biomarker 1
-5000
0
5000
10000
15000
-15000
-5000
0
5000
10000
Figure C1: Simulated prior distribution for μ1 (histogram) based on proposed prior distributions
for 𝜹, 𝜮0 , and, 𝜮1 , and 𝝁0 (solid line).
7
Appendix D: Se/Sp-prior distribution sensitivity
Results of the analysis of the ADNI data using two different forms of the prior distribution for Se
and Sp.
Se/Sp Prior
Parameter
𝑺𝒆 ~ 𝑩𝒆𝒕𝒂(𝟏, 𝟏)
𝑩𝒆𝒕𝒂(𝟏, 𝟏)𝑻[𝟎. 𝟓𝟏, 𝟏)
𝑺𝒑|𝑺𝒆 ~𝑩𝒆𝒕𝒂(𝟏, 𝟏)𝑻[𝟏. 𝟎𝟎𝟏 − 𝑺𝒆, 𝟏)
0.983
[0.959,0.994]
0.984 [0.959,0.994]
𝑨𝑼𝑪𝒂
0.826 [0.726,0.908]
0.825 [0.726,0.905]
𝑺𝒆
0.887
[0.803,0.949]
0.888 [0.805,0.951]
𝑺𝒑
0.498 [0.412,0.584]
0.499 [0.414,0.584]
𝜽
8
Appendix E: Probability of AD of the resulting diagnostic score
Figure E1 shows the probabilities of AD based on the results from the proposed Bayesian latentclass model fitted to the ADNI dataset. Panel 𝑎 of Figure E1 shows the estimated probability of
AD for the resulting score based on the optimal linear-combination of the three biomarkers by
clinical diagnosis. Panel 𝑏 of Figure E1 illustrates the posterior probability of AD based on the
combined information from the biomarkers and clinical diagnosis, by clinical diagnosis.
a.
b.
10
12
14
16
18
20
0.2
0.4
0.6
0.8
ooooooooo
oo
ooooo
o
oooooooooo
+o+oooo o o o
+ooo
+oooo
ooooo
++
ooo o+ +
+
+
o
oo
+
oo
+
o
+
o
+
+
+
o
+
o
+
+
+
o o ++
++ +
++++
+++
+
+
+++
o
+++
o
o+++ +++++++++o
+++++++
+++++++
+++
++
+++++o+
+++++
o
0.0
Posterior probability of AD
0.8
0.6
0.4
0.2
ClDiag=0
ClDiag=1
0.0
Estimated probability of AD
1.0
Posterior probabilities by clinical diagnosis
1.0
Estimated probabilities by clinical diagnosis
10
Diagnostics index
12
14
+ ClDiag=0
o ClDiag=1
16
18
20
Diagnostic index
Figure E1: Probability of AD for estimated optimal combination score by clinical diagnosis.
Clinical controls indicated in grey, clinical cases in black. a. Estimated probability of AD. b.
Posterior probability of AD.
9