METAGRAPHITI BY STATA
Ben A. Dwamena, MD
Department of Radiology, University of Michigan Medical School
Nuclear Medicine Service, VA Ann Arbor Health Care System
Ann Arbor, Michigan
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
1
METAGRAPHITI
Statistical Graphics For
Interpretation, Exploration
And Presentation Of
Meta-analysis Data
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
2
METAGRAPHITI BY STATA
VISUOGRAPHIC FRAMEWORK FOR USING STATA
FOR:
‰ Testing and correcting for publication bias
‰ Investigating heterogeneity
‰ Summary of Data and Sensitivity
Analyses
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
3
METAGRAPHITI
‰Avoid potential misrepresentation by
faulty distributional and other statistical
assumptions.
‰Facilitates greater interaction between the
researcher and the data by highlighting
interesting and unusual aspects of the
quantitative data.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
4
METAGRAPHITI
‰User-friendlier summaries of large,
complicated quantitative data sets.
‰Preliminary exploration before definite
data synthesis.
‰Effective emphasis of important
features rather than details of data.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
5
CONTINGENCY TABLE FOR
EXTRACTION OF DATA
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
6
DIAGNOSTIC VS. TREATMENT TRIAL
‰ True Positives =Experimental Group With the
Monitored Outcome Present (a).
‰ False Positives = Control Group With the Outcome
Present (b).
‰ False Negatives=Experimental Group With the
Outcome Absent (c).
‰ True Negatives= Control Group With the Outcome
Absent (d).
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
7
DIAGNOSTIC VS. TREATMENT
TRIAL
‰ The Expression for the Odds Ratio (OR) =(a x d)/(b x
c).
‰ Relative risk in experimental group {[a/(a + c)]/[b/(b+
d)]} =Likelihood Ratio for a Positive Test.
‰ Relative Risk in Control Group = Likelihood Ratio for
a Negative Test.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
8
EXPLORING PUBLICATION BIAS
‰Published studies do not represent all
studies on a specific topic.
‰ Trend towards publishing statistically
significant (p < 0.05) or clinically relevant
results.
‰ Publication bias assessed by examining
asymmetry of funnel plots of estimates of
odds ratios vs. precision.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
9
EXPLORING PUBLICATION BIAS
‰Funnel plot
‰Begg’s rank correlation plot
‰Egger’s regression plot
‰Harbord’s modified radial plot
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
10
FUNNEL PLOT
‰A funnel diagram (a.k.a. funnel plot, funnel
graph, bias plot):
‰Special type of scatter plot with an
estimate of sample size on one axis vs.
effect-size estimate on the other axis
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
11
FUNNEL PLOT
‰ Based on statistical principle that sampling error
decreases as sample size increases
‰ Used to search for publication bias and to test
whether all studies come from a single population
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
12
FUNNEL PLOTS
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
13
FUNNEL PLOTS
STATA SYNTAX/COMMAND
• metafunnel ldor seldor, xlab(0(2)8) xtitle (Log
odds ratio) ytitle(Standard error of log OR)
saving(zfunnel, replace)
• metafunnel ldor seldor, xlab(0(2)8) xtitle(Log
odds ratio) ytitle (Standard error of log OR)
egger saving (eggerfunnel, replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
14
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
15
FUNNEL PLOT
2
S tandard error of log OR
1.5
1
.5
0
Funnel plot with pseudo 95% confidence limits
0
8/26/2004
2
4
Log odds rat io
Ben Dwamena: 3rd North American
Stata Users Group Meeting
6
8
16
FUNNEL PLOT
2
Standard error of log OR
.5
1
1.5
0
Funnel plot with pseudo 95% confidence limits
0
8/26/2004
2
4
Log odds ratio
Ben Dwamena: 3rd North American
Stata Users Group Meeting
6
8
17
BEGG’S BIAS TEST
‰An adjusted rank correlation method to
assess the correlation between effect
estimates and their variances.
‰Deviation of Spearman's rho from
zero=estimate of funnel plot asymmetry.
‰Positive values=a trend towards higher
levels of effect sizes in studies with smaller
sample sizes
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
18
BEGG’S BIAS TEST
STATA SYNTAX/COMMAND
metabias LogOR seLogOR,
graph(b) saving(beggplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
19
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
20
BEGG’S BIAS PLOT
Begg's funnel plot with pseudo 95% confidence limits
logOR
10
5
0
0
8/26/2004
.5
1
s.e. of: logOR
Ben Dwamena: 3rd North American
Stata Users Group Meeting
1.5
2
21
BEGG’S BIAS TEST
adj. Kendall's Score (P-Q) =
26
Std. Dev. of Score = 40.32
Number of Studies =
24
z = 0.64
Pr > |z| = 0.519
z = 0.62 (continuity corrected)
Pr > |z| = 0.535 (continuity corrected)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
22
EGGER’S REGRESSION METHOD
‰Assesses potential association b/n effect
size and precision.
‰Regression equation: SND = A + B x
SE(d)-1. SND=standard normal deviate
(effect, d divided by its standard error
SE(d)); A =intercept and B=slope. .
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
23
EGGER’S REGRESSION METHOD
‰The intercept value (A) = estimate
of asymmetry of funnel plot
‰Positive values (A > 0) indicate
higher levels of effect size in
studies with smaller sample sizes.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
24
EGGER’S REGRESSION METHOD
‰The intercept value (A) = estimate of
asymmetry of funnel plot
‰Positive values (A > 0) indicate
higher levels of effect size in studies
with smaller sample sizes.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
25
EGGER’S PLOT
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
26
EGGER’S METHOD
STATA SYNTAX/COMMAND
metabias logOR selogOR,
graph(e) saving(eggerplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
27
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
28
EGGER’S PLOT
Egger's publication bias plot
standardized effect
8
6
4
2
0
0
8/26/2004
1
2
precision
Ben Dwamena: 3rd North American
Stata Users Group Meeting
3
4
29
EGGER’S METHOD
------------------------------------------------------------Std_Eff |
Coef.
P>|t|
[95% CI]
-------------+----------------------------------------------slope |
1.737492
0.001
.8528166
2.622168
bias
1.796411
0.002
.7487423
2.84408
|
-------------------------------------------------------------
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
30
HARBORD'S MODIFIED BIAS TEST
‰Test for funnel-plot asymmetry Regresses
Z/sqrt(V) vs. sqrt (V), where Z is the efficient
score and V is Fisher's information (the
variance of Z under the null hypothesis).
‰Modified Galbraith plot of Z/sqrt(V) vs. sqrt(V)
with the fitted regression line and a
confidence interval around the intercept.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
31
HARBORD'S MODIFIED BIAS
TEST
STATA SYNTAX/COMMAND
metamodbias tp fn fp tn, graph
z(Z) v(V) mlabel(index)
saving(HarbordPlot, replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
32
15
HARBORD'S MODIFIED BIAS TEST
23
Z/sqrt(V)
10
5
5
16
19
20
14
17
2
8
7
13
11
22 12
9
24
10
6
4
1
0
21
3
18
15
0
1
Study
8/26/2004
2
sqrt(V)
3
4
regression line
90% CI for intercept
Ben Dwamena: 3rd North American
Stata Users Group Meeting
33
HARBORD'S MODIFIED BIAS TEST
----------------------------------------------------------------------------ZoversqrtV |
Coef.
Std. Err.
P>|t|
[90% Conf. Interval]
--+--------------------------------------------------------------------------
sqrtV|
2.406756
.3464027
0.000
1.811933
3.00158
bias|
.9965934
.6383554
0.133
-.0995549
2.092742
-----------------------------------------------------------------------------
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
34
TRIM AND FILL
• A rank-based data augmentation
technique used to estimate the number
of missing studies and to produce an
adjusted estimate of test accuracy by
imputing suspected missing studies.
Both random and fixed effect models
may be used to assess the impact of
model choice on publication bias.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
35
TRIM AND FILL
STATA SYNTAX/COMMAND
• metatrim LogOR seLogOR,
eform funnel print graph
id(author)saving(tweedieplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
36
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
37
TRIM AND FILL
Filled funnel plot with pseudo 95% confidence limits
6
theta, filled
4
2
0
-2
0
8/26/2004
.5
1
s.e. of: theta, filled
1.5
Ben Dwamena: 3rd North American
Stata Users Group Meeting
2
38
INVESTIGATING HETEROGENEITY
‰ Heterogeneity means that there is
between-study variation.
‰ Potential sources of heterogeneity:
1. Study population
2. Study design
3. Statistical methods,
4. Covariates adjusted for (if relevant).
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
39
GALBRAITH PLOT
‰Standardized effect vs. reciprocal
of the standard error.
‰Small studies/less precise results
appear on the left side and the
largest trials on the right end .
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
40
GALBRAITH PLOT
‰ A regression line , through the origin,
represents the overall log-odds ratio.
‰ Lines +/- 2 above regression line =95 per cent
boundaries of the overall log-odds ratio.
‰ The majority of within area of +/- 2 in the
absence of heterogeneity.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
41
GALBRAITH PLOT
STATA SYNTAX/COMMAND
• galbr LogOR seLogOR,
id(index) yline(0)
saving(gallplot, replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
42
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
43
GALBRAITH PLOT
Fitted values
b/se(b)
13.1045
5
23
b/se(b)
20
16
17
2
19
14
11
8
7 13
1512
22
21
3
18
2
0
1
9
6 24
10
4
.
-2
3.80729
0
1/se(b)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
44
L’ABBE PLOT
‰This plots the event rate in the
experimental (intervention) group
against the event rate in the control
group
‰An aid to exploring the heterogeneity of
effect estimates within a meta-analysis.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
45
L’ABBE PLOT
STATA SYNTAX/COMMAND
labbe tp fn fp tn, s(O)
xlab(0,0.25,0.50,0.75,1)
ylab(0,0.25,0.50,0.75,1) l1("TPR)
b2("FPR") saving(flabbeplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
46
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
47
L’ABBE PLOT
1
TPR
.75
.5
.25
0
0
8/26/2004
.25
.5
FPR
Ben Dwamena: 3rd North American
Stata Users Group Meeting
.75
1
48
DATA SUMMARY
STATA 8 SYNTAX
twoway (rcap dorlo dorhi Study, horizontal
blpattern(dash))(scatter Study dor, ms(O)msize(medium)
mcolor(black))(scatter DOR_with_CIs eb_dor, yaxis(2)
msymbol(i) msize(large) mcolor(black))(scatteri 26 83,
msymbol(diamond) msize(large)), ylabel(1(1)25 26
"OVERALL", valuelabels angle(horizontal)) xlabel(0 10
100 1000 10000) xscale(log) ylabel(1(1)25 26 "Pooled
Estimate", valuelabels angle(horizontal) axis(2))
legend(off) xtitle(Odds Ratio) xline(83,
lstyle(foreground)) saving(OddsForest, replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
49
DATA SUMMARY
• metan tp fn fp tn, or fixed nowt sortby(year)
label(namevar=author, yearvar=year)
t1(Summary DOR, Fixed Effects) b2(Diagnostic
Odds Ratio) saving(SDORFE, replace) force
xlabel(0,1,10,100,1000)
• metan tp fn fp tn, or random nowt sortby(year)
label(namevar=author, yearvar=year)
t1(Summary DOR, Random Effects)
b2(Diagnostic Odds Ratio) saving(SDORRE,
replace) force xlabel(0,1,10,100,1000)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
50
P o o le d E s t im a t e
9 8 .8 0 ( 1 5 .2 4 - 6 4 0 .5 0 )
9 .0 0 ( 0 .5 8 - 1 4 0 .7 1 )
7 .0 9 ( 1 .0 6 - 4 7 .4 2 )
6 .8 6 ( 1 .8 1 - 2 6 .0 1 )
6 .1 0 ( 3 .9 5 - 9 .4 2 )
5 6 .0 8 ( 4 .6 2 - 6 8 0 .3 3 )
4 6 .9 3 ( 1 7 .4 8 - 1 2 5 .9 6 )
4 5 .0 0 ( 3 .7 5 - 5 3 9 .3 8 )
4 2 9 .0 0 ( 1 4. 65 - 1 2 5 6 2 .
4 .3 3 ( 1 .0 5 - 1 7 .9 0 )
3 3 .0 0 ( 8 .8 3 - 1 2 3 .2 6 )
3 2 .8 2 ( 5 .4 3 - 1 9 8 .3 7 )
2 6 6 .0 0 ( 3 3. 46 - 2 1 1 4 .7
2 6 2 .6 6 ( 2 4. 39 - 2 8 2 9 .1
2 5 .0 0 ( 1 .7 2 - 3 6 4 .1 1 )
2 4 0 .8 8 ( 6 8. 75 - 8 4 3 .9 6
2 1 .0 0 ( 1 .4 3 - 3 0 8 .2 1 )
1 9 1 .6 7 ( 1 2. 02 - 3 0 5 5 .9
1 8 9 .0 0 ( 1 1. 36 - 3 1 4 3 .3
1 7 5 .0 0 ( 2 8. 22 - 1 0 8 5 .2
1 7 .5 0 ( 2 .6 5 - 1 1 5 .6 6 )
1 2 .3 3 ( 2 .0 4 - 7 4 .4 1 )
1 0 7 .6 7 ( 6 .5 7 - 1 7 6 3 .9 9
1 0 7 .2 3 ( 4 0. 31 - 2 8 5 .2 8
1 0 6 .3 3 ( 6 .4 0 - 1 7 6 5 .7 5
O V E R A LL
v an H o e v e n 2 0 0 2
Z o rn o za 2 0 0 4
Y u ta n i 2 0 0 1
Y a n g 2 0 01
W ahl 2004
U te c h 1 9 9 6
T se 1992
S m it h 1 9 9 8
S c h ir r m e is t e r 2 0 0 1
S c h e id h a u e r 1 9 9 6
R os to m 19 9 9
P a lm e d o 1 9 9 7
O h ta 20 0 0
N oh 1 99 8
N a k a m o t o (b ) 2 0 0 2
N a k a m o t o (a ) 2 0 0 2
L in 2 0 0 2
In o u e 2 0 0 4
H ub n er 2 0 0 0
G u lle r 2 0 0 2
G re c o 2 0 0 1
D a n fo rt h 2 0 0 2
B assa 1996
A v r il 1 9 9 6
A d le r 1 9 9 7
0
8/26/2004
10
10 0
1000
O d d s R a t io
DOR _w ith_CIs
STATA 8 FOREST PLOT
10000
Ben Dwamena: 3rd North American
Stata Users Group Meeting
51
FIXED EFFECTS META-ANALYSIS
‰Assumes homogeneity of effects across
the studies being combined the true
effect size has a common true value for
all studies.
‰In the summary estimate only the
variance of each study is taken into
account.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
52
FIXED EFFECTS FOREST PLOT
Summary DOR, Fixed Effects
Odds ratio
(95% CI)
Study
Adler (1997)
36.27 (4.27,308.02)
Avril (1996)
98.80 (10.66,916.11)
Bassa (1996)
21.00 (0.86,515.50)
Danforth (2002)
4.33 (0.80,23.49)
Greco (2001)
107.23 (33.42,344.07)
Guller (2002)
12.00 (1.23,117.41)
Hubner (2000)
429.00 (7.67,23982.81)
Lin (2002)
189.00 (6.63,5384.60)
Nakamoto (2002)
17.50 (1.84,166.04)
Nakamoto (2002)
6.86 (1.40,33.57)
Noh (1998)
208.33 (7.72,5621.57)
Ohta (2000)
60.23 (3.09,1174.51)
Palmedo (1997)
106.33 (3.74,3023.90)
Rostom (1999)
289.00 (15.83,5276.04)
Scheidhauer (1996)
107.67 (3.85,3013.13)
Schirrmeister (2001)
46.93 (14.47,152.17)
Smith (1998)
266.00 (22.50,3145.19)
Tse (1992)
9.00 (0.34,238.21)
Utech (1996)
262.66 (15.47,4459.70)
Wahl (2004)
6.10 (3.64,10.24)
Yang (2001)
25.00 (1.03,608.09)
Yutani (2001)
45.00 (2.33,867.81)
Zornoza (2004)
240.88 (54.08,1072.94)
van Hoeven (2002)
12.33 (1.45,104.97)
Overall (95% CI)
19.85 (14.16,27.82)
.1
8/26/2004
1
100
1000
Odds ratio
10000
Ben Dwamena: 3rd North American
Stata Users Group Meeting
53
RANDOM EFFECTS META-ANALYSIS
‰ Heterogeneity is incorporated into the pooled
estimate by including a between study
component of variance.
‰ Assumes sample of studies included in the
analysis is drawn from a population of
studies.
‰ Each sample of studies has a true effect
size.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
54
RANDOM EFFECTS FOREST PLOT
Summary DOR, Random Effects
Odds ratio
(95% CI)
Study
Adler (1997)
36.27 (4.27,308.02)
Avril (1996)
98.80 (10.66,916.11)
Bassa (1996)
21.00 (0.86,515.50)
Danforth (2002)
4.33 (0.80,23.49)
Greco (2001)
107.23 (33.42,344.07)
Guller (2002)
12.00 (1.23,117.41)
Hubner (2000)
429.00 (7.67,23982.81)
Lin (2002)
189.00 (6.63,5384.60)
Nakamoto (2002)
17.50 (1.84,166.04)
Nakamoto (2002)
6.86 (1.40,33.57)
Noh (1998)
208.33 (7.72,5621.57)
Ohta (2000)
60.23 (3.09,1174.51)
Palmedo (1997)
106.33 (3.74,3023.90)
Rostom (1999)
289.00 (15.83,5276.04)
Scheidhauer (1996)
107.67 (3.85,3013.13)
Schirrmeister (2001)
46.93 (14.47,152.17)
Smith (1998)
266.00 (22.50,3145.19)
Tse (1992)
9.00 (0.34,238.21)
Utech (1996)
262.66 (15.47,4459.70)
Wahl (2004)
6.10 (3.64,10.24)
Yang (2001)
25.00 (1.03,608.09)
Yutani (2001)
45.00 (2.33,867.81)
Zornoza (2004)
240.88 (54.08,1072.94)
van Hoeven (2002)
12.33 (1.45,104.97)
Overall (95% CI)
42.54 (20.88,86.68)
.1
8/26/2004
1
1000
100
Odds ratio
10000
Ben Dwamena: 3rd North American
Stata Users Group Meeting
55
CUMULATIVE META-ANALYSIS
‰studies are sequentially pooled by
adding each time one new study
according to an ordered variable. For
instance, the year of publication; then, a
pooling analysis will be done every time
a new article appears.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
56
CUMULATIVE META-ANALYSIS
‰metacum LogOR seLogOR,
eform id(author) effect(f) graph
cline saving(year_fcummplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
57
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
58
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
59
CUMMULATIVE META-ANALYSIS
D a nf ort h 2 002
Li n 20 02
P al m ed o 19 97
S c h eid ha uer 1 996
N o h 19 98
T s e 19 92
B as s a 19 96
H u bn er 20 00
O ht a 20 00
G rec o 2 00 1
U t e c h 19 96
S m i th 1 99 8
S c h irrm e is t er 20 01
G ul ler 20 02
A v ri l 199 6
R o s t om 19 99
A dl er 19 97
v an H oe v en 2 00 2
N a k am o t o(a) 2 002
N a k am o t o(b) 2 002
I no ue 20 04
Z orn oz a 2 004
Y ut a ni 2 001
Y an g 20 01
W a hl 20 04
7.6 73 8 7
2 39 82 .8
L og O R
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
60
INFLUENCE ANALYSIS
‰studies are pooled according
influence of a trial on overall effect
defined as the difference between the
effect estimated with and without the
trial
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
61
INFLUENCE ANALYSIS
‰metaninf tp fn fp tn, id(author)
saving(influplot, replace)
save(infcoeff, replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
62
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
63
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
64
INFLUENCE ANALYSIS
Meta-analysis estimates , given named study is omitted
Low er C I Limit
Estim ate
Upper C I Limit
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
4.75
8/26/2004
5.41
6.53
7.88
Ben Dwamena: 3rd North American
Stata Users Group Meeting
10.30
65
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
66
ROC PLOT
‰A scatter plot true positive fraction
(sensitivity) vs. false positive fraction (1specificity)
‰aids in visualization of range of results from
primary studies
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
67
ROC PLOT
twoway (scatter TPF FPF, sort ) (lfit uTPR FPF, sort
range(0 1) clcolor(black) clpat(dash) clwidth(vthin)
connect(direct)) (lfit sTPR FPF, sort range(0 1)
clcolor(black) clpat(dot) clwidth(vthin)
connect(direct)), ytitle(Sensitivity) ylabel(0(.1)1, grid)
xtitle(1-Specificity) xlabel(0(.1)1, grid) title(ROC Plot
of SENSITIVITY vs. 1-SPECIFICITY, size(medium))
legend(pos(3) col(1) lab(1 "Observed Data") lab(2
"Uninformative Test") lab(3 "Symmetry Line"))
saving(ROCplot, replace) plotregion(margin(zero))
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
68
R O C P lo t o f S E N S IT IV IT Y v s . 1 -S P E C I F I C IT Y
S ensitiv ity
.4 .5 .6
.7
.8
.9
1
ROC PLOT
0
.1
.2
.3
O b s e r v e d D a ta
U n i n fo rm a tiv e T e s t
S y m m e t r y L in e
0
8/26/2004
.1
.2
.3
.4
.5
.6
1 -S p e c ific ity
.7
.8
.9
1
Ben Dwamena: 3rd North American
Stata Users Group Meeting
69
LINEAR REGRESSION
MODELS
ORDINARY LEAST SQUARES METHOD:
Studies are weighted equally
WEIGHTED LEAST SQUARES METHOD:
Weighted by the inverse variance weights of the
odds ratio, or simply the sample size
ROBUST-RESISTANT METHOD:
Minimizes the influence of outliers
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
70
REGRESSION ANALYSIS
‰ Logit transformations of the TP rate (sensitivity) and FP rate
(1 - specificity).
D=ln(DOR) =logit(TPR) – logit(FPR)
‰ Differences in logit transformations, D, regressed on sums
of logit transformations, S.
S=logit(TPR)+logit(FPR)
‰ Logit(TPR)=natural log odds of a TP result and logit(FPR)
=natural log of the odds of a FP test result.
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
71
ACCURACY-THRESHOLD
STATA SYNTAX/COMMAND
• twoway (scatter D S, sort msymbol(circle)) (lfit
tfitted S, clcolor(black) clpat(solid) clwidth(thin)
connect(direct))(lfit wfitted S, clcolor(black)
clpat(dash) clwidth(thin) connect(direct)),
ytitle(Discriminatory Power/D) xtitle(Diagnostic
Threshold/S) title(REGRESSION PLOT)
legend(lab(1 "Observed Data")lab(2
"EWLSR")lab(3 "VWLSR"))saving(regplot,
replace) xline(0) yscale(noline)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
72
ACCURACY-THRESHOLD
1
2
Discriminatory Power/D
3
4
5
6
REGRESSION PLOT
-4
-2
0
Diagnostic Threshold/S
Observed Data
VWLSR
8/26/2004
2
4
EWLSR
Ben Dwamena: 3rd North American
Stata Users Group Meeting
73
SUMMARY ROC CURVE
‰Back transformation of logistic regression
to conventional axes of sensitivity [TPR]
vs. (1 – specificity) [FPR]) with the equation
‰TPR = 1/{1 + exp[- a/(1 - b )]} [(1 FPR)/(FPR)](1 + b )/(1 - b ).
‰Slope (b) and intercept (a) are obtained
from the linear regression analyses
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
74
SUMMARY ROC CURVES
•
•
STATA SYNTAX/COMMAND
twoway (scatter TPF FPF, sort msymbol(circle) msize(medium)
mcolor(black))(fpfit tTPR FPF, clpat(dash)clwidth(medium)
connect(direct ))(fpfit wTPR FPF, clpat(solid)clwidth(medium)
connect(direct ))(lfit uTPR FPF, sort range(0 1) clcolor(black)
clpat(dash) clwidth(thin) connect(direct)) (lfit sTPR FPF, sort
range(0 1) clcolor(black) clpat(dot) clwidth(medium)
connect(direct)), ytitle(Sensitivity/TPF) yscale(range(0 1)) ylabel(
0(.2)1,grid ) xtitle(1-Specificity/FPF) xscale(range(0 1))
xlabel(0(.2)1, grid) legend(lab(1 "Observed Data")lab(2
"EWLSR")lab(3 "VWLSR")lab(4 "RRLSR")lab(5 "Uninformative
Test") lab(6 "Symmetry Line") pos(3) col(1)) title(SUMMARY
ROC CURVES) graphregion(margin(zero)) saving(aSROCplot,
replace)
8/26/2004
Ben Dwamena: 3rd North American
Stata Users Group Meeting
75
SUMMARY ROC CURVES
.8
1
SUMMARY ROC CURVES
0
.2
Sensitivity/TPF
.4
.6
Observed Data
EWLSR
VWLSR
RRLSR
Uninformative Test
0
8/26/2004
.2
.4
.6
1-Specificity/FPF
.8
1
Ben Dwamena: 3rd North American
Stata Users Group Meeting
76