Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
Translating science problems
to statistical questions :
some challenges and examples
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Constantine Frangakis
April 6, 2015
1. Background
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
experiences based on collaborations with the Depts of Radiology (JF
Geschwind) and Psychiatry (C Lyketsos)
2. Translation versus decidability
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Accuracy of translation versus decidability
Accuracy of translation of a scientific question usually requires
placing the question within a larger hypothetical framework than
the one accessible (design choices, data types etc).
Decidability can trade-off against accuracy given the accessible
framework
This trade-off affects and can lead to poor choice of
the statistical question
the design
the analysis methods
3.1 Translation and statistical question
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
Medical question:
Liver cancer is assessed by many staging systems. How do we
compare their value in predicting survival ?
1. Background
2. Translation
versus
decidability
3. Examples
Design of accessible data:
Cohort [JHH];
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Georgiades, CS, Liapi, E, Frangakis, CE, Park, J, Kim, HW, Hong, K, and
Geschwind, JFH (2006). J Vasc Interv Radiol 17, 1619–1624 (with discussion in
18, 455–456)
3.1 Translation and statistical question
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
Translation of medical question: “How do we assess(a) the value(b) of
a staging system in predicting survival(c) ? ”
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Observations:
to predict survival(c) , we need to say how we would value(b) it.
if the outcome is fully observed, the literature usually uses R 1 or
R 2 for the value of a predictor
but if the outcome is censored, the literature usually uses a
model-based value (e.g., based on Cox regr).
this is because the assessment(a) is more difficult with censoring
can we keep a clinically meaningful value(b) , and change the way
of assessment(a) ?
3.2 Translation and statistical analysis
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Medical question:
A clinical trial showed a treatment vs usual care (placebo) benefit.
Also, among patients who had responded after 3 weeks on placebo,
68% remained responders at week 9; whereas of the patients who
had not responded at 3 weeks on placebo, only 10% responded at
week 9.
Can we estimate the benefit that patients “who do not respond after
3 wks of placebo” would have after 9 more wks on treatment ?
Design of accessible data:
3, 9 week data of response for patients in each of treatment/placebo
group
Porsteinsson et al. (2014). JAMA 311, 682–691 (with editorial)
3.2 Translation and statistical question
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Translation of medical question : what is the benefit that patients
“who do not respond after 3 wks of placebo” would have after 9
more wks on treatment ?
Observations:
understanding the question needs an expanded framework that
did not occur in the trial:
to consider patients who do not respond under 3 wks placebo,
then assign them to treatment/placebo and measure the
outcome after 9 wks.
3.2 Translation and statistical question
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Translation of medical question: “Can we estimate the benefit
focusing on patients who “do not respond” after 3 wks of placebo ?”
Observations:
we consider instead the outcomes at week 9, say 9t, that a
person who has factually been assigned to placebo and did no
respond (3p = 0) would have had if, contrary to fact, that
person had originally been assigned to treatment.
what would be a usual approach to estimate the benefit for
these patients ?
can we use a different analysis that does not make the
assumption of the usual method ?
3.2 Translation and statistical question
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
3.3 Translation and design
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
Medical question (context as before)
What sample size would we need to test for benefit in a 3wk placebo
runin + 9 wk experiment trial ?
2. Translation
versus
decidability
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
Design of accessible data:
3, 9 week data of response for patients in each of treatment/placebo
group
3.3 Translation and design
Translating
science problems
to statistical
questions :
some challenges
and examples
Constantine
Frangakis
1. Background
2. Translation
versus
decidability
Translation of medical question: “ What sample size would we need
to test for benefit in a 3wk placebo runin + 9 wk experiment trial ”
Observations:
3. Examples
3.1 Translation
and statistical
question
3.2 Translation
and statistical
analysis
3.3 Translation
and design
the answers differ by a 3-fold depending on what translation to
statistical question is used