Research Series: Research Design
and Methods, Part 2
Paul A. Estabrooks, PhD
Objectives
• Describe the purpose of cohort studies.
• Describe cohort studies
• Describe case control studies
• Describe randomized controlled trials.
• Balancing internal and external validity
Quick Review
• Independent variable
• The manipulated variable that determines a change in the dependent
variable (or outcome).
• Dependent variable (objective; subjective)
• The outcome variable that is hypothesized to depend on the context of
the independent variable.
• Qualitative research
• Focus is on exploration and context
• Quantitative research
• Focus is on a quantifiable outcome
• Mixed methods
Planning your study

Who is your target audience:




Inclusion criteria
Exclusion criteria
Are they easily identified? Do YOU have access to
the population or information?
How will you recruit your sample?


Can YOU do it or do you have someone that can
do it?
How will you determine if the sample is
representative of the target population?
Planning your study

What are the patient outcomes you are
interested in changing?





Are these data readily available to you?
In the medical record?
Who (and how) will the data be gathered?
Will you use any qualitative data?
Who will implement your intervention?


Is anyone else affected by the changes you plan to
make?
How will you determine if the intervention is
delivered with fidelity?
Planning your study



How will you assess sustained patient
changes?
How will you assess sustained changes in
practice procedures?
How does your research question align
with Carilion Clinic’s mission?
Understanding type 1, 2, & 3 error

Type 1 error (related to probability): An intervention or
exposure is unrelated to the dependent variable, but the
study suggests it is.

Type 2 error (related to power): An intervention or exposure
is related to the dependent variable, but the study suggests it
isn’t

Type 3 error (related to implementation): An intervention or
exposure is related to the dependent variable, but was not
implemented as intended and findings are non-significant
Research Designs

What is a cohort study?




Advantages?




One or more samples
Prospective
Relationship between outcomes and exposure is assessed
Can match subjects on characteristics
Typically uses available outcome data (efficient)
Good for examining different exposures over time
Disadvantages?


Susceptible to unrecognized confounding variables
Infer rather than determine causality
0.5
0
Average Weight Loss
-0.5
Pounds
-1
-1.5
Matched Control-Group Visit
-2
-2.5
-3
Intervention
-3.5
IFG to
Class
9
Class to 3
Months
3 to 6
Months
6 to 9
Months
6 to 12
Months
Research Designs

What is a case control?




Advantages?




Cohort of cases and a cohort of controls
Observational and Retrospective
Relationship between risk factors and outcomes as odds ratios
Good for identifying potential patterns of risk factors and outcomes
Uses available outcome data (efficient)
Can examine multiple independent variables (risk factors) at the same
time
Disadvantages?



Susceptible to unrecognized confounding variables
If recall is used—potential memory biases
Infer rather than determine causality
11
12
Research Designs

What is a Randomized Controlled Trial?




Advantages?




At least 2 groups (experimental and control)
Prospective
Causation between the independent variable and the outcome
Randomization reduces likelihood of confounding
Well developed statistical procedures
Stronger causal inference
Disadvantages?



External validity is typically low
High participant burden and resultant attrition
Really expensive to conduct
Move More

Target audience: Insufficiently active adult patients.

Recruitment: Waiting room PA screener, Physician
referral, & active outreach call from health educator

Participants were randomly assigned to the teambuilding or enhanced standard care control.
N=56 Health
Educators
Health Educators
interested
N=36
R
ALED
N= 18
Fit Ex
N=18
Telephone Introduction
Online Training
In person training
64% Adoption at
study level
No significant diffs
between adopters
and non adopters
Delivery agent training and delivery
*
*χ2(1)=7.2, p<.01
Balancing Internal and External
Validity
In case control, cohort, and RCTs it is important to consider

three temporal issues:
Reach



Implementation (initial and ongoing)




Initial number, proportion and representativeness of your sample
relative to the target population
Ongoing number, proportion and representativeness based on
exposure
Degree to which exposure is consistent
For RCTs or cohort trials intervention delivery as intended
Cost and resource needs
Effectiveness



Changes in dependent variable (initial and beyond exposure)
Changes in quality of life
Monitoring potential negative outcomes
Let’s get back to your designs!

What is the best design for your work?

What are some new things you should think
about based on todays information?

What adaptations should you make to
improve either internal or external validity