Sample Design
Part 2
Produced in Collaboration between World Bank Institute and
the Development Data Group (DECDG)
Copyright 2010, The World Bank Group. All Rights Reserved.
Sample Size
Sample size must be selected to meet:
 reliability criteria
 cost constraints
• Steps to determine sample size
1. Set reliability criteria and budget constraints
2. Develop an equation to express the relationship between
reliability and sample size
3. Develop an equation to express the relationship between
cost and sample size
4. Select sample size that satisfies both equations
2
Copyright 2010, The World Bank Group. All Rights Reserved.
CV, a Measure of Reliability
•
•
•
•
CV – Coefficient of Variation
Relative measure of an estimate’s accuracy
Generally expressed in %; e.g., 10% CV
Better to have small CVs than large CVs
3
Copyright 2010, The World Bank Group. All Rights Reserved.
Estimate of CV
Standard Error Estimate
cv( x) 
Labor Force Estimate
se( x)

x
4
Copyright 2010, The World Bank Group. All Rights Reserved.
Estimated Standard Error se(x)
• Square root of the estimated variance
• Often compared to a simple random sample of size n from a
population N
N 2s2  N  n 
v( x)  de


n 1  N 
x  Nx
• Design effect de accounts for differences of a particular sample from
the “ideal” of simple random sampling
Copyright 2010, The World Bank Group. All Rights Reserved.
Estimated Standard Error se(x)
• For a proportion p, the parameter s2 takes the simple form np(1p)/(n-1)
N 2 p(1  p)  N  n 
v(Np)  de


n 1  N 
• A “safe” design effect de=4 can be useful
Copyright 2010, The World Bank Group. All Rights Reserved.
Factors Affecting Sample Size
• National & Subnational Reliability
• Response Rates
• Clustering (households sampled, not persons, can sample
clusters of households)
• Type of Sampling (Stratification? 2-stage? Are first-stage
areas small or large?)
• Advanced Techniques
– Sample Rotation
– Population controls in estimation
Copyright 2010, The World Bank Group. All Rights Reserved.
Sample Size Formulas
 
N de  S
n 2
2
2
X CV  Nde S
2

2

 
N de P1  P  /( N  1)
n 2
2
2
X CV  N deP1  P  /( N  1)
3


Copyright 2010, The World Bank Group. All Rights Reserved.
Sample Size Formulas
• In previous slide, large letters that represent assumed
“known” values replace estimates
• S2 = NP(1-P)/(N-1) in second formula for a proportion P
• P is a proportion with respect to N, for ex:
–
UE/N = unemployment /(adult population)
–
NOT the unemployment rate UER
–
UER = UE/(labor force)
Copyright 2010, The World Bank Group. All Rights Reserved.
Final Sample Size Determination
• Several variables of interest?
• National and subnational reliability criteria?
• If the sample size is too costly, then rethink
– Relax reliability criteria?
– Increase allowed cost?
Copyright 2010, The World Bank Group. All Rights Reserved.