Exercise 14-3: Power PROJECT BACKGROUND Your team is evaluating the impact of a local governance program that provides training and outreach at the community level on participatory budgeting. Your primary outcome of interest is increased citizen participation in the budgeting and monitoring process, particularly from marginalized groups. The evaluation team has settled on a clustered randomized evaluation where communities are randomly assigned to the treatment or control group. During the planning phase, the IE team conducts revised power calculations based on data from a similar project implemented recently in the country by another donor. TEAM TASKS Consider the following scenarios and questions related to the power analysis: 1. In doing power analysis, how would you set the relative level of significance for these cases (i.e. would you require a higher level of significance for (a) or (b))? a. Pilot project will be the basis for large scale funding allocation b. One-off project where we think we have two effective approaches but want to get a sense of whether one is more effective than the other. 2. Your IE contractor recommends using a minimum detectable effect size of 0.6 SD. You notice that this is the same effect size that was identified in the similar program funded by the World Bank. Do you agree with this? a. If so, why? b. If not, what would you recommend (or how would you identify a different effect size)? 3. No secondary data exists, and data collection is expected to be expensive. The budget only allows for a sample size sufficient to reach a power of 30%. The team feels there is no flexibility in the assumptions in the calculation. Do you recommend continuing with the evaluation? 4. Since the evaluation uses a clustered design, you are concerned about the intracluster correlation (icc). You know that the final sample size, due to budget limitations, will be 2,000 (equally split between treatment and control). The 1 program will operate in 150 communities. You have two choices for how to structure your sample: a. Collecting data from 100 individuals in each of 20 communities, 10 treatment and 10 control, for a total sample size of 2,000 b. Collecting data from 10 individuals in each of 200 communities, 100 treatment and 100 control, for a total sample size of 2,000 4.1 To minimize the effect of icc on power, which scenario would you recommend? 4.2 Which scenario do you think would be the cheapest to implement? 2

© Copyright 2021 Paperzz