AUTHOR ACCEPTED MANUSCRIPT
FINAL PUBLICATION INFORMATION
The Development Impact of a Best Practice Seasonal Worker Policy
The definitive version of the text was subsequently published in
Review of Economics and Statistics, 96(2), 2014-05
Published by MIT Press and found at http://dx.doi.org/10.1162/REST_a_00383
THE FINAL PUBLISHED VERSION OF THIS MANUSCRIPT
IS AVAILABLE ON THE PUBLISHER’S PLATFORM
This Author Accepted Manuscript is copyrighted by World Bank and published by MIT Press. It is posted here by
agreement between them. Changes resulting from the publishing process—such as editing, corrections, structural
formatting, and other quality control mechanisms—may not be reflected in this version of the text.
You may download, copy, and distribute this Author Accepted Manuscript for noncommercial purposes. Your license
is limited by the following restrictions:
(1) You may use this Author Accepted Manuscript for noncommercial purposes only under a CC BY-NC-ND
3.0 IGO license http://creativecommons.org/licenses/by-nc-nd/3.0/igo.
(2) The integrity of the work and identification of the author, copyright owner, and publisher must be preserved
in any copy.
(3) You must attribute this Author Accepted Manuscript in the following format: This is an Author Accepted
Manuscript by Gibson, John; McKenzie, David The Development Impact of a Best Practice Seasonal Worker
Policy © World Bank, published in the Review of Economics and Statistics96(2) 2014-05 CC BY-NC-ND
3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo http://dx.doi.org/10.1162/REST_a_00383
© 2017 World Bank
Review of Economics and Statistics ONLINE APPENDIX
John Gibson and David McKenzie
“The Development Impact of a Best Practice Seasonal Worker Policy”
Online Appendix 1 : Robustness to Attrition in the Vanuatu sample
There was almost no attrition in the Tongan sample. In contrast, 16.7 percent of the PS-2 screened
Vanuatu sample had attrited by the final wave. The attrition rate was slightly higher for the non-RSE
households (19.3 percent) than for the RSE households (13.7 percent), although this difference is not
significant. Attrition is significantly associated with lower baseline asset levels and lower baseline per
capita income. This is consistent with the observations from the field, whereby some poor non-RSE
households told our enumerators that their lives had not changed at all, so that they refused to answer
the same questions again and again.
Given our estimation is based on difference-in-differences, attrition will only bias our results if it is
associated with changes in outcomes, not levels. A first test of this is to see whether the change in
income between rounds 1 and 2 is associated with whether or not the household then appears in the
final wave (round 4). A probit examining this finds no significant relationship (p=0.64). A second
approach is to construct Lee (2009) bounds on the size of the treatment effect. The key identifying
assumption for implementing these bounds is a monotonicity assumption that participation in the RSE
affects sample selection only in one direction. Since non-RSE households are more likely to attrit, this
amounts to assuming that no household who attrits when in the RSE would have not attrited had they
not participated in the RSE. This appears reasonable to a first-order, although may be violated for the
small number of households in which divorce occurred.
We then construct a lower bound for the RSE treatment effect by trimming out the top 7 percent of RSE
households in terms of changes in per capita income between round 1 and round 4. This would be the
treatment effect under the unlikely scenario that it was the non-RSE households who experienced large
positive income shocks who decided to attrit. Using the PS-2 screened sample, we get difference-indifferences point estimates of 27,822 vatu (p=0.02) for the effect on per capita income, and 0.301
(p=0.01) for the effect on log per capita income. An upper bound can likewise be formed by eliminating
the 7 percent of RSE households with the smallest (most negative) income change between round 1 and
round 4. Doing this yields a 53,354 vatu (p=0.001) effect for per capita income and 0.399 (p=0.001)
effect for log per capita income. These bounds are thus quite tight, and show that the large increase in
income in Vanuatu from the RSE is robust to attrition.
Online Appendix 2: Measurement of Income and Consumption
Household income and expenditure can be difficult to measure in developing country settings in which
agricultural production for home use, self-employment, and family labor are all common. We used
several years of experience in surveying Pacific Island households, along with questions used in the
national income and expenditure surveys of the retrospective countries to attempt to measure these
outcomes as accurately as possible.
Household income is measured as the sum of net remittances (remittance inflows less remittance
outflows, including RSE remittances as an inflow), cash sales of agricultural production, the value of food
produced for own consumption, wage and salary income, other income such as interest or rent income,
and repatriated earnings that RSE migrants carry back with them instead of remitting while abroad. Cash
sales of agricultural production are measured by asking separately about sales from fish, crops, livestock
and poultry, and forestry products. The value of food produced for own consumption asks the quantity
and value of 19 most commonly produced items (e.g. bananas, cassava, chicken, fish, coconut, two
types of taro, etc.) consumed in the last week. Household cash expenditure is measured via a 20
category recall module, with reference periods ranging between one week and six months depending on
the source of expenditure. This is aggregated to the semi-annual level and added to the value of food
produced for own use to arrive at total expenditure.
We measure the income and expenditure of household members in Tonga and Vanuatu, with the
seasonal worker counted as part of the household for periods when in the home country but not
included when in New Zealand. Migrants earn and likely consume more than the average remaining
household member while they are abroad, so we are therefore underestimating the average impact on
individuals originally present in the household, even though we get the average impact for individuals
present in the household in the sending countries.1 The travel expenses and costs while in New Zealand
of the migrant do not enter into either the income or the consumption expenditure of the remaining
household unit in Tonga or Vanuatu. Instead we treat migration like a household business, in which the
net income to the household from this business (remittances and repatriated savings) comes after the
migrant has paid travel expenses and costs while in New Zealand (including income tax) from their gross
earnings.
Online Appendix 3: Robustness to Alternative Estimation Techniques
With four rounds of survey data, and some households entering into RSE status at different points in
time, the propensity-score screened difference-in-differences and fixed-effects method used in the
paper offers a clear and transparent way to use all of the data. Moreover, the work of Crump et al.
(2009) and Angrist and Pischke (2009) have shown this methodology works well in practice in
approximating the experimental results obtained in a U.S. work experience program.
An alternative approach is to use the propensity scores to directly estimate a treatment effect using
propensity-score matching methods.2 The most common matching estimators just use a single-round of
follow-up data (e.g. Dehejia and Wahba, 2002) and are not suited for our purposes. Heckman et al.
(1997) and Smith and Todd (2005) consider a difference-in-differences matching strategy, with
estimator given by:
1
Our per-capita consumption estimates will also be conservative for the impact on well-being if migrants consume
more than the average household member when they are present in the household.
2
A further alternative method is to control for a polynomial in the propensity score in the regression. Since our
determinants of participation in the RSE are all time-invariant, this approach would be subsumed under our fixed
effects estimator.
Where
and
are the values of the outcome of interest in the first and second rounds of data
collection respectively, the treatment or program participation occurs between rounds 1 and 2, I1 and I0
are the sets of program participants and non-participants respectively, S is the common support of their
propensity score distributions, W(i,j) is a weight determined by the propensity score distance between
observations i and j , and n1 is the number of participants in the program. Alternative matching
estimators differ in how they determine the weighting function W(). Three common approaches are
radius matching, which puts equal weight on all non-participants within a specified radius or distance of
the treated unit in terms of propensity score; stratified matching, which compares participants to nonparticipants in the same block, where blocks are strata of the propensity score; and kernel matching,
which takes a kernel-weighted average over non-participants, with those units whose propensity score
is closest to the treated unit getting highest weight. Users then have a choice of what radius to use or
what kernel to use in implementing this.
When more than 2 rounds of surveys are available, additional modification of this technique is required.
Given our desire to maximize power by using as many rounds of follow-up data as possible and
estimating an average effect of ever participating in the RSE over these first two years of the program,
we modify the difference-in-differences matching procedure to be:
Since most RSE households join between wave 1 and wave 2, in order to be able to estimate this using
standard matching software (the Becker and Ichino pscore and attk, attr, and atts commands in STATA),
we average over the last three rounds, calculate the difference in this average from the baseline value,
and then implement matching on this. For observations not present in all three follow-up rounds, we
take an average over the rounds they are available. We consider five versions of the matching estimator:





Kernel matching with a Gaussian kernel
Kernel matching with an Epanechnikov kernel and the default bandwidth of 0.06.
Radius matching with the default radius of 0.10.
Radius matching with a radius of 0.05
Stratified matching
We implement each of these five methods first within the common support of the propensity-score PS-1
(which does not restrict on whether households applied to the RSE), and then within the common
support of PS-2 (which is restricted just to households which had an RSE applicant).
We carry this out for both countries for per-capita income and per-capita consumption, and report the
results in Appendix Table 2. The first column of the appendix shows the Crump et al. (2009) propensity-
score pre-screened difference-in-difference estimator reported in Table 3. The remaining columns show
the propensity-score estimates. We see that in all cases the difference-in-difference point estimates lie
within the range of parameter estimates obtained from the different propensity score methods.
Moreover, the general pattern of significant results for per-capita income, and only sometimes
significant results for per-capita expenditure continues to hold with these alternative methods. As a
result, we view our results as robust to these alternative estimation methods.
Online Appendix 4: Impacts of RSE Participation on Specific Asset Ownership
Appendix Table 3 examines the impact of participating in the RSE on ownership of specific household
durable assets, with results discussed in the text.
Online Appendix 5: Changes in Durable Goods Ownership for Non-Migrant Households
Appendix Table 4 examines the changes in durable asset ownership for the PS-1 control group
households between the baseline survey and the wave 4 survey two years after the baseline. If the nonmigrant households are experiencing growing incomes as a result of multiplier effects from RSE
households spending their remittances, or from increased job opportunities, then we should expect to
see asset growth. Of course this is suggestive only, since asset growth may occur anyway if households
are naturally accumulating assets over time, and, since we do not have control groups for our control
groups since most communities had someone participate in the RSE, we can’t rule out the possibility
that zero asset growth represents an improvement on what would have happened without RSE
involvement.
We see in Tonga that non-migrant households have fewer DVD players and ovens, and slightly more
kerosene cookers, but experience no changes in livestock or other durable asset holdings. This suggests
the living conditions of these households have not changed much over these two years, making it
unlikely that we are obtaining biased impacts due to large spillovers from the RSE to non-RSE
households. In Vanuatu, the two big changes are a massive increase in cellphone usage (from 25 percent
at baseline to 85 percent at endline), and an increase in DVD ownership, along with a marginally
significant increase in chicken stocks. The increase in cellphone ownership coincides with the aggressive
roll-out of cellphone coverage by Digicel in Vanuatu, and so likely reflects falling prices and increased
access more than increased household wealth. There were no significant changes in ownership of other
assets in Vanuatu during this time. Therefore it seems unlikely that there are large spillovers in Vanuatu
either.
Online Appendix 6: Qualitative impressions of impact from community leaders
Appendix Table 5 summarizes the results of questions in both the household and community leader
surveys intended to measure qualitatively the impressions of the broader community-level impacts of
the RSE. The main text discusses these results.
RSE Status by Survey Round
The table below shows the number of households interviewed in each survey round that have
participated in the RSE up to that survey date, as well as how many new and repeat workers there were
by this round.
Appendix Table 1: RSE Status by Survey Round
TONGA
VANUATU
Number ever New in
Repeat
Number ever
New in
Repeat
in RSE
RSE
RSE worker
in RSE
RSE
RSE worker
Round 1
0
0
0
0
0
0
Round 2
180
180
0
102
102
0
Round 3
187
6
50
124
15
75
Round 4
195
10
77
118
9
67
Notes:
Data refer only to households interviewed in the round specified. New in RSE denotes
households which did not have an RSE worker in the previous round, but do in this round.
Repeat RSE worker denotes households which already had an RSE worker in the previous
round, and have increased the duration of time in the RSE program since the previous round.
Appendix Table 2: Robustness to Alternative Estimation Techniques
Propensity-Score Matching Estimates
Gaussian Epanechnikov Radius
Radius
Table 3 Kernel
Kernel
Matching Matching
DD result Matching Matching
R=0.1
R=0.05
PANEL A: TONGA
Per Capita Income
Based on PS-1
278.4*** 214.3***
230.9***
321.8*** 293.9***
(105.3)
(74.4)
(74.5)
(91.2)
(84.4)
Based on PS-2
233.1* 237.5**
236.7***
235.3** 231.7**
(129.5)
(96.5)
(78.0)
(103.8) (106.2)
Per Capita Expenditure
Based on PS-1
127.1
109.0
117.2*
148.7** 130.8**
(81.8)
(71.0)
(69.1)
(73.8)
(66.3)
Based on PS-2
104.6
67.6
72.8
92.7
95.1
(104.1)
(74.3)
(73.8)
(81.4)
(83.4)
PANEL B: VANUATU
Per Capita Income
Based on PS-1
44441*** 40566*
48104***
n.a.
n.a.
(15659) (20840)
(17389)
Based on PS-2
48241*** 39064**
34706
30058
29614
(16388) (18228)
(23631)
(18577) (18610)
Per Capita Expenditure
Based on PS-1
12353** 15960*
9637
n.a.
n.a.
(6131)
(9185)
(11958)
Based on PS-2
13020** 14952
17528
5321
5048
(5559) (11115)
(12943)
(8033)
(8063)
Notes:
n.a. not available since program did not converge.
Robust standard errors in parentheses, *** p<0.01, ** p<0.05, * p<0.1
Stratified
Matching
228.1***
(75.2)
223.0**
(107.5)
115.6*
(66.4)
26.4
(103.3)
41543**
(20274)
50916**
(22075)
18265**
(8464)
15211
(11509)
Appendix Table 3: Impact of RSE participation on durable household assets
TONGA
VANUATU
Ever in RSE
Ever in RSE
Asset
PS-1
PS-2
PS-1
PS-2
Pigs
-0.202
-0.193
-1.031
-0.0802
(0.225)
(0.303)
(0.715)
(0.443)
Cattle
0.0354
0.0407
0.199
0.416
(0.0654) (0.0891) (0.506)
(0.496)
Chickens
-0.164
-0.169
-7.203
-0.225
(0.262)
(0.318)
(5.960)
(1.381)
Cellphone
0.0563* 0.0611 -0.0284 -0.0374
(0.0334) (0.0435) (0.0450) (0.0469)
Radios/Stereos
0.0445* 0.0275 0.288*** 0.261***
(0.0229) (0.0282) (0.0601) (0.0651)
Television Sets
0.0794*** 0.0736** 0.0320 -0.0214
(0.0273) (0.0349) (0.0498) (0.0534)
DVD Player
0.0543 0.0740* 0.190*** 0.235***
(0.0349) (0.0423) (0.0591) (0.0643)
Computer
0.000151 -0.00176 0.0666* 0.0623*
(0.0110) (0.0141) (0.0340) (0.0327)
Gas or electric oven
0.0868** 0.117** 0.102** 0.125***
(0.0374) (0.0461) (0.0451) (0.0432)
Kerosene Cooker
-0.0804***-0.0946*** 0.0345 -0.000831
(0.0274) (0.0363) (0.0242) (0.0197)
Boats
0.0018
0.000
0.111** 0.0827*
(0.0022) (0.000) (0.0447) (0.0466)
Bicycles
0.0686* 0.0853* 0.0511
0.0381
(0.0362) (0.0473) (0.0394) (0.0400)
Cars or Pick-up trucks
0.0524
0.0767
0.0188 0.00625
(0.0408) (0.0496) (0.0223) (0.0223)
Number of Households
372
271
Notes:
All asset regressions control for baseline asset levels
Robust standard errors shown in parentheses
269
225
Appendix Table 4: Was the Control Group Increasing Assets over this period?
TONGA
VANUATU
Means among PS-1
Means among PS-1
in Control Group
in Control Group
Wave 1 Wave 4 P-value
Wave 1 Wave 4 P-value
Pigs
5.46
5.67
0.427
3.64
3.60
0.946
Cattle
0.45
0.40
0.423
1.60
2.10
0.155
Chickens
5.08
5.00
0.830
11.16
16.87
0.090
Cellphone
0.75
0.74
0.689
0.25
0.85
0.000
Radios/Stereos
0.77
0.80
0.233
0.46
0.40
0.147
Television Sets
0.86
0.85
0.677
0.24
0.26
0.577
DVD Player
0.54
0.44
0.006
0.27
0.40
0.000
Computer
0.04
0.04
0.890
0.06
0.07
0.432
Gas or electric oven
0.72
0.64
0.012
0.18
0.13
0.119
Kerosene Cooker
0.09
0.13
0.052
0.03
0.03
0.899
Boats
0.48
0.45
0.858
0.11
0.13
0.269
Bicycles
0.33
0.33
0.786
0.09
0.10
0.532
Cars or Pick-up trucks
0.02
0.01
0.433
0.03
0.03
0.712
P-value tests for difference in means between wave 1 and wave 4
Appendix 5: Qualitative impressions of the broader impacts of the RSE
TONGA
Better No change
Opinions of RSE workers
Percent who think involvement in RSE:
Made Family Life (at 1 year)
Made Community Life (at 1 year)
Opinions of non-RSE households in these communities
Percent who think involvement of community members in RSE has:
Made Community Life (at 6 months)
Made Community Life (at 2 years)
Made availability of paid jobs (at 6 months)
Made availability of paid jobs (at 2 years)
Made schooling opportunities for children in the community (at 6 months)
Made schooling opportunities for children in the community (at 2 years)
Opinions of Community Leaders
Percent who think involvement of community members in RSE has:
Made Community Life (at 1 year Tonga/6 months Vanuatu)
Made Community Life (at 2 years)
Made availability of paid jobs (at 1 year Tonga/6 months Vanuatu)
Made availability of paid jobs (at 2 years)
Made schooling opportunities for children in the community (at 1 year Tonga/6 months Vanuatu)
Made schooling opportunities for children in the community (at 2 years)
Community leader perception of overall impact for the community (at 6 months)
Community leader perception of overall impact for the community (at 1 year)
Community leader perception of overall impact for the community (at 2 years)
Worse
Better
VANUATU
No change
Worse
81.8
83.0
11.9
15.9
6.3
1.1
48.6
27.1
50.0
70.0
1.4
2.9
49.4
10.6
38.7
39.4
18.4
52.9
46.7
88.2
49.0
60.2
77.0
47.2
3.8
1.2
12.3
0.4
4.6
0.0
16.1
14.6
13.2
7.0
27.2
12.4
67.4
77.7
67.0
72.9
64.1
72.9
18.5
7.7
19.8
20.2
8.7
14.7
3.0
68.0
2.0
89.3
10.0
69.3
97.0
25.3
98.0
10.7
90.0
26.7
0.0
6.7
0.0
0.0
0.0
4.0
32.6
41.8
25.4
29.1
70.9
0.0
24.1
74.1
1.9
Positive No effect Negative Positive No effect Negative
72.2
13.0
14.8
98.3
0.0
1.7
92.0
2.7
5.3