Chapter 12
Healthy Steps: The First Three Years
12. Variation in Implementation
Variation in Implementation
Over the course of the evaluation several hypotheses were proposed to explain potential variation in the
implementation of Healthy Steps. To better inform future programs and funders, we examined two areas of
potential variation in site characteristics. These related to variation in the National Program Office’s rating of the
quality of implementation at each site and variation in the extent to which the site had experience implementing the
program at the time the family entered care.
Irrespective of whether a family received care at a “top rated” site or whether they enrolled after sites had more
experience with the program, families in the intervention group within each category compared had a higher odds of
receiving Healthy Steps services than control families.
We found stronger effects on receipt of some services for intervention families at “top rated” sites than for those at
less highly rated sites.
The variation in effects between intervention children enrolled later (when sites had more experience in delivering
program services) and those enrolled earlier differed for randomization and quasi-experimental sites. Children
enrolled later in the program had a significantly lower odds than those enrolled earlier of receiving some Healthy
Steps services and an age-appropriate well child visit at 24 months of age. However, these differential effects were
seen only at QE sites. At RND sites, intervention children enrolled later had a higher odds of receiving information
on community resources as well as diagnosis and referral for developmental problems. Intervention children
enrolled later, at both RND and QE sites, had a significantly higher odds of being up-to-date on their vaccinations by
24 months of age.
In both comparisons, the effects of raising the already high levels of service receipt to even higher levels generally
did not seem to affect outcomes. These findings reinforce key informant reports, which indicated that the Healthy
Steps components were made available at the start of the program and were implemented at all 15 sites.
12-1
Chapter 12
Healthy Steps: The First Three Years
12. VARIATION IN IMPLEMENTATION
12.1. Introduction
Over the course of the evaluation several hypotheses were
proposed to explain potential variation in the implementation of
Healthy Steps (HS). Key informants at the sites associated
smoother implementation with strong consistent leadership to
assure the structural changes needed to accommodate the
program, thorough orientation and buy-in at all levels, and a welldeveloped training and orientation program. The National
Program leadership and funders also developed impressions about
implementation from their contacts with the sites---by and large
forming similar views to those expressed above and noting
variation in the “quality” of implementation. Other early
childhood interventions, most recently, Early Head Start (Love et
al., 2002) have noted variations in implementation due to
differences in program characteristics.
To better inform future programs and funders, we examined two
areas of potential variation in characteristics related to
implementing the program: (1) variation in the National Program
Office’s (NPO’s) assessment of the quality of implementation at
each site; and (2) variation in the extent to which the site had
experience implementing the program at the time the family
entered care.
This evaluation component addresses the first question of the
evaluation:
1. How did providers and local foundations implement the Healthy
Steps program in their sites and what were the factors that
facilitated or impeded implementation?
In the first comparison, we found stronger effects on receipt of
some services for intervention families at “top rated” sites.
However the effect of raising the already high levels of service
receipt to even higher levels did not seem to affect outcomes.
In the second analysis, intervention children enrolled later had a
lower odds than children who enrolled earlier of receiving some
services and a higher odds of receiving others. There were few
differential effects on outcomes. The findings from these two
analyses reinforce key informant reports, which indicated that the
HS components were made available at the start of the program
and were implemented at all 15 sites.
12-2
Interpreting Odds Ratios
Regression results for dichotomous
outcomes, adjusting for baseline
family characteristics and site, are
reported in terms of odds ratios. An
odds ratio of greater than 1 indicates
that subjects in the intervention
group were more likely to report a
given characteristic than were
subjects in the control group; an
odds ratio of less than 1 indicates
that subjects in the intervention
group were less likely to report a
given characteristic than were
subjects in the control group. An
odds ratio of 1 indicates that there
was no difference between
intervention and control groups.
Ninety-five percent confidence
intervals are estimated; when this
interval does not include one, it
indicates a statistically significant
difference (at the 0.05 level) between
the intervention and control group.
Chapter 12
Healthy Steps: The First Three Years
12.2 Variation in National Program Site Ratings
The intent of the first comparison was to assess whether the
variation in implementation observed by the NPO affected receipt
of services by families and in turn parent and child outcomes. In
this analysis, the quality ratings the NPO assigned to each site
provided the measure of variation. The program ratings likely
capture not only the “quality” of program implementation but also
the “quality” of the site overall.
The NPO used an interactive statistical technique to elicit site
ratings from staff at the NPO, The Commonwealth Fund, and
Boston University (Trochim, 1985; Trochim, 1997; Trochim,
1998). For purposes of both developing the site rating factors and
in providing ratings, the NPO team could draw upon multiple
sources of information on implementation at the sites that had
been available to them throughout the project. During two
monitoring visits to sites, the team met with key clinicians, staff,
and funders, and observed HS activities. After each visit, a formal
report was circulated to staff at the NPO offices. Prior to the
second monitoring visit, each site completed a self-report form
that provided information on clinical and operational issues. In
addition, regular Boston University technical assistance
teleconferences were monitored and summarized by NPO staff,
who circulated these to staff throughout the program.
The combined information from the interactive process provided
a set of key site factors and quantitative rankings of the quality of
implementation. The NPO analysis suggested that high quality
sites were characterized by strong “buy-in” by practice staff,
presence of a champion for HS, strong communication with the
practice, and teamwork.
Seeking to answer the question, “Is the impact of HS greater at
sites with the highest NPO ratings than at the less highly rated
sites?”, we assessed the difference in the effects of HS between
families in care at the four “top rated” sites and those at the other
11 sites. The same co-variates were included in these analyses as
were included in the overall analyses.
Children in the intervention group, whether at a “top rated” site
or less highly rated site had significantly higher odds than
children in the control group of receiving HS services (Table
12.1). Children at the four “top rated” sites had a higher odds than
children at the other sites of receiving four or more HS services
and information on community resources, discussing six topics,
and receiving books. With few random exceptions, there were no
12-3
Chapter 12
Healthy Steps: The First Three Years
differential effects on parent and child outcomes. The effect of
raising the already high levels of service receipt to even higher
levels did not seem to affect outcomes.
12.3 Variation in Experience with Implementation
In the second analysis, we hypothesized that as sites gained
experience with the program, they might become more efficient at
delivering HS services to families or that the nature or quality of
the services might improve. If found, these changes might affect
families in terms of receipt of services or program effects. Receipt
of services and program effects were compared for families that
entered care when sites had substantial experience implementing
HS (after the first 100 families had been enrolled) and when they
had less program experience (when 100 or fewer families had been
enrolled). We included the same co-variates in these analyses as
were in the overall analyses.
Intervention families, whether they enrolled early or late, had a
far higher odds than their counterparts in the control group of
receiving HS services (Table 12.2). Intervention children had
higher odds of being vaccinated on time (although the differences
for up-to-date vaccination were not significant for children who
enrolled earlier). Mothers of children in the intervention group
had a higher odds of being satisfied with care, regardless of when
their children enrolled.
We found variation in the impact of HS on receipt of some
services and on some outcomes between the two groups but they
were not consistent between RND and QE sites. At QE sites,
intervention children who enrolled later (after the first 100
children) had a significantly lower odds than children who
enrolled earlier (first 100 children enrolled) of receiving several of
the HS services. These services included one or more home visits,
developmental assessments, and books to read. At QE sites, laterenrolled intervention children also had a lower odds than those
enrolled earlier of receiving their 24-month well child visit on
time. At RND sites, later-enrolled intervention children had a
higher odds of receiving information on community resources as
well as diagnosis and referral for developmental problems. At
both RND and QE sites, intervention children in the later
enrolled group also had a higher odds of being up-to-date on their
vaccinations by 24 months of age.
The meaning of these results is not clear. It is possible that as the
HS Specialist’s workload increased at the QE sites, it became
more difficult to schedule home visits, to schedule 24-month office
visits, and to provide developmental assessments. HS Specialists
12-4
Chapter 12
Healthy Steps: The First Three Years
may have had less time to spend developing relationships with
families. Also, they may have taken advantage of referrals to
outside resources rather than providing the service on site.
Having more experience in providing services seemed to promote
up-to-date vaccination. Nonetheless, the timing of the child’s
entry into the program seemed to have little influence on parent
and child outcomes.
12-5
Chapter 12
Healthy Steps: The First Three Years
Healthy Steps Intervention Families:
„ Significantly More Likely than Control
¡ Significantly Less Likely than Control
† Significant Effect Not Noted
‹ Significant Difference in Effects Between “Top Rated” and “Less Highly Rated” Groups
Table 12.1. Results of Regression Analyses Comparing Impacts on Receipt of Services and Program Effects for Families
at 4 “Top Rated” Sites with Families at 11 Other Sites: Adjusted Odds Ratios and 95% Confidence Intervals
Families at 4 “Top
Rated” Sites
Families at 11 “Less
Highly Rated” Sites
Difference Between
Groups
(t statistic)
„ 34.68
„ 15.55
‹ 3.57
(23.69, 50.79)
(12.41, 19.47)
RECEIPT OF SERVICES
Received 4 or More HS Services from Practice
( Excluding Home Visiting )
Someone Visited Parent or Child in Their Home Since 6
Months
Someone in the Practice Talked with Parent or Gave
them Information more than 6 topics
Given Developmental Assessment by Someone in
Practice
Received Books to Read to Their Child from Practice
Received Information About Community Resources
From Someone in Practice
Child Received Services (Other Than Testing) For
Problem With Walking, Taking, Hearing, or Using
His/Her Hands
„ 16.21
„ 14.74
(11.56, 22.73)
(11.92, 18.24)
„ 14.22
„ 8.78
(9.75, 20.76)
(7.02, 10.98)
„ 8.87
„ 7.51
(6.29, 12.52)
(6.12, 9.21)
„ 75.63
„ 20.84
(46.55, 122.88)
(16.54, 26.24)
„ 6.46
„ 3.59
(4.67, 8.94)
(2.94, 4.38)
† 0.85
(0.56, 1.29)
„ 1.42
† 0.47
‹ 2.16
† 0.82
‹ 4.71
‹ 3.03
‹ -1.98
(1.07, 1.89)
SATISFACTION WITH CARE
Someone Went Out of Way to Help
Disagree That MDs And NPs Provided “Support”To
Parent
Disagree That MDs And NPs “Listened” To Parent
Disagree That MDs And NPs Respected Parent’s
Knowledge, Knew What Was Going On with the Child,
and Made Them Feel Like They Were Doing a Good Job
Overall Perception Of Care At Practice ( Good /
Excellent )
Overall Perception that Doctors and Nurses at the
Practice Are Easy to Reach by Telephone
„2.17
„2.08
(1.64, 2.89)
¡0.24
(0.14, 0.39)
¡0.49
(0.31, 0.79)
(1.74, 2.48)
¡0.42
(0.32, 0.54)
¡0.74
(0.57, 0.97)
†0.7
†0.83
(0.45, 1.08)
(0.64, 1.09)
†1.47
†1.1
(0.89, 2.44)
(0.82, 1.48)
†0.87
†0.94
(0.58, 1.29)
(0.76, 1.16)
†0.26
†-1.95
†-1.48
†-0.69
†0.99
†-0.34
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: MATERNAL DEPRESSION
Percentage of Mothers with Depressive Symptoms (Score
of 11 or Higher on Modified CES-D)
†1.43
†1
(0.99, 2.08)
(0.8, 1.25)
†1.62
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENT BEHAVIOR
(Modified ) Parent Behavior Checklist: nurturing % More
Nurturing (≥63)
(Modified ) Parent Behavior Checklist: nurturing % Less
Nurturing (≤44)
(Modified) Parent Behavior Checklist: Higher
Expectations ( >1 SD above mean)
(Modified) Parent Behavior Checklist: Lower
Expectations( >1 SD below mean)
†1.15
†1.08
(0.85, 1.55)
(0.89, 1.31)
†0.77
†0.98
(0.42, 1.4)
(0.72, 1.33)
†1.03
†0.97
(0.71, 1.49)
(0.79, 1.2)
†0.98
†1.14
(0.67, 1.43)
(0.9, 1.46)
12-6
†0.35
†-0.71
†0.27
†-0.68
Chapter 12
Healthy Steps: The First Three Years
Families at 4 “Top
Rated” Sites
Families at 11 “Less
Highly Rated” Sites
Difference Between
Groups
(t statistic)
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENT BEHAVIOR
(Modified): Parent Response to Misbehavior: Severe
Physical Discipline: % ever slapped child in face or
spanked with object
(Modified): Parent Response to Misbehavior: Discipline:
% Using Harsh Discipline (≥6)
(Modified): Parent Response to Misbehavir: Discipline: %
Using More Reasoning (≥9)
†0.64
†0.76
(0.37, 1.11)
(0.55, 1.04)
†0.64
†0.79
(0.38, 1.06)
(0.61, 1.03)
†1
†1.17
(0.72, 1.4)
(0.96, 1.42)
†-0.53
†-0.75
†-0.78
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENT PRACTICES
Parent Showed Picture Books Every day or More Often
Family Follows At Least 3 Routines at Bedtime, Naptime,
or Mealtime
Mother And Father Equally Or Father Usually Takes
Child To Well Child Visits
Parent Lowered Temperature on Water Heater
Family Uses Covers on Electric Outlets
†0.81
†1.02
(0.6, 1.09)
(0.85, 1.22)
†1.05
†1.04
(0.78, 1.41)
(0.87, 1.24)
†1.32
†1.04
(0.94, 1.84)
(0.85, 1.28)
†0.95
†1.06
(0.71, 1.25)
(0.89, 1.27)
†0.79
(0.49, 1.28)
Family Has Safety Latches on Cabinets
Parent Knows a Number to Call if Concerned Child May
Have Swallowed Something Harmful
Current Smokers in Household Who Smoke Outside
„1.32
†-1.32
†0.03
†1.17
†-0.69
†-1.81
(1.01, 1.72)
†0.88
†1.11
(0.67, 1.17)
(0.94, 1.32)
†1.5
†0.99
(0.93, 2.43)
(0.76, 1.3)
†0.93
†1.33
(0.52, 1.66)
(0.93, 1.91)
†-1.39
†1.47
†-1.02
CHILD OUTCOMES: HEALTH AND DEVELOPMENT
Child is Very Healthy
Child’s Health is Excellent
Since Child Came Home from Hospital S/He Had Been
Seriously Ill
Age Child spoke Two-Word Sentences (did not speak 2word sentences before 24 months of age)
Age Child First Walked Without Holding On
Parent Very Satisfied with Child’s Eating Habits
Parent Very Satisfied With Child’s Sleeping Habits
Parent Very Satisfied With Child’s Bowel Habits
Parent Very Satisfied With Progress Walking And Talking
Very Satisfied With How Well Child Understands What
Respondent Says
†1.39
†1.1
(0.996, 1.94)
(0.9, 1.36)
†0.86
†1.01
(0.66, 1.13)
(0.85, 1.2)
†0.88
†0.9
(0.61, 1.26)
(0.7, 1.16)
†0.94
†0.98
(0.69, 1.28)
(0.8, 1.19)
†1.28
†0.92
(0.99, 1.66)
(0.78, 1.08)
†0.85
†0.92
(0.72, 1.00)
¡0.76
(0.64, 0.91)
(0.71, 1.19)
†1.13
(0.86, 1.48)
†0.93
†0.94
(0.69, 1.25)
(0.79, 1.13)
†1.1
†0.92
(0.74, 1.64)
(0.72, 1.18)
†0.9
†0.88
(0.55, 1.47)
(0.66, 1.16)
†1.16
†-0.98
†-0.14
†-0.21
†2.14
†0.51
„2.41
†-0.07
†0.76
†0.11
CHILD OUTCOMES: CHILD’S PROBLEM BEHAVIOR
CBCL: % More Aggressive (≥14)
†1.06
(0.73, 1.55)
CBCL: % More Destructive (≥8)
„1.4
†0.99
†1.01
(0.61, 1.62)
(0.77, 1.33)
12-7
†-1.21
(1.11, 1.76)
†-0.06
Chapter 12
Healthy Steps: The First Three Years
Families at 4 “Top
Rated” Sites
Families at 11 “Less
Highly Rated” Sites
Difference Between
Groups
(t statistic)
†1.35
†0.86
(0.94, 1.95)
(0.68, 1.09)
‹2.04
CHILD OUTCOMES: INJURIES AND HEALTH CARE UTILIZATION
Injured Badly Enough To See Doctor
Number Of Emergency Room Visits In Past Year (1 or
more )
One or More Emergency Room Visit for Injury-Related
Causes in Past Year
Number Of Hospitalizations ( 1 or More Times In Past
Year)
†0.95
†1.07
(0.7, 1.27)
(0.9, 1.27)
†0.91
†0.75
(0.58, 1.42)
(0.57, 0.97)
†1.57
†0.94
(0.89, 2.77)
(0.66, 1.34)
†-0.71
†0.75
†1.51
CHILD OUTCOMES: AGE-APPROPRIATE WELL CHILD CARE
1 Month Well Child Visit (Visit within 41 days of birth)
2 Month Well Child Visit (Visit between 42 days (1.5 months)
and 92 days(3 months), inclusive)
4 Month Well Child Visit (Visit between 93 days (3 months)
and 151 days (5 months), inclusive)
6 Month Well Child Visit (Visit between 152 days (5 months)
and 213 days (7 months), inclusive)
9 Month Well Child Visit (Visit between 244 days (8 months)
and 305 days (10 months), inclusive)
12 Month Well Child Visit (Visit between 336 days (11
months) and 397 days (14 months), inclusive)
15 Month Well Child Visit (Visit between 427 days (14
months) and 488 days (17 months), inclusive)
18 Month Well Child Visit (Visit between 519 days (17
months) and 580 days (20 months), inclusive)
24 Month Well Child Visit (Visit between 701 days (23
months) and 762 days (28 months), inclusive)
„4.94
„1.71
(2.11, 11.55)
(1.15, 2.52)
„2.51
„1.86
(1.68, 3.75)
(1.5, 2.3)
„2.07
„1.42
(1.44, 2.97)
(1.17, 1.72)
„1.89
„1.39
(1.35, 2.64)
(1.16, 1.66)
„2.08
„1.55
(1.55, 2.8)
(1.31, 1.83)
„1.86
„1.77
(1.28, 2.72)
(1.44, 2.18)
„1.69
„1.83
(1.26, 2.27)
(1.53, 2.2)
„2.73
„2.74
(2.03, 3.66)
(2.28, 3.3)
„1.83
„1.66
(1.23, 2.71)
(1.34, 2.06)
‹2.23
†1.29
†1.78
†1.59
†1.71
†0.23
†-0.48
†-0.04
†0.42
CHILD OUTCOMES: AGE-APPROPRIATE VACCINATIONS
DTP 1
DTP 3
MMR 1
„2.15
„1.71
(1.33, 3.47)
(1.35, 2.18)
„1.99
„1.36
(1.46, 2.7)
(1.15, 1.61)
„1.78
„1.45
(1.2, 2.64)
(1.16, 1.82)
†0.83
‹2.12
†0.89
CHILD OUTCOMES: CHILD UP-TO-DATE ON VACCINATIONS
Up To Date At 24 Months (4 DTP, 3 OPV/IPV, 1MMR)
„1.78
„1.36
(1.27, 2.49)
(1.07, 1.74)
12-8
†1.25
Chapter 12
Healthy Steps: The First Three Years
Healthy Steps Intervention Families:
„ Significantly More Likely than Control
¡ Significantly Less Likely than Control
† Significant Difference Not Noted
‹ Significant Difference in Effects Between Experience Groups
Table 12.2. Receipt of Developmental Services and Healthy Steps Program Effects for Children Enrolled When the
Site Had More Experience or Less: Adjusted Odds Ratios and 95% Confidence Intervals
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to 100
children enrolled at time of
entry
Difference Between Groups
(t statistic)
RECEIPT OF SERVICES
Received 4 or More HS Services from Practice (Excluding Home Visiting)
Randomization
„ 17.75
„ 16.11
Quasi-Experimental
Pooled
(12.66, 24.88)
„ 20.95
„ 26.02
(15.33, 28.62)
(18.64, 36.33)
„ 18.58
„ 20.64
(14.86, 23.24)
(16.37, 26.02)
Someone Visited Parent or Child in Their Home Since 6 Months (30-33 Months)
Randomization
„ 11.6
„ 15.62
Quasi-Experimental
Pooled
(8.47, 15.89)
† 0.51
(11.56, 22.46)
(11.22, 21.75)
„ 16.02
„ 21.99
(12.12, 21.17)
(16.31, 29.65)
„ 13.11
„ 17.99
(10.69, 16.07)
(14.48, 22.34)
† -1.37
† -0.88
† -1.68
‹ -2.22
‹ -2.88
Someone in the Practice Talked with Parent or Gave them Information More than 6 Topics @ 30-33
Months
Randomization
„ 8.54
„ 8.58
† -0.02
Quasi-Experimental
Pooled
(6.03, 12.11)
(6.01, 12.24)
„ 11.39
„ 13.55
(8.23, 15.77)
(9.47, 19.4)
„ 9.49
„ 10.66
(7.51, 11.99)
(8.31, 13.68)
Given Developmental Assessment by Someone in Practice
Randomization
„ 6.79
„ 7.46
Quasi-Experimental
Pooled
(4.89, 9.44)
„ 7.45
„ 11.01
(5.61, 9.89)
(7.94, 15.28)
„ 6.87
„ 9.16
(5.56, 8.49)
(7.27, 11.54)
Received Books to Read to Their Child from Practice
Randomization
„ 32.88
Quasi-Experimental
Pooled
(5.35, 10.4)
(22.52, 48.01)
„ 25.85
(18.05, 37.02)
„ 24.09
„ 37.74
(17.26, 33.6)
(25.85, 55.09)
„ 26.1
„ 30.09
(20.46, 33.29)
(23.28, 38.89)
Received Information About Community Resources From Someone in Practice
Randomization
„ 4.19
„ 2.93
Quasi-Experimental
Pooled
(3.11, 5.63)
† -0.78
† -0.45
‹ -2.09
‹ -2.11
† 1.11
‹ -2.33
† -1.01
‹ 2.26
(2.18, 3.94)
„ 4.97
„ 4.94
(3.81, 6.48)
(3.73, 6.54)
„ 4.55
„ 3.96
(3.74, 5.53)
(3.24, 4.84)
12-9
† -0.84
† 0.04
† 1.35
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to 100
children enrolled at time of
entry
Difference
Between Groups
(t statistic)
Child Diagnosed or Referred for Problem with Walking, Talking, Hearing, or Using
His/Her Hands
Randomization
¡ 0.62
‹ 3.02
† 1.34
Quasi-Experimental
(0.91, 1.99)
„ 1.79
(1.23, 2.6)
Pooled
„ 1.51
(1.15, 1.97)
(0.39, 0.98)
† 1.37
(0.9, 2.09)
† 0.91
† 1.25
‹ 3.11
(0.67, 1.24)
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: SATISFACTION WITH
CARE
Someone Went Out of Way to Help
Randomization
Quasi-Experimental
Pooled
„ 1.98
„ 2.15
(1.50, 2.60)
(1.63, 2.83)
„ 2.00
„ 2.25
(1.58, 2.54)
(1.74, 2.91)
„ 1.99
„ 2.24
(1.67, 2.38)
(1.86, 2.69)
Disagree that MDs and NPs Provided “Support” to Parent 6
Randomization
¡ 0.41
¡ 0.47
Quasi-Experimental
Pooled
(0.26, 0.65)
(0.3, 0.74)
(0.18, 0.39)
(0.27, 0.56)
(0.24, 0.43)
(0.32, 0.56)
¡ 0.27
¡ 0.32
Disagree that MDs and NPs “Listened” to Parent
Randomization
¡ 0.61
(0.38, 0.97)
Quasi-Experimental
¡ 0.55
(0.37, 0.8)
Pooled
¡ 0.57
(0.43, 0.76)
¡ 0.39
¡ 0.42
† 0.76
† -0.51
† -0.82
† -1.07
† -0.52
† -1.59
† -1.5
† -0.78
(0.49, 1.17)
† 0.8
† -1.72
(0.55, 1.15)
† 0.79
† -1.91
(0.6, 1.04)
Disagree that MDs and NPs Respected Parent’s Knowledge, Knew What Was Going
On with the Child, and Made Them Feel Like Thy Were Doing a Good Job 8
Randomization
† 0.96
† 1.09
† -0.52
Quasi-Experimental
Pooled
(0.62, 1.48)
(0.71, 1.65)
(0.47, 0.96)
(0.44, 0.95)
¡ 0.67
¡ 0.65
† 0.78
† 0.81
(0.59, 1.02)
(0.61, 1.07)
Overall Perception of Care at Practice (good/excellent )
Randomization
† 1.05
† 1.34
Quasi-Experimental
Pooled
(0.65, 1.69)
† 0.14
† -0.28
† -0.86
(0.82, 2.2)
† 1.31
† 1.12
(0.87, 1.97)
(0.74, 1.7)
† 1.19
† 1.18
(0.87, 1.62)
(0.86, 1.62)
† 0.64
† 0.04
Overall perception that Doctors and Nurses at the Practice Are Easy to Reach by
Telephone
Randomization
† 0.93
† 1.13
† -1
Quasi-Experimental
Pooled
(0.67, 1.29)
(0.81, 1.58)
† 0.84
† 0.79
(0.62, 1.14)
(0.58, 1.09)
† 0.89
† 0.95
(0.72, 1.11)
(0.76, 1.2)
12-10
† 0.34
† -0.52
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Difference Between
Groups (t statistic)
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: MATERNAL
DEPRESSION
CES-D 11 Or Higher
Randomization
† 1.02
† 1.34
† -1.39
Quasi-Experimental
Pooled
(0.71, 1.45)
(0.95, 1.88)
† 1.02
† 1.03
(0.75, 1.39)
(0.74, 1.43)
† 1.02
† 1.19
(0.81, 1.29)
(0.95, 1.51)
† -0.05
† -1.16
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENT BEHAVIOR
(Modified) Parent Behavior Checklist: % More Nurturing (> 63)
Randomization
† 1.06
† 1.18
Quasi-Experimental
Pooled
(0.78, 1.44)
† 1.08
† 1.09
(0.84, 1.4)
(0.84, 1.43)
† 1.06
† 1.14
(0.87, 1.29)
(0.93, 1.39)
(Modified) Parent Behavior Checklist: % Less Nurturing (< 44)
Randomization
† 1.17
† 0.82
Quasi-Experimental
Pooled
(0.73, 1.89)
† -0.65
(0.88, 1.6)
† -0.07
† -0.65
† 1.22
(0.49, 1.38)
† 0.81
† 0.9
(0.52, 1.26)
(0.56, 1.44)
† 0.98
† 0.89
(0.71, 1.35)
(0.63, 1.25)
† -0.42
† 0.5
(Modified) Parent Behavior Checklist: Higher Expectations (>1 SD above mean)
Randomization
† 0.92
† 1.01
† -0.48
Quasi-Experimental
Pooled
(0.66, 1.29)
(0.73, 1.41)
† 1.03
† 0.92
(0.78, 1.38)
(0.67, 1.25)
†1
† 0.97
(0.8, 1.24)
(0.78, 1.21)
† 0.70
† 0.22
(Modified) Parent Behavior Checklist: Lower Expectations (> 1 SD below mean)
Randomization
† 1.13
† 0.78
† 1.63
Quasi-Experimental
Pooled
(0.78, 1.62)
(0.53, 1.16)
† 1.1
† 1.36
(0.79, 1.54)
(0.97, 1.9)
† 1.1
† 1.09
(0.86, 1.4)
(0.85, 1.39)
† -1.16
† 0.10
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENTING
BEHAVIOR
(Modified): Parent Response to Misbehavior: % slapped child in face or spanked with
object
Randomization
† 0.9
† 0.75
† 0.60
Quasi-Experimental
Pooled
(0.54, 1.5)
† 0.66
(0.42, 1.02)
(0.44, 1.28)
0.69
(0.43, 1.11)
† 0.74
† 0.71
(0.53, 1.02)
(0.5, 1.01)
† -0.19
† 0.16
(Modified): Parent Response to Misbehavior: % Using Harsh Discipline (>6)
Randomization
† 0.63
† 0.91
† -1.38
Quasi-Experimental
Pooled
(0.4, 1)
(0.59, 1.4)
† 0.86
† 0.73
(0.6, 1.23)
(0.49, 1.1)
† 0.74
† 0.78
(0.56, 0.97)
(0.59, 1.05)
12-11
† 0.68
† -0.38
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Difference Between
Groups (t statistic)
(Modified): Parent Response to Misbehavior: Discipline: % Using More Reasoning (> 9)
Randomization
† 1.23
† 1.1
† 0.61
Quasi-Experimental
Pooled
(0.89, 1.7)
(0.8, 1.51)
† 0.97
† 1.27
(0.75, 1.26)
(0.95, 1.69)
† 1.06
† 1.2
(0.87, 1.29)
(0.97, 1.48)
† -1.71
† -1.07
PARENT KNOWLEDGE, BELIEFS, PSYCHOLOGICAL HEALTH: PARENT PRACTICES
Parent Showed Picture Books Every Day or More Often
Randomization
† 0.92
† 0.96
Quasi-Experimental
Pooled
(0.69, 1.21)
† 0.99
† 0.98
(0.77, 1.26)
(0.75, 1.27)
† 0.95
† 0.97
(0.79, 1.14)
(0.8, 1.17)
Family Followed at Least 3 Routines at 30-33 Months
Randomization
† 0.9
† 1.02
Quasi-Experimental
Pooled
(0.68, 1.2)
† -0.28
(0.73, 1.27)
† 0.06
† -0.17
† -0.71
(0.76, 1.35)
† 0.99
† 1.24
(0.78, 1.25)
(0.96, 1.61)
† 0.96
† 1.15
(0.8, 1.15)
(0.95, 1.39)
† -1.6
† -1.73
Mother and Father Equally or Father Usually Take Child to Well Child Visits
Randomization
† 1.11
† 1.18
† -0.35
Quasi-Experimental
Pooled
(0.81, 1.52)
† 1.13
(0.74, 1.32)
(0.84, 1.51)
† 1.05
† 1.17
(0.85, 1.3)
(0.95, 1.45)
Parent Lowered Temperature on Water Heater
Randomization
† 1.19
Quasi-Experimental
Pooled
(0.9, 1.57)
Pooled
† 0.81
† 0.88
(0.69, 1.14)
† 0.96
† 1.1
(0.81, 1.15)
(0.92, 1.32)
(0.7, 1.61)
Pooled
„ 2.05
† 1.2
† 0.85
(0.58, 1.24)
† 1.14
† 1.2
(0.86, 1.5)
(0.9, 1.61)
(0.89, 1.51)
† -1.22
† -0.65
† -1.27
‹ -2.34
† 1.07
† 1.59
† -0.32
† 0.5
(0.82, 1.39)
† 1.05
† 0.9
(0.84, 1.33)
(0.7, 1.15)
† 1.11
† 0.98
(0.93, 1.32)
(0.82, 1.17)
12-12
† -0.93
(1.24, 3.4)
(0.82, 1.76)
Family Has Safety Latches on Cabinets
Randomization
† 1.16
Quasi-Experimental
„ 1.45
† -0.8
(1.1, 1.91)
(0.63, 1.03)
Family Uses Covers on Electric Outlets
Randomization
† 1.06
Quasi-Experimental
(0.87, 1.61)
† 0.99
† 1.19
† 1.27
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Difference Between
Groups (t statistic)
Parent Knows a Number to Call if Concerned Child May have Swallowed Something
Harmful
Randomization
† 1.2
† 1.56
† -0.94
Quasi-Experimental
Pooled
(0.76, 1.88)
† 0.9
(0.68, 1.41)
(0.62, 1.32)
† 1.06
† 1.13
(0.8, 1.41)
(0.85, 1.52)
Current Smokers in Household Who Smoke Outside
Randomization
† 1.14
Quasi-Experimental
Pooled
(0.96, 2.54)
† 0.98
(0.67, 1.94)
† 1.01
† 0.39
† -0.38
† 0.38
(0.6, 1.71)
† 1.03
† 1.76
(0.62, 1.71)
(0.97, 3.2)
† 1.1
† 1.34
(0.77, 1.58)
(0.91, 1.97)
† -1.65
† -0.91
CHILD OUTCOMES: HEALTH AND DEVELOPMENT
Child Is Very Healthy
Randomization
Quasi-Experimental
Pooled
† 1.05
† 1.33
(0.76, 1.44)
(0.97, 1.81)
† 1.13
† 1.26
(0.86, 1.5)
(0.94, 1.69)
† 1.09
„ 1.27
(0.89, 1.35)
(1.03, 1.57)
† -1.33
† -0.67
† -1.28
Child’s Health Is Excellent
Randomization
Quasi-Experimental
Pooled
† 0.99
† 0.82
(0.76, 1.3)
(0.63, 1.06)
† 1.16
† 0.89
(0.92, 1.47)
(0.7, 1.14)
† 1.08
† 0.86
(0.91, 1.29)
(0.72, 1.03)
Since Child Came Home from Hospital S/He Had Been Seriously Ill
Randomization
† 0.85
† 0.79
Quasi-Experimental
Pooled
(0.59, 1.23)
† 1.29
† 1.94
„ 2.26
† 0.37
(0.54, 1.14)
† 0.88
† 1.06
(0.62, 1.26)
(0.74, 1.51)
† 0.87
† 0.92
(0.68, 1.12)
(0.71, 1.18)
† -0.92
† -0.32
Age Child Spoke Two-Word Sentences (did not speak 2-word sentences before 24
months of age)
Randomization
† 0.78
† 0.99
† -1.29
Quasi-Experimental
Pooled
(0.56, 1.08)
(0.73, 1.36)
† 1.14
† 0.92
(0.88, 1.47)
(0.7, 1.22)
† 0.98
† 0.95
(0.8, 1.19)
(0.78, 1.17)
† 1.39
† 0.26
Age Child First Walked without Holding On (walked before 12 months of age)
Randomization
† 1.23
† 1.26
† -0.18
Quasi-Experimental
Pooled
(0.95, 1.59)
(0.98, 1.63)
† 0.84
† 0.86
(0.68, 1.04)
(0.68, 1.08)
† 0.99
† 1.02
(0.84, 1.17)
(0.86, 1.2)
12-13
† -0.20
† -0.24
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Parent Very Satisfied with Child’s Eating Habits
Randomization
† 0.83
Quasi-Experimental
Pooled
(0.76, 1.26)
(0.65, 1.00)
(0.69, 1.10)
¡ 0.80
¡ 0.83
(0.70, 0.97)
Parent Very Satisfied with Child’s Sleeping Habits
Randomization
† 0.88
Quasi-Experimental
Pooled
(0.67, 1.15)
Pooled
† 0.87
† 0.92
¡ -0.64
† –1.16
† 0.82
† 0.47
(0.63, 1.07)
† 0.83
† 0.86
(0.67, 1.10)
† 0.86
† 0.85
(0.73, 1.03)
(0.71, 1.01)
(0.64, 1.14)
† -1.10
(0.78, 1.09)
(0.66, 1.05)
Parent Very Satisfied with Child’s Bowel Habits
Randomization
† 0.85
Quasi-Experimental
† 0.98
(0.65, 1.07)
Difference Between
Groups (t statistic)
† 0.88
† –0.25
† 0.18
† -0.17
(0.66, 1.17)
†1
† 0.94
(0.79, 1.28)
(0.72, 1.21)
† 0.94
† 0.94
(0.78, 1.13)
(0.78, 1.13)
† 0.49
† 0.04
CHILD OUTCOMES: HEALTH AND DEVELOPMENT
Parent Very Satisfied with Progress Walking and Talking
Randomization
† 1.03
„ 1.65
Quasi-Experimental
Pooled
(0.69, 1.53)
(1.07, 2.54)
(0.52, 0.99)
(0.63, 1.3)
¡ 0.72
† 0.91
† 0.84
† 1.16
(0.66, 1.07)
(0.88, 1.52)
† -1.91
† -1.25
‹ -2.17
Very Satisfied with How Well Child Understands What Respondent Says
Randomization
† 1.14
† 1.37
† -0.66
Quasi-Experimental
Pooled
(0.72, 1.81)
(0.86, 2.17)
† 0.69
† 0.68
(0.47, 1.01)
(0.45, 1.03)
† 0.86
† 0.91
(0.65, 1.15)
(0.67, 1.23)
CHILD OUTCOMES: CHILD’S PROBLEM BEHAVIOR
CBCL: % More Aggressive (> 14)
Randomization
„ 1.48
† 0.96
Quasi-Experimental
Pooled
(1.05, 2.09)
(0.66, 1.39)
(1.06, 1.98)
(0.95, 1.88)
¡ 1.45
„ 1.44
† 1.14
(1.15, 1.81)
(0.89, 1.46)
CBCL: % More Problems Sleeping (> 6)
Randomization
† 1.43
Quasi-Experimental
Pooled
† 1.34
(0.99, 2.06)
† 1.3
† -0.31
‹ 2.11
† 0.45
† 1.76
† 0.45
(0.9, 1.89)
† 1.24
† 1.18
(0.9, 1.71)
(0.84, 1.67)
„ 1.3
† 1.23
(1.03, 1.65)
(0.96, 1.57)
12-14
† 0.03
† 0.27
† 0.44
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Difference Between
Groups (t statistic)
CHILD OUTCOMES: INJURIES AND HEALTH CARE UTILIZATION
Injured Badly Enough to See a Doctor
Randomization
† 1.13
Quasi-Experimental
Pooled
(0.79, 1.62)
† 0.97
† 0.93
† 0.99
(0.67, 1.27)
(0.71, 1.38)
†1
† 0.98
(0.79, 1.26)
(0.77, 1.25)
Number of Emergency Room Visits in Past Year (1 or more )
Randomization
† 1.19
† 1.23
Quasi-Experimental
Pooled
(0.9, 1.57)
† 0.75
(0.67, 1.39)
† -0.34
† 0.14
† -0.21
(0.93, 1.62)
† 0.84
† 1.02
(0.67, 1.06)
(0.8, 1.3)
† 0.97
† 1.13
(0.81, 1.15)
(0.94, 1.35)
† -1.37
† -1.5
One or More Emergency Room Visit for Injury-Related Causes in Past Year
Randomization
† 0.96
† 0.91
† 0.2
Quasi-Experimental
Pooled
(0.62, 1.49)
(0.59, 1.41)
(0.46, 0.95)
(0.46, 0.99)
¡ 0.66
¡ 0.67
† 0.78
† 0.79
(0.59, 1.03)
(0.6, 1.05)
† -0.06
† -0.06
Number of Hospitalizations (1 0r More Times in Past Year @ 30-33 Months)
Randomization
† 1.5
† 1.11
† 1.04
Quasi-Experimental
Pooled
(0.91, 2.46)
(0.66, 1.87)
† 0.81
† 1.25
(0.48, 1.38)
(0.74, 2.11)
† 1.08
† 1.1
(0.76, 1.54)
(0.76, 1.57)
† -1.42
† -0.07
CHILD OUTCOMES: AGE-APPROPRIATE WELL CHILD CARE
1 Month Well Child Visit (Visit within 41 days of birth)
Randomization
† 1.88
† 1.5
Quasi-Experimental
Pooled
(0.97, 3.66)
† 0.57
(0.82, 2.73)
„ 3.13
„ 2.62
(1.6, 6.13)
(1.37, 5.02)
„ 2.42
„ 1.89
(1.51, 3.86)
(1.23, 2.92)
† 0.41
† 0.85
2 Month Well Child Visit (Visit between 42 days (1.5 months) and 92 days (3 months),
inclusive)
Randomization
Quasi-Experimental
Pooled
„ 2.74
„ 1.9
(1.82, 4.14)
(1.32, 2.73)
„ 2.03
„ 1.86
(1.5, 2.76)
(1.37, 2.53)
„ 2.22
„ 1.79
(1.74, 2.83)
(1.42, 2.26)
(1.04, 2)
(1.12, 2.15)
† 1.5
† 0.48
† 1.47
4 Month Well Child Visit (Visit between 93 days (3 months) and 151 days (5 months),
inclusive)
Randomization
„ 1.44
„ 1.56
† -0.39
Quasi-Experimental
Pooled
„ 1.72
„ 1.58
(1.3, 2.26)
(1.19, 2.1)
„ 1.59
„ 1.5
(1.29, 1.96)
(1.22, 1.86)
12-15
† 0.48
† 0.42
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
Difference Between
Groups (t statistic)
6 Month Well Child Visit (Visit between 152 days (5 months) and 213 days (7 months),
inclusive)
Randomization
Quasi-Experimental
Pooled
„ 1.88
„ 1.47
(1.36, 2.61)
(1.08, 2.01)
„ 1.41
„ 1.44
(1.09, 1.81)
(1.11, 1.87)
„ 1.54
„ 1.44
(1.27, 1.88)
(1.18, 1.75)
† 1.28
† -0.17
† 0.61
9 Month Well Child Visit (Visit between 244 days (8 months) and 305 days (10 months),
inclusive)
Randomization
Quasi-Experimental
Pooled
† 1.23
† 1.1
(0.9, 1.68)
(0.81, 1.48)
„ 2.15
„ 1.87
(1.73, 2.67)
(1.5, 2.33)
„ 1.78
„ 1.55
(1.49, 2.12)
(1.3, 1.85)
† 0.65
† 1.08
† 1.31
12 Month Well Child Visit (Visit between 336 days (11 months) and 397 days (14
months), inclusive)
Randomization
Quasi-Experimental
Pooled
„ 1.85
„ 1.68
(1.27, 2.69)
(1.17, 2.4)
„ 1.85
„ 1.85
(1.4, 2.46)
(1.37, 2.49)
„ 1.82
„ 1.76
(1.46, 2.28)
(1.4, 2.21)
† 0.43
† 0.01
† 0.26
15 Month Well Child Visit (Visit between 427 days (14 months) and 488 days (17
months), inclusive)
Randomization
Quasi-Experimental
Pooled
„ 1.61
† 1.17
(1.18, 2.19)
(0.87, 1.57)
„ 2.22
„ 2.14
(1.75, 2.82)
(1.67, 2.74)
„ 1.94
„ 1.65
(1.61, 2.34)
(1.37, 1.99)
† 1.8
† 0.27
† 1.49
18 Month Well Child (Visit between 519 days (17 months) and 580 days (20 months),
inclusive)
Randomization
Quasi-Experimental
Pooled
„ 2.17
„ 2.01
(1.61, 2.93)
(1.5, 2.71)
„ 3.35
„ 3.42
(2.61, 4.32)
(2.63, 4.43)
„ 2.73
„ 2.74
(2.26, 3.31)
(2.26, 3.33)
† 0.44
† -0.12
† -0.02
24 Month Well Child Visit (Visit between 701 days (23 months) and 762 days (28
months), inclusive)
Randomization
Quasi-Experimental
Pooled
„ 1.91
„ 2.84
(1.34, 2.71)
(1.94, 4.15)
† 1.16
„ 1.68
(0.86, 1.55)
(1.22, 2.31)
„ 1.41
„ 2.09
(1.13, 1.76)
(1.65, 2.66)
12-16
† -1.78
‹ -2.16
‹ -2.91
Chapter 12
Healthy Steps: The First Three Years
MORE
EXPERIENCE
More than 100
children enrolled at
time of entry
LESS EXPERIENCE
Less than or equal to
100 children enrolled
at time of entry
CHILD OUTCOMES: AGE-APPROPRIATE VACCINATIONS
DTP 1
Randomization
„ 2.15
„ 1.89
Quasi-Experimental
Pooled
(1.37, 3.38)
Difference Between
Groups (t statistic)
† 0.48
(1.24, 2.87)
„ 1.91
„ 1.59
(1.34, 2.73)
(1.12, 2.26)
„ 1.95
„ 1.66
(1.48, 2.58)
(1.27, 2.16)
† 0.83
† 0.98
DTP 3
Randomization
Quasi-Experimental
Pooled
„ 1.55
„ 1.42
(1.15, 2.09)
(1.06, 1.91)
„ 1.67
„ 1.39
(1.33, 2.11)
(1.1, 1.75)
„ 1.6
„ 1.38
(1.34, 1.92)
(1.15, 1.66)
† 0.49
† 1.37
† 1.39
MMR 1
Randomization
Quasi-Experimental
Pooled
† 1.37
„ 1.52
(0.93, 2.02)
(1.04, 2.23)
„ 1.66
„ 1.54
(1.21, 2.27)
(1.12, 2.13)
„ 1.54
„ 1.52
(1.21, 1.96)
(1.19, 1.94)
CHILD OUTCOMES: CHILD UP-TO-DATE ON VACCINATIONS
Up-to-Date at 24 Months (4 DTP, 3 OPV/IPV, 1 MMR)
Randomization
„ 1.88
† 1.26
Quasi-Experimental
Pooled
(1.31, 2.69)
(0.91, 1.76)
„ 1.84
† 1.1
(1.29, 2.64)
(0.79, 1.53)
„ 1.9
† 1.22
(1.48, 2.44)
(0.97, 1.53)
12-17
† -0.44
† 0.38
† 0.08
‹ 2.00
‹ 2.59
‹ 3.22