Chapter 13
Healthy Steps: The First Three Years
13. The Evaluation at Affiliate Sites
The Evaluation at Affiliate Sites
In this chapter, the results of the affiliate evaluation are summarized. Six sites, comprising seven primary care
practices, participated in the affiliate evaluation. These sites met the same requirements as the sites selected for the
national evaluation except they did not have a comparison group.
The affiliate sites fully implemented the same program as the sites in the national evaluation. However, families
participating in the affiliate evaluation were somewhat different from families participating in the national evaluation.
They tended to be younger, less well educated, poorer, and more diverse racially and ethnically.
Results from the affiliate evaluation reinforce the results of the national evaluation. Healthy Steps was well
implemented. The key to the program was the relationship that developed between the Healthy Steps Specialist and
families, which in turn strengthened the relationship of the family with their primary care provider and ultimately the
practice. Healthy Steps improved clinicians’ and families’ satisfaction with pediatric care. It increased the amount
of preventive health care children received. The unique contribution of the affiliation evaluation is that it
demonstrated that HS can be successfully implemented with a low income, high risk population.
13-1
Chapter 13
Healthy Steps: The First Three Years
13. THE EVALUATION AT AFFILIATE SITES
13.1. Introduction
In this chapter, the results of the affiliate evaluation are
summarized. Six sites, comprising seven primary care
practices, participated in the affiliate evaluation. These
sites met the same requirements as the sites selected for
the national evaluation except they did not have a comparison
group. The sites fully implemented the same program as
the sites in the national evaluation.
Families participating in the affiliate evaluation were
somewhat different from families participating in the
national evaluation. Compared to families at the national
sites, families at the affiliate sites tended to be:
younger -- 24% were teenagers at the time of their
child’s birth compared to 14%;
less well educated -- 46% had not graduated from
high school compared to 18%;
of Hispanic origin -- 50% described their race as
White and 56% reported they were of Hispanic
origin, compared to 58% and 20%, respectively; and,
poorer -- for 54%, maternity care was paid for by
Medicaid, compared to 32%.
Families and clinicians participating in the affiliate
evaluation completed many of the same evaluation
instruments used in the national evaluation. These
included key informant interviews at baseline and 30
months; provider surveys at baseline and 30 months;
Healthy Steps (HS) Specialists’ logs of contacts; a newborn
form at enrollment; and parent questionnaires at 6, 12, 18
and 24 months. Unlike families in the national evaluation,
affiliate families did not participate in the telephone
interviews at 2-4 months or 30-33 months. Instead, they
participated in a telephone interview when their child was
18 months old. In addition, although a review of the
child’s medical record was conducted, data were abstracted
through only the first year of the child’s life.
In general, the results from the affiliate evaluation tell a
story that is very similar to the results of the national
13-2
Chapter 13
Healthy Steps: The First Three Years
evaluation.
All affiliate sites implemented the HS
program. This included not only implementing the seven
components of the program, but also integrating the HS
Specialist into the practice and establishing a team
approach to the delivery of pediatric care.
13.2. Implementing Healthy Steps
As was the case with sites in the national evaluation:
Lead pediatricians at affiliate sites ranked the role of
the HS Specialist as the most valuable part of the
program. The HS components ranked most valuable
by the majority of lead pediatricians and HS
Specialists included linked/joint well child visits and
enhanced pediatric strategies.
Key informants reported the overall practice
environment generally improved from start-up to 30
months into the program. Informants at affiliate
sites had a slightly more favorable impression of the
practice environment than did key informants at
national sites. At 30 months, all site administrators
and lead pediatricians who were interviewed at
affiliate sites rated the practice environment as good
or very good, compared to 73% and 87% (respectively)
at national sites. At both national and affiliate sites,
the HS Specialists interviewed rated the practice
environment less favorably than other respondents.
However, the HS Specialists at affiliate sites were
slightly more positive than those at national sites: 30
months into the program, 37% of HS Specialists rated
the practice environment as okay, poor or very poor,
compared to 42% at national sites.
Key informants and other providers at affiliate sites
reported that team work improved over the course of
the program—perhaps less so than at the national
sites. In a survey of providers at start-up, 38% of
clinicians (physicians and nurse practitioners) at
affiliate sites said they rarely or never worked as a
team during well child visits; 30 months into the
program, only 14% felt that way. However, only
38% said they always or often worked as a team,
compared to 65% of clinicians at national sites.
13-3
Chapter 13
Healthy Steps: The First Three Years
Interestingly, HS Specialists’ reports of their overall
relationships with other clinicians and administrative
staff at the practice were, in general, more favorable
at affiliate sites. HS Specialists reported the least
favorable relationship with the lead pediatrician; 67%
rated their overall relationship with the lead
pediatrician as good or very good. At national sites,
HS Specialists reported the least positive relationship
with the site administrator. Only 39% rated this
relationship as good or very good.
Affiliate sites had the most difficulty implementing
the parent groups. No sites had weekly parent
groups; few sites had monthly parent groups. Two
sites discontinued the parent groups.
13.3. Clinicians and Practice Staff
As was the case with clinical and administrative staff in
the national evaluation:
All those in the practice who worked with the HS
Specialists acknowledged the benefits that this new
professional brought to the practice. In general,
however, they were less favorable than clinicians and
staff at national sites. Their appreciation of the HS
Specialists’ role increased over time. Nurses and
other clinical staff generally had a less favorable view
of the benefits of the HS Specialist and program than
clinicians.
13.4. Affiliate Families
As was the case with families in the national evaluation:
The vast majority of affiliate families received HS
services. According to HS Specialists’ reports:
o 99% of affiliate families received at least one
office visit; 84% had at least one phone
contact; 81% had at least one home visit;
and 20% attended at least one parent group
during the first 32 months of life.
o During the first year, the average family
received 5.4 office visits, 3 telephone
contacts and 1.4 home visits. The average
13-4
Chapter 13
Healthy Steps: The First Three Years
affiliate family did not attend a parent
group during the first year.
o Sites varied considerably in the type and
number of contacts made with families.
o Child development was almost universally
discussed with affiliate families. Other
important topics such as child nutrition and
health, injury prevention, family and
maternal health, and support were
addressed with a large proportion of
families.
Affiliate parents reported receiving HS services at
high levels similar to those reported by national
evaluation families. At 18 months, the majority of
families who were interviewed reported receiving
enhanced well child visits (99%), home visits (91%),
and telephone contacts with the HS Specialist
(67%). Only 39% of interviewed parents said they
attended a parent group. They also reported
receiving information on: home safety (94%); child
development (91%); car seats (86%); routines
(86%); and discipline (81%).
Fewer parents
reported receiving information on: sleep problems
(74%); language development (74%); child
independence (61%); sibling rivalry (42%); and
toilet training (41%).
The majority of affiliate families found all HS services
to be very helpful or helpful. The most helpful service
was the enhanced office visits: 71% found it to be very
helpful and 25% helpful. Nearly all families who
received information on child development found it
to be very helpful or helpful (less than 1% reported it
was not useful at all). The least useful information
parents received had to do with sibling rivalry: 8% of
families who received information on this topic said it
was not useful at all (41% of affiliate families were
first-time parents).
Affiliate families were extremely satisfied with their
HS Specialists -- 84% found the HS Specialist to be
very helpful and 72% said the HS Specialist was the
person at the practice who went out of their way to help
them.
13-5
Chapter 13
Healthy Steps: The First Three Years
Affiliate families appeared to be highly satisfied with
the care they received as part of HS. Nearly all
families (97%) said they would recommend their
pediatric provider to a friend. Nearly half of
interviewed families (48%) said they would spend
$100 or more to continue to receive HS services for a
year.
Levels of recommended parenting practices were
high among affiliate parents. The majority of families
reported using safety devices; establishing routines
regarding mealtime, naptime, and bedtime; and
talking and playing with their child. There was
limited evidence to suggest that parenting practices
changed during the program. Over the course of the
program, the probability that an affiliate mother or
father read or showed a book to their child every day
increased. However, at 18 months, only 60% of
mothers said they read to their child at least once a
day and reported that even fewer fathers (38%) did
so.
As was the case with children in the national evaluation:
Affiliate children received age-appropriate well child
care. A greater percentage of affiliate children than
children who received care at the practice prior to HS
received a Denver Developmental Screening Tool
(DDST) by 12 months and made age-appropriate
well child visits. At several sites, more affiliate
children had immunizations that were up-to-date at
12 months than did children who received care at the
practice prior to HS.
Results from the affiliate evaluation reinforce the results of
the national evaluation.
Healthy Steps was well
implemented.
The key to the program was the
relationship that developed between the HS Specialist and
families, which in turn strengthened the relationship of the
family with their primary care provider and ultimately the
practice. Healthy Steps improved clinicians’ and families’
satisfaction with pediatric care. It increased the amount
of preventive health care children received. The unique
contribution of the affiliation evaluation is that it
demonstrated that HS can be successfully implemented
with a low income, high risk population.
13-6