The Affiliate Evaluation
Final Report
Prepared by:
The Women’s & Children’s Health Policy Center
Johns Hopkins University
Bloomberg School of Public Health
May 2003
The Affiliate Evaluation
Final Report
 Johns Hopkins Bloomberg School of Public Health
Authors:
Tess Miller, Bernard Guyer, David Bishai, Diane Burkom, Becky
Clark, Allison Cosslett, Janice Genevro, Holly Grason,William Hou,
Nancy Hughart, Alison Snow Jones, Cynthia Minkovitz, Heather
Rutz, Daniel Scharfstein, Heather Stacy, Donna Strobino, Eleanor
Szanton, and Chao Tang.
Editor:
Janice Genevro
Contact:
Kamila Mistry
Project Director
Bloomberg School of Public Health
Johns Hopkins University
624 N. Broadway, Room 195
Baltimore, MD 21205
410-955-8694
[email protected]
Healthy Steps for Young Children is a program of The Commonwealth Fund
in collaboration with Boston University School of Medicine,
and local funders and health care providers across the nation.
Healthy Steps is co-sponsored by the American Academy of Pediatrics.
The Healthy Steps affiliate evaluation was carried out with grants from
The Atlantic Philanthropies, The Robert Wood Johnson Foundation and local funders.
The views presented here are those of the authors.
Table of Contents
Acknowledgments..................................................................................................................i
Executive Summary ...............................................................................................................ii
Introduction............................................................................................................................1
What is Healthy Steps? ..........................................................................................................1
How was Healthy Steps Evaluated? ......................................................................................2
Who Participated in the Affiliate Evaluation?.......................................................................4
Pediatric Practices......................................................................................................4
Families......................................................................................................................5
How was Healthy Steps Implemented at Affiliate Sites? ......................................................7
Integrating the HS Specialist into the Practice ..........................................................7
Implementing the Healthy Steps Services .................................................................8
What Services did HS Specialists Provide to Affiliate Families? .........................................12
What Services did Affiliate Families Report Receiving? ......................................................15
In What Ways Did Healthy Steps Add Value to Standard Pediatric Care at Affiliate Sites?
................................................................................................................................................17
Enhanced Relationships .............................................................................................17
Increased Parent Satisfaction with Care ....................................................................18
Increased Provider Satisfaction .................................................................................19
Improved Teamwork and Practice Environment .......................................................19
Positive Parenting Practices and Improved Child Outcomes ....................................20
What Did We Learn from the Affiliate Evaluation?..............................................................26
What is happening at Affiliate Sites Now?............................................................................28
Appendix I – Methodology....................................................................................................30
Appendix II – Selected Outcomes .........................................................................................47
Acknowledgments
We wish to thank Margaret E. Mahoney, Karen Davis, Kathryn Taaffe McLearn, Edward
Schor, and members of the Healthy Steps National Advisory Committee for their guidance,
advice and support. We also are grateful to Michael Barth at ICF Consulting for his tireless
leadership and to his staff, Juliet Konvisser, Philip Rizzi, Cynthia Hansel, Nita Hassan,
Samantha Gill and Sara Rogers. The affiliate evaluation has greatly benefited from the close
collaboration of Barry Zuckerman, Margot Kaplan-Sanoff, Steven Parker, Andrea Bernard
and Tracy Magee at the Boston University School of Medicine.
Funding for the Affiliate evaluation was made possible through the generous support of The
Atlantic Philanthropies, The Robert Wood Johnson Foundation, and the following local
funders: Brown Foundation; The Chicago Community Trust; Children’s Trust Fund of
Texas; The Duke Endowment; Harris Foundation; Hogg Foundation for Mental Health; The
Houston Endowment; John D. and Catherine T. MacArthur Foundation; Kansas Health
Foundation; Michael Reese Health Trust; Prince Charitable Trusts; Rockwell Fund, Inc.; San
Antonio Metropolitan Health Department; Texas Children’s Hospital; W.P. and H.B. White
Foundation; Washington Square Health Foundation. The views presented here are those of
the authors and not necessarily those of the financial supporters, or their directors, officers or
staff.
The following members of the evaluation team at Johns Hopkins led by Bernard Guyer
assisted in the affiliate evaluation: Mary Benedict; Allison Cosslett; Brandy Fauntleroy;
Janice Genevro; Holly Grason; Nancy Hughart; William Hou; Ashraful Huq; Avanti
Johnson, Alison Snow Jones; Armenta Jones; Pat Lanocha; Tess Miller; Cynthia Minkovitz;
Lexie Motyl; Stephanie Neal, Becky Newcomer; Laura Pagels, Heather Rutz; Daniel
Scharfstein; Jane Schlegel; Heather Stacy; Brenda Sterling; Lavonne Sumler; Kristie Susco;
Donna Strobino; Eleanor Szanton; Chao Tang; and Marsha Young. In addition, Diane
Burkom, Helen Gordon and the interviewing staff of Battelle Centers for Public Health
Research and Evaluation conducted the parent telephone interview. Jean Su and her staff at
SOSIO, Incorporated provided data entry for the evaluation. Thank you all for your hard
work and dedication.
We are grateful to the Healthy Steps Specialists, lead physicians, administrators, and staff at
each of the sites for their devotion to the program, their commitment to the evaluation, and
their hard work to ensure that both succeeded. Their names are listed below.
Lead Physicians, Administrators and Others: Anita Berry; Mary Martha Bledsoe Felkner;
Julia Bowers-McLain; Mohammad Chaudhary; Glenna Dawson; Jan Drutz; Fernando
Guerra; Cynthia Henderson; Molly Jacob; Crystal Mobley; Jane Moss; Jerry Neiderman;
Karen Nonhof; Isabelle Patton; Consuelo Sandoval; Penny Schwab; Silvana Shilapochnik;
Daniel Treviño; Juan Vargas; Darlene Victorson; and Gail Wilson.
Healthy Steps Specialists: Catalina Ariza; JoAnn Allen; Melanie August; Juanita Brown;
Jennifer Dubrow; Rosa Fernandez; Patricia Garza; Kris Hawkins; Elaine Nishioka; Janie
Ochoa; Esther Oppliger; Sabrina Provine; Veronica Serano; and Claudia Yañez.
Finally, we wish to thank all the families who generously gave of their time. Without them,
our work would not have been possible.
Executive Summary
This report focuses on the evaluation of the Healthy Steps for Young Children program at six
affiliate sites1. Affiliate sites were selected based on the same criteria as national evaluation
sites except they did not have a comparison population. They implemented the Healthy
Steps program fully, offering the same program as the national sites. Six sites, comprising
seven primary pediatric practices, participated in the affiliate evaluation.
On average, mothers participating in the affiliate evaluation tended to be young (more than
one half were less than 25 years old), with limited education (45% had not graduated from
high school), of Hispanic origin (56% reported they were of Hispanic origin) and poor (for
54% of the families, maternity care was paid for by Medicaid). They differed from the
families in the national evaluation in terms of these demographic characteristics.
The full Healthy Steps program was implemented at all affiliate sites. All affiliate sites hired
two HS Specialists and delivered the package of Healthy Steps services from the time the
first family was enrolled into the program. There were some barriers encountered in
implementation as well as variability in the programs.
Results from both providers and parents indicated that affiliate families received Healthy
Steps services in addition to routine pediatric primary care. According to the Healthy Steps
Specialists, the average family who participated in the program at least 15 months received a
variety of services from their Specialist:
7 office visits
2 home visits
6 telephone calls
2 other contacts such as mailings.
The average family did not attend a parent group. Only 20% of families attended at least one
parent group during the program.
Child development was universally discussed with families, and other important topic areas
such as nutrition, child health, injury prevention, family support and maternal health were
addressed with a large proportion of families. Families with higher incomes and older, better
educated and first-time mothers appeared to receive more Healthy Steps services than their
counterparts, but the differences between groups were small.
These results are particularly noteworthy because five of the six affiliate sites served lowincome, transient populations at high risk for poor outcomes. Initially, there was doubt
whether Healthy Steps could even be implemented at these sites. Not only did these sites
1
Three other affiliate sites participated in Healthy Steps. Two sites implemented variations of the Healthy
Steps program and evaluated their programs: one site included a prenatal component; another offered
telephone counseling in lieu of enhanced well child visits. Each utilized a randomized, case/control design for
their evaluation. A third site participated in a local evaluation.
successfully implement the program, but they also delivered developmental services to
populations that are traditionally difficult to reach and to engage in health care programs.
Healthy Steps added value to the primary pediatric health care delivered at affiliate sites. It
enhanced the relationship between the family and the practice. The key to the program was
the relationship that developed between the family and the HS Specialist. This relationship,
and the additional services provided, seems to account for increased parents’ satisfaction
with the care they received. According to all the evaluation data, affiliate families—whether
at high or low risk, new parents or more experienced, young or older mothers—were highly
satisfied with the program.
In addition, based on surveys conducted with health care providers at start-up and 30 months
into the program, Healthy Steps appeared to have improved the satisfaction of pediatricians
and nurse practitioners with the care they provided. All those in the practice who worked
with the HS Specialist acknowledged the benefits that this new professional brought to the
practice. Overall teamwork improved over the course of the program. Increased satisfaction
occurred among health care professionals that work with at-risk populations. Again, this is
particularly noteworthy as these providers are traditionally at higher risk of job
dissatisfaction and turnover.
The level of positive parenting practices was high among affiliate families. The majority of
families reported using safety devices, establishing routines and talking to and playing with
their child. There was some evidence to suggest that Healthy Steps improved parents’ use
of the health care system for their children. A greater percentage of HS affiliate children
than children who received care at the practice prior to HS were given a Denver
Developmental Screening Tool (DDST) by 12 months and made age-appropriate well child
visits. At several sites, more affiliate children were fully immunized at 12 months of age than
were children who received care at the practice prior to HS.
Results from the affiliate evaluation reinforced the results of the national evaluation. Healthy
Steps was well implemented. The key to the program seemed to be the relationship that
developed between the HS Specialist and families, which in turn strengthened the
relationship of the family with their primary care provider and ultimately the practice overall.
Healthy Steps improved clinicians’ and families’ satisfaction with pediatric care. The
program increased the amount of preventive health care children received. Most important,
the invaluable contribution of the affiliation evaluation is that it demonstrated that Healthy
Steps could be successfully implemented with a low income, high risk population as well as
in a high income population.