Developmental Disabilities Program
Independent Evaluation (DDPIE)
Project
Technical Assistance Institute
UCEDDs
June 12, 2008
Westat
Lynn Elinson, Ph.D.
Developmental Disabilities Program
Independent Evaluation (DDPIE) Project
Also known as “ADD Independent Evaluation”
Purpose of Presentation




To provide update on progress since last
national UCEDD technical assistance
meeting (May, 2007)
To present next steps
To provide feedback on pilot study
To answer questions, obtain comments
Presentation Outline
1.
2.
3.
4.
5.
6.
Background
Update on progress
Further Validation
Timing
Feedback on pilot study
Comments/questions
1.
Background
Purpose of the DDPIE Project




Demonstrate impact of DD Network programs on:
– Individuals
– Families
– Service providers
– State systems
Provide feedback to ADD to help improve the
effectiveness of its programs and policies
Promote positive achievements of DD Network programs
by “storytelling”
Promote accountability to the public
Purpose of Phase I



Develop tools for independent evaluation
Test the tools
Report and make recommendations to ADD
Key Functions
Content of Measurement Matrices
Benchmarks
Indicators
Performance Standards
Reporting at National Level



Data will be collected on individual programs
and rolled up to national level.
Independent evaluation will NOT be
comparing programs to one another
Independent evaluation will NOT replace
MTARS, which is specific to individual
programs.
2. Update on Progress
Issues Raised by All Programs



Transparency
Efficiency - use of existing data
Simplicity
Transparency





Working Groups
All programs
ERoom
Telephone calls
Written comments
National TA meetings
Advisory Panel
ADD website
Validation Meeting
Efficiency

Testing use of existing data (5-year plan,
annual report, MTARS background
information) in pilot study
Simplicity


One of the purposes of the pilot study was to
inform the revision or elimination of
benchmarks/indicators
Advisory Panel recommended not reducing
number of benchmarks/indicators until after
the pilot study.
3.
Further Validation
Opportunities for validation






Working Group process*
Advisory Panel meetings*
Opportunities for feedback from DD Network
programs (at national meetings, by telephone, ERoom, in writing)*
Pilot study*
Validation Meeting*
OMB process* – new contract
*DD Network programs represented
Purpose of Validation Meeting



To review benchmarks, indicators, and
performance standards
To rate the indicators on importance or value
to ADD independent evaluation goals
To make recommendations on process for
further development of performance
standards
Validation Meeting Representation







Self-advocates
Family members
DD Council members
Members of Boards of Directors (P&A)
DD Network program staff
Policy analysts
Researchers
4. Timing
DDPIE Project Milestones
Advisory Panel meeting
March 18, 2008
P&A national meeting
March 31, 2008
Advisory Panel Meeting
May 20-21, 2008
DD Council national meeting
June 9, 2008
UCEDD National meeting
June 12, 2008
Validation Meeting
July 9-11, 2008
Further analysis and synthesis
July – September, 2008
Final report to ADD
September 29, 2008
5. Feedback on the Pilot Study
Purpose of Pilot Study





To inform the revision or elimination of some
benchmarks/indicators
To test data collection instruments
To inform the further development of performance
standards
To determine the usefulness of existing data in
reports to ADD
To test the logistics for a full-scale independent
evaluation
Programs Participating in Pilot Study
P&A
UCEDD
New Hampshire New Hampshire
Ohio
Iowa
Alaska
California – UCLA
DD Council
New Hampshire
Ohio
Wyoming
New Mexico
Criteria for DD Council Inclusion in
Pilot Study





Geographic diversity (e.g., rural/urban);
Geographic region of the United States
(west, midwest, and east);
Participation in MTARS in 2005 – 2007 (at
least 1)
Organizational structure (is and is not in a
medical school); and
Is and is not a LEND program.
Pilot Study Activities







Development of data collection instruments
Training
Review of program materials (policies, procedures,
forms, reports)
Program visits
In-person and telephone interviews
Review and analysis of reports to ADD – 5-year
plan, annual report, MTARS materials (for some)
Transcriptions, write-ups, analysis, synthesis
Interviews
P&A





Exec. Dir.
and staff
Recipients of
public education
Board of Directors/
Commission
P&A clients
Group (systemic
advocacy issue)
DD Council





Exec. Dir. and
staff
DD Council
members
Recipients of
comm. capacity
development activities
Recipients of
advocacy and leadership
training
Group (system
design issue)
UCEDD





Exec. Dir. and staff
Consumer Advisory
Comm. members
Students
Recipients of comm.
services
Colleagues/
administrators
Pilot Study Experiences




Organizing program visit
Developing agenda with Executive Director
Logistics
Some observations to date
Pilot Study Participants – Feedback
from Executive Directors
6.
Comments and questions
Contacts:

Lynn Elinson, 412 421-8610,
[email protected]

Cynthia Thomas, 1-800-937-8281, ext. 4364,
[email protected]