2014 Program Evaluation /
Outcomes Management Report
A summary of our program goals, results and conclusions
Page 1 of 20
CONTENTS
Midwest Special Services, Inc. is committed to ongoing quality review and service improvement. The
information in this report reflects just a few of the ways in which we measure our performance,
identify trends, and gather feedback from our clients and other stakeholders of the organization.
Client Satisfaction
pg 3 – 4
Family & Team Member Satisfaction
pg 5 – 6
Employer Satisfaction
pg 7 – 8
Demographics
pg 9 – 12
Program Outcome Measures – Vocational Services
pg 13 – 14
Program Outcome Measures – DTH Programs
pg 15 – 16
Service Access Improvements
pg 17 – 18
Conclusions
pg 19 – 20
Page 2 of 20
CLIENT SATISFACTION
337 clients completed the survey, representing approximately 38% of the total clients served (667) in all
MSS programs. Survey items included both ratable statements and open-ended questions. Responses
to ratable items were 95% positive (“Agreed” or “Strongly Agreed” to the statements below).
Ratable Survey Items
1. The staff at MSS treat me with respect and dignity.
Positive Ratings
96.70%
2. Staff communicate important issues with me in a timely manner.
93.4%
3. I am making progress on my goals.
92.2%
4. MSS staff are creative and resourceful.
98.4%
5. My life has improved as a result of the services received at MSS.
95.00%
6. Overall, I am satisfied with the services I’ve received at MSS.
95.00%
7. MSS takes a person-centered approach in planning/implementing my
services.
8. MSS takes my culture (language, ethnicity/race, religion, sexual orientation,
sex/gender roles, socioeconomic status, and age) into consideration when
planning/implementing, my services.
94.30%
97.2%
Overall Satisfaction
Strongly Disagree
2%
Client Satisfaction
Agree
Strongly Agree
34%
Disagree
Strongly Agree
Agree
61%
Disagree
3%
Page 3 of 20
Strongly Disagree
CLIENT SURVEY COMMENTS/SUGGESTIONS
Misc.
 I like the fact that we talk about our problems if we have them. For example, I said at a
meeting that the sidewalks were too narrow and the doors didn't stay open long enough
for me to wheel my electric wheelchair through them. Because of these comments,
MSS widened the sidewalk ramp and extended how long the doors stay open.
 Creating art makes me feel beautiful. I love being able to sell my art.
 I love having lunch and doing yoga with our volunteers.
What does MSS do best?
 They give people an opportunity to work and allow people to be productive members of
society regardless of disability. They give people the dignity and respect that they
deserve.
 They give me choices on things, like having lunch where I want to.
 I get a great paycheck and I’m happy with the mobile work crew participation.
 If you can’t figure out a job, they teach you how to do it before you start.
 One good thing about MSS is that the staff and the schedules are very flexible and I get
to choose the type of job I want.
 The staff here are very friendly.
If you could change anything about MSS, what would it be?
 Find more jobs for people who cannot use their hands well.
 I would like MSS to increase my wages.
 If we had more staff, people would have more time to do things (outings, work, etc.)
 I would like to start later in the day.
 More work. (This was a common response.)
 Do art every day.
Page 4 of 20
SUPPORT TEAM SATISFACTION
210 Support Team members (family, referral sources, residential providers, guardians) completed the
survey. Survey items included both ratable statements and open-ended questions. Responses to
ratable items were 97% positive (“Agreed” or “Strongly Agreed” to the statements below).
Ratable Survey Items
1. The staff at MSS treat [client] with respect and dignity.
Positive Ratings
99.6%
2. Staff communicates important issues with me in a timely manner.
95.6%
3. My client/family member is making progress on his/her goals..
95.00%
4. MSS staff are creative and resourceful.
97.5%
5. [Client’s life] has improved as a result of the services at MSS.
96.2%
6. Overall, I am satisfied with the services [client] has received at MSS.
98.1%
7. MSS takes the culture of my client/family member (language, ethnicity/race,
religion, sexual orientation, sex/gender roles, socioeconomic status, and age)
into consideration when planning/implementing services.
8. MSS takes a person-centered approach in planning/implementing services.
99.00%
96.5%
Overall Satisfaction
Support Team Member Satisfaction
Disagree
2%
Disgree
Agree
41%
Strongly Agree
57%
Agree
Strongly Disagree
Strongly Agree
Strongly Disagree
0%
Page 5 of 20
SUPPORT TEAM COMMENTS/SUGGESTIONS
Misc.
 I am pleased and surprised by my client’s progress in just three years. He really likes the
beekeeping and is especially looking forward to using and learning the iPad. Overall we
couldn't be more pleased with MSS.
 MSS staff and case managers really listen and accept input and suggestions from other
team members in order to determine how best to meet a specific individual's needs.
What does MSS do best?
 Staff at MSS are compassionate, and willing to go above and beyond to assist my client.
 Staff are very supportive, helpful, always ready to advocate for client when needed, and
just plain awesome. Thank you with all my heart.
 MSS is very respectful, kind, and caring towards their clients. I also look to MSS for
support and info that the group home is doing a good job because my daughter cannot
advocate for herself.
 MSS keeps coming up with creative ideas to make it a fun place to work. One of the
people I know who attends this program waits at the door, watching for the bus each
morning. MSS is flexible, giving the people that they serve room to be themselves.
 I enjoy the artwork that MSS does with their clients. I think using the sale of artwork as
a way for clients that may be unable to work typical jobs to make money is great.
 I believe that MSS is very client centered.
If you could change anything about MSS, what would it be?

I wish there was more paid community employment

I wish some of the centers did not have a waiting list so they would be able to serve all
of my clients.

I wish MSS was open 7 days a week!

I wish MSS had the capacity to be part of everyone's family that has a member with
special needs.

I would have MSS have longer business hours.
Page 6 of 20
EMPLOYER SATISFACTION
14 employers/contract customers completed the survey. Responses to ratable items were 98% positive
(“strongly agreed” or “agreed”) with the following quality-indicating statements:
Answer Options
The quantity delivered was correct
Strongly Agree
4
Agree
5
Disagree
0
Strongly Disagree
0
My order was received on time
3
6
0
0
Work performed by MSS was done
as specified
Quality of work done by MSS met my
expectations
MSS staff were courteous and
pleasant
Response to inquiries was timely and
efficient
The MSS website was informative
and helpful
MSS addressed concerns effectively
4
5
0
0
5
7
0
0
6
3
0
0
4
5
0
0
1
8
0
0
5
7
0
0
MSS responded to questions and
concerns in a timely manner
Prices offered by MSS were
competitive and reasonable
Efforts were made to minimize
overall project cost
Production and communication
methods utilized by MSS were
appropriate for my industry
MSS offers a broad range of services
and capabilities that I can use in my
business
The initial meeting and tour of MSS
was helpful
We look forward to working with
MSS in the future
3
6
0
0
7
5
0
0
4
5
1
0
4
5
1
0
6
6
0
0
3
6
0
0
4
5
0
0
In addition, 100% of respondents reported that they would refer Midwest Special Services to other
businesses.
Page 7 of 20
MISC SURVEY COMMENTS/SUGGESTIONS
As always, we received incredible service from MSS. We placed our order last minute and they were
able to exceed our expectations without any complications. The entire staff we have worked with are
wonderful – especially the loading/shipping crew Dan Rooney. Nitasha Wallace, Premiere Banks
We have not yet had an opportunity to tour MSS but hope to do so in the future. Staff has been very
helpful in trying to consistently keep up with our needs. Keep up the good work. Jessica Gerena,
Climatech Industries
Dave, Marlon and the whole MSS crew have been great to work with for our curriculum products. We
truly value this ongoing relationship between MNHS and MSS. Thank You! Kevin Maijala, MN Historical
Society
Working with you is top-notch and exceeds expectations. Deb Meier, Brown & Bigelow
The mobile crew is doing an outstanding job. The coaches are phenomenal and the staff communicates
and reacts timely to all of our requests. It is a very affordable service that fills our needs very well.
Anonymous
Page 8 of 20
DEMOGRAPHICS OF PERSONS SERVED
Total Clients Served
800
700
600
640
613
563
539
559
2008
2009
2010
689
667
2013
2014
500
400
300
200
100
0
2011
2012
Clients Served By Center/Program
81
Apple Valley
66
62
Brooklyn Park
59
Eagan
221
Shoreview
78
St. Paul - DTH
100
St. Paul - Voc. Services
Oakdale
Intakes & Demits - DTH Programs
32
29
Intakes
Demits
8
2
4
6
6
Apple ValleyBrooklyn Park Eagan
5
4
2
Shoreview
Page 9 of 20
4 3
St. Paul
Oakdale
Intakes & Demits - Vocational Services
100
80
60
40
20
0
Intakes
Demits
Age Range
7%
47%
18 - 40
46%
41 - 65
66 - 85
Primary Disability
4% 4% 3%
Developmental Disability
17%
Mental Illness
76%
Medical/Physical
Brain Injury
Other
Page 10 of 20
Ambulation
26%
Assisted
10%
64%
Independent
Uses Wheelchair
Race
5% 2%
12%
4%
African American
Asian
Caucasian/White
Other
77%
Hispanic/Latino
Client Work Hours (Center - Based
Employment)
80000
60000
Client Work Hours (Center Based Employment)
40000
20000
0
2008
2009
2010
2011
2012
2013
Page 11 of 20
Client Work Hours (Community - Based/Enclave Employment)
32404
35000
26786
30000
21494
25000
20000
14332
Client Work Hours (Community…
15000
10000
5000
0
2010
$12.00
$10.00
2011
$10.42
$9.29
$8.74
$8.00
2012
2013
Client Wages by Placement Type
$6.60$6.56
$6.14
2011
$6.00
$3.71$3.72$4.08
$4.00
2012
2013
$2.00
$Competitive
Community - Based
(Enclaves)
Page 12 of 20
Center - Based
2014 PROGRAM OUTCOME MEASURES – VOCATIONAL SERVICES
Measures of Effectiveness
Objective
Maximize
Achievement of
Personal Goals
Measure
Client input: % of “agree” or “strongly agree” client responses to
satisfaction survey item “I am making progress on my goals”.
IDT input: % of “agree” or “strongly agree” IDT responses to satisfaction
survey item “my client/family member is making progress on my goals”.
MSS Designated Coordinator input: % of “made progress” or “maintained”
responses to survey assessment re: goal attainment.
Data Source
Annual Satisfaction
Survey Results
Annual Satisfaction
Survey Results
Designated
Coordinator
Assessment/ Survey
Goal
2014
2013
95%
94%
93%
95%
93%
91%
90%
89%
83%
44
23
27
9 (6+3)
5
0
Increase
Competitive Job
Placements
# of clients who secure competitive employment
# of clients who move from center-based employment to competitive
employment (Ramsey Co EDTH and DEED grant programs).
Director of
Vocational Services
Increase Enclave
Work Opportunities
Total client work hours added.
Accounting Dept.
Increase
+2,896
+5,618
Maximize Job
Retention
% of clients placed in competitive employment who maintain employment
for 90 days or more.
Director of
Vocational Services
75%
78%
79%
$9.60
$10.09
$8.74
$5.93
$6.98
$6.56
$3.45
$5.90
$4.08
Total # of client work hours (center-based only).
65,000+
56,981
40,121
Total # of client volunteer hours (# of client participants x hrs volunteering).
Not set
4 sites
4 types
(33%
increase)
654 hrs
575 hrs
3 sites
2 types
2 sites
1 type
Average hourly wages of clients who secure COMPETITIVE employment.
Maximize Client
Earnings
Average hourly wages of clients who secure ENCLAVE employment.
Average hourly wages of clients who secure CENTER-BASED employment.
Increase Production
Employment Hours
Maximize use of
Production
Downtime
Total # of unique sites and types of volunteer activities that clients in which
clients participated.
Page 13 of 20
Accounting Dept.
Program
Supervisors
Measures of Efficiency
Objective
Minimize Time to
Enrollment
Minimize Time to
Job Placement
Measure
Average # of days from referral to program enrollment for all Voc Rehab
clients.
Average # of days from referral to scheduled intake meeting for all other VS
clients.
Data Source
Director of Vocational
Services
Average # of weeks from start of job search to competitive job placement.
Goal
2014
2013
<10 days
4.6 days
4.7 days
<10 days
3.2 days
8 days
24 wks
24 wks
36.8
wks
Measures of Service Access
Objective
Measure
Improve Service Access
# of service access improvements implemented
Data Source
Service Access
Improvement Log
Goal
2014
2013
5
27
22
Data Source
Goal
2014
2013
95%
96%
100%
95%
92%
99%
98%
99%
95%
87%
100%
100%
100%
98%
Measures of Customer Satisfaction
Objective
Maximize Client
and Stakeholder
Satisfaction
Measure
Client input: % of “agree” or “strongly agree” responses to all ratable items
on the satisfaction survey.
Referral source input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Family member input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Residential input: % of “agree” or “strongly agree” responses to all ratable
items on the satisfaction survey.
Other IDT member input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Enclave/Mobile Work Crew Employer input: % of “agree” or “strongly
agree” responses to all ratable items on the satisfaction survey.
Production Employer input: % of “agree” or “strongly agree” responses to
all ratable items on the satisfaction survey.
CARF accredited programs under Vocational Services
CES: Community Employment Services: Job Development, Job Site Training, Job Supports
EDS: Employee Development Services
EPS: Employment Planning Services
OES: Organizational Employment Services
Page 14 of 20
Satisfaction Surveys
95%
2014 PROGRAM OUTCOME MEASURES – DTH PROGRAMS
Measures of Effectiveness
Objective
Maximize
Achievement of
Personal Goals
Measure
Client input: % of “agree” or “strongly agree” client
responses to satisfaction survey item “I am making
progress on my goals”.
IDT input: % of “agree” or “strongly agree” IDT responses
to satisfaction survey item “my client/family member is
making progress on my goals”.
MSS Designated Coordinator input: % of “made progress”
or “maintained” responses to survey assessment re: goal
attainment.
Total # of client volunteer/community integration hours
(# of client participants x hrs volunteering).
Increase Variety Total # of unique sites and types of community
of COI Activities integration activities provided.
Page 15 of 20
Data Source
Annual
Satisfaction
Survey Results
Annual
Satisfaction
Survey Results
Designated
Coordinator
Assessment/
Survey
Program
Supervisors
Goal
2014
2013
95%
90%
90%
95%
96%
98%
90%
92%
92%
MAINTAIN 8,982
MAINTAIN
185 sites
19 types
7,458
149 sites
14 types
Measures of Efficiency
Objective
Measure
Data Source
Goal
2014
2013
Minimize Time to
Enrollment
Days from referral to acceptance notification
Directors
<14 days
83% < 14 days
87% < 14 days
Measures of Service Access
Objective
Measure
Data Source
Goal
2014 2013
Improve Service
Access
# of service access improvements implemented
Service Access
Improvement Log
5
27
22
Data Source
Goal
2014
2013
94%
96%
100%
99%
94%
100%
94%
100%
98%
100%
100%
100%
100%
98%
Measures of Customer Satisfaction
Objective
Measure
Maximize Client
and Stakeholder
Satisfaction
Client input: % of “agree” or “strongly agree” responses to all ratable items
on the satisfaction survey.
Referral source input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Family member input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Residential input: % of “agree” or “strongly agree” responses to all ratable
items on the satisfaction survey.
Other IDT member input: % of “agree” or “strongly agree” responses to all
ratable items on the satisfaction survey.
Enclave/Mobile Work Crew Employer input: % of “agree” or “strongly
agree” responses to all ratable items on the satisfaction survey.
Production Employer input: % of “agree” or “strongly agree” responses to
all ratable items on the satisfaction survey.
CARF accredited programs under DTH
COI: Community Integration
ASD: Supports for Persons with Autism Spectrum Disorder
Specific Population Designations: Older Adults and Medically Fragile
*Vocational Services overlap: EPS and OES
Page 16 of 20
Satisfaction Surveys
95%
SERVICE ACCESS IMPROVEMENTS
“Service Access” refers to our capacity to provide services for those who desire them, thus the
monitoring and assessment of services is ongoing at MSS. Service access improvements
identified and implemented in 2014 include:






















Increased the use of volunteers to teach a variety of skills
Creative Arts received a second exemplary from CARF
Partnered with “The Show” – displaying artwork in the community & sold reusable tote
(canvas) bags to them
Clients had an opportunity to do paid work doing print screening reusable totes and t-shirts
Increased transportation within Washington County – contracted with NewTrax & Maximal
Mobility
Began replicating the successful “Life Skills” group model from our Apple Valley center at
our Eagan center to prepare to serve more individuals with Autism
Purchased a wheelchair-accessible, ADA compliant ramp for the stage in the Eagan
performance space
Purchased additional Assistive Technology (specifically an interactive, “SmartBoard-style”
projector)
Purchased 4 additional vehicles
Upgraded the computer systems used by our clients
Began yoga classes for our clients at our Shoreview and Eagan centers
Received two grants to work collaboratively with Upstream Arts, culminating in evening
performances open to the community
Bigger emphasis on staff training / not just the required but outside the box stuff
Increasing technology within the centers
Caregiver group becoming an information/resource provider for the legislative changes
Annual “Celebrate” fundraiser success
Gallery and improvements on our website
Increased our staff competency by creating quizzes to show understanding surrounding
individual care
Created a business relationship with Mann Lake LTD through buying our bee supplies, but
also created a new job with them where we put together hive bodies for them to sell.
Created “classes” to enhance the programming in the areas of music, art, sports and
recreation, fitness, healthy living, and daily life skills.
Had a Food Drive where everyone brought in food for the Ralph Reader Food Shelf. Clients
took a trip over to the food shelf to drop it off.
Began hosting a “Celebration of Life” ceremony (memorial) internally as many who
experienced the death of an MSS co-worker/friend were unable to attend funeral services.
Page 17 of 20
This allowed individuals to be able to share their thoughts and have closure for those who
had passed.





Installed wall mounted all-in-one monitors and computers that were more up to date and
larger so they were more accessible and easier to see.
Created a “hot room” and purchased honey extraction equipment that could be used by
people who use wheelchairs as well as others.
Created bee books for the 1st year beekeeping and the 2nd year beekeeping which
highlighted the bee program and its participants. This was shared with clients and their
friends/family/teams, consents provided, on a “share site” on Shutterfly so everyone has
access to buy the books.
Bought an observation hive which allowed us to bring a frame of bees safely inside the
building for people that were unable to get down to the hives.
Created a New Employee Orientation Book to streamline the training process to get staff
trained and on the floor quicker.
Page 18 of 20
CONCLUSIONS
Below are some of the conclusions, learning points, and action steps resulting from our 2014
performance analysis:

MSS served 667 individuals in 2014, which remains within our average of 665 over a 3 year
period. While our DT&H programs have remained relatively stable over the past year, our
employment services have experienced a decrease. Interest in these services remains
strong; however with a push toward more community based employment opportunities, we
have limited our intakes. The program showing the greatest increase in the past year is our
Oakdale center. This program was only in operation for 5 of the 12 months in 2013, but has
experienced a significant increase in 2014, moving from supporting 34 individuals in 2013 to
serving 81 in 2014. Conversely, our St. Paul Vocational Services decreased from serving 296
individuals in 2013 to 221 individuals in 2014. As was stated earlier, with community
placement being the preferred choice of funders, we have limited the referrals we can
intake into our employment programs. We continue to experience at or near capacity
enrollment at all of our DT&H programs, with the exception of our Eagan center, which was
given a licensing capacity increase in 2014. We have experience a steady growth in demand
for our services and anticipate this trend to continue, unless legislative and governmental
initiatives restrict service options.

Our client/stakeholder satisfaction remained virtually unchanged from our 2013 data.
Collective data responses show a 96% overall satisfaction. While lasts years data reflected a
97% overall satisfaction; the only remarkable difference is in the number of respondents,
which increased from 492 IDT members (family, referral source, residential providers, and
guardians) and individuals served in 2013 to 547 stakeholders in 2014. Although this
increase of 55 individuals may appear small, it is important to note that our effort to solicit
feedback on our services is improving. A few data points of note; 98% of those surveyed
believe we treat individuals with respect and dignity. The same number indicated that we
take the culture of those we support and their families into consideration when
planning/implementing services. In both client and stakeholder responses it should be
noted that an overall rating of 93.5 was reported in the area of making progress on
individual goals. This suggests that further discussion may be warranted on the goal
process. Additionally, in light of the strong satisfaction numbers we have received for the
last several years, we will be reevaluating the methods we use and the data we collect.

Securing additional employment opportunities both in-center and in the community
continues to remain a priority. Community based work hours increased by nearly 6,000
hours from 2013. Additionally, our center-based work increased by approximately 40%,
rising to a level more in line with what we have experienced in years prior to 2013. We also
saw an increase in wages across all service domains; this uptick is a direct result of increases
in minimum wage and prevailing wages. Competitive job placements have been on a decline
again this year. While this remains a priority for our agency, we have struggled with
turnover in our placement staff and this has weakened our referral base. Assessing how to
Page 19 of 20
best provide employment opportunities is a strategic initiative of the agency and will be a
significant part of our work in the coming years.

Community integration and volunteerism continues to be an integral part of our services. In
2014 we logged 8,982 hours within the community, which is an increase of 600 hours over
2013. We intend to continue to make this a priority, as our stakeholders continue to feel
this is a valuable and meaningful part of their day services.

In 2014, we moved forward with selling our online job skills training program, e-TRAC®. To
date, we have placed 675 licenses in outstate Minnesota, and have placed an additional 60
throughout the metro area at a number of high schools and day programs. Furthermore, we
gained national recognition when video of an individual in our program was recorded
accepting a position with Wal-Mart and sighted completing e-Trac® as assisting in his
success.

We are grateful for the arts grant we have received in 2014 to collaborate with Upstream
Arts to teach The Art of Me to individuals in all of our programs.

2014 was the first year of our new 5 year plan outlining our strategic goals. Goals include:
optimizing resources and diversifying revenue streams, building an adaptive and innovative
culture, developing leadership bench strength and planning for leadership transitions.
Progress has been made in all of these areas and is a priority for future years Embedded
within the goals is the aim to ensure long and short-term financial viability is taken into
account. This involves maintaining a 90 day fund balance, as well as maintaining, on
average, the days of sales outstanding at 30 days or less. Currently, in all vendor categories
we have an average of 93% under 30 days or less.
Given the client and stakeholder input obtained during this reporting period, along with other
key performance indicators, we believe that our 2014-2019 strategic plan is relevant to our
goals and mission.
Page 20 of 20