NYSILC 2016 SPIL Evaluation Report
Consultant’s Report on the Third Year of the 2014-2016
State Plan for Independent Living (SPIL)
Submitted by Alan Krieger
Krieger Solutions, LLC
May 5, 2017
Contents
Background ................................................................................................................................ 1
2016 SPIL Evaluation Findings ..................................................................................................... 2
Summative Analysis – How well were the objectives and targets met? ..................................................2
Overview: _____________________________________________________________________________ 2
Summary of the evaluation of each of the objectives and consultant observations. __________________ 3
Objective 1: NYSILC Operation - ___________________________________________________________ 3
Objective 2: Support a Statewide Systems Advocacy Network (SSAN) _____________________________ 7
Objective 4: NYS CB (formerly CBVH) will provide IL services on a fee-for-service basis to eligible
individuals who are legally blind. _______________________________________________________ 8
Objective 5: Capacity Building _____________________________________________________________ 9
Objective 7: Coaching and consulting for Centers ____________________________________________ 11
Objective 9: Develop a Database _________________________________________________________ 11
Overall Summary................................................................................................................................ 12
Formative Analysis ............................................................................................................................. 12
Recommendations ............................................................................................................................. 15
SPIL COMMITTEE REPORT.......................................................................................................... 16
NYSILC 2016 SPIL Objectives/Evaluation Summaries .................................................................. 16
Objective # 1 – NYSILC Operations ...................................................................................................... 16
Objective # 2 – SSAN .......................................................................................................................... 22
Objective # 4 – NYSCB IL FFS ............................................................................................................... 29
Objective # 5 – CBILCO........................................................................................................................ 31
Objective # 7 – CIL Coaching .......................................................................Error! Bookmark not defined.
Objective # 9 – Database .................................................................................................................... 33
NYSILC SPIL Evaluation Report, FY 2015-2016
Background
This evaluation covers the activities from October 1, 2015 through September 30, 2016
related to objectives in the statewide plan for independent living. This is the final year of the
plan and some objectives were already complete or otherwise not active this year. The evaluation
consultant had limited discussions with most of the committees about their evaluation process.
NYSILC staff, as always, did a great job pulling together the data and working with the committees
to finalize their assessments. The evaluation consultant participated in committee meetings to
review the data collected and to continue to focus on the shift from activity or inputs to outputs
or impacts. The evaluation consultant did not review any of the original data, only what the
Council staff pulled together.
As noted in previous reports, the Council has come a long way with writing a more specific
and measurable plan. The objectives and targets are more outcome or impact oriented; vague
terms and standards have been defined more clearly to make the evaluation more objective. The
Council is currently in the process of developing a monitoring protocol so the Council can take an
active role in measuring success in accomplishing the objectives and to get an early awareness if
issues arise so they can be addressed during the program year rather than after the fact.
This year’s evaluation consulting activities included helping finalize the objectives and
targets for the new SPIL, and continuing to look at “value added” components for each objective
that focused more on outcomes and results. Earlier plans consisted mostly of activity or process
oriented objectives. They discussed what the Council or its subcontractors would be doing. The
new SPIL has a few more outcome oriented objectives, but in the push to get it finished, a number
of activity oriented objectives remained in place. So, we continued to add some “value added”
elements to bring more of an outcome orientation to the evaluation.
There were several targets that were not clearly defined in the SPIL (e.g. what is a “system
change”) and committees have taken steps to better define some of these so they can be more
accurately evaluated.
At the June 2015 Council meeting additional training was provided for Council members
on the evaluation process in advance of the development of the new SPIL. The training focused
on developing outcome based goals and objectives to help continue the transition from
process/activity objectives to outcome objectives. In addition, as new individuals continue to be
appointed to NYSILC, it may be useful to plan additional training for them. New members
continue to be provided a link to the webinar recorded earlier. The outcome evaluation training
materials developed and presented the first year the evaluation consultant worked with the
Council are also included in the new member orientation manual.
-1-
NYSILC SPIL Evaluation Report, FY 2015-2016
Some objectives from the previous SPIL were purposely delayed to provide enough time
to develop the RFP’s. The final one or two years of these objectives were extended into this SPIL
with the use of unspent Part B funds.
Similar to the earlier years, each committee developed an evaluation process that
generally relied on the NYSILC staff gathering the relevant data from records for projects being
conducted by NYSILC, or from the DSU for projects that were handled through RFP’s. Some
additional work was done to evaluate some of the “value added” components. Committees are
paying more attention to the actual impact of the projects, not just the amount of effort that was
expended.
Each of the committees conducted their evaluation between November 2016 and
February 2017. The phone calls in which the SPIL evaluator participated usually have very few
committee members participating. This low level of involvement is a concern. Only one
committee had a SPIL evaluation session where they were fully engaged in discussion. This
would be the goal for every committee. Written summaries of the committee meetings were
transmitted to the SPIL evaluator who then drafted an evaluation report based on those
summaries and committee discussions. The full committee summaries are appended to this
report.
2016 SPIL Evaluation Findings
Summative Analysis – How well were the objectives and targets met?
Overview:
Most of the objectives in the SPIL initially look to be very straight forward to evaluate.
While the objectives themselves are not usually written in measurable and specific terms, they
each have measurable indicators that are measurable. These indicators generally relate to
whether or not certain activities took place, how many people participated in the activity, what
the impact was of the activities, and/or how often or to what degree the activity was
accomplished. On a deeper review, some questions emerge. Some of the objectives were
technically met, i.e. the number of activities/people served were achieved. However, the impact
or outcome of these activities in some cases appears to be minimal and not what the objective’s
intention was.
The reverse is also true, some of the targets that the committee felt were “not met” due
to data problems or technicalities were partially met when the data was examined more closely
and the intention or desired outcome was considered. In some cases, the objectives or indicators
were written too specifically. Therefore, the evaluation must show that the objective was
technically not met, although the intention appears to have been met. Similarly, by writing some
-2-
NYSILC SPIL Evaluation Report, FY 2015-2016
of the objectives too broadly, the objectives were technically met although the impact appears
to be minimal and not what was hoped for.
This report is generally based on the technical rating – did the objective and indicators, as
written literally get met. In addition, the consultant added comments where a different rating
would actually be more accurate.
There were 10 objectives in the 2014-2016 SPIL. Four of the objectives (#6 Voter
Registration, #7 CIL Coaching and Consulting, #8 Capacity Building and #10 Deaf-Blind SSP) were
carried over from the prior SPIL for only one or two years, so were inactive this year. There was
no IL Conference this year, so Objective 3 was also inactive. The 5 remaining objectives had 30
specific measurable performance targets set for this year, but only 28 were active. The Council
has adopted a scale or range for rating each target and objective. The scale includes: exceeded,
fully met (100%), substantially met (at least 60%), partially met, not met/no progress. This
reflects the understanding that even when an objective or indicator is not fully met, if there was
substantial progress, that has substantial impact and is worth noting.
Overall, based on strictly numbers, the Council fully met and exceeded the measures for
one objective (#2). Three of the five active objectives were substantially met (objectives 1, 4,
and 5) and one was partially met (#9). The fully completed rate of 20% is lower than any
previous year. Four of the five (67%) were at least substantially met which is slightly lower than
previous years, but overall, the SPIL was substantially met for the past year.
For the five active objectives, there was a total of 30 performance targets, but only 28
were active this year. Thirteen exceeded the target (46%) and five met the target (18%), so 18
out of 28 active targets were accomplished, a rate of 64%, about the same as last year, but
falling short of the ambitious goal of 90%.
As noted in the introduction, these numbers do not tell the whole story. As explained in
the report which follows, the numbers often show the extent of the activities that were
conducted, but don’t always reflect the intention of the objective or the target in terms of
impact or outcomes.
Summary of the evaluation of each of the objectives and consultant observations.
(For more detail see the SPIL committee’s full report, which follows the consultant’s report)
Objective 1: NYSILC Operation - In order to effectively coordinate, monitor, and evaluate the
SPIL, NYSILC will strengthen its operations and capability.
This objective has nine targets, three were exceeded, three were met, one was
substantially met and two were partially met. Overall, the objective was substantially met.
As noted in prior year reports, this objective was revamped in this SPIL to show more of
-3-
NYSILC SPIL Evaluation Report, FY 2015-2016
an impact or outcome. In the earlier SPIL, one target was that meetings were held or the 704
report was completed. In this SPIL, some qualitative elements were added to these targets –
Council meetings had to have a quorum so business could be transacted. The 704 report was
completed accurately and submitted on time to RSA (which has since transitioned to the
Administration for Community Living (ACL).
In actuality, the six targets that were exceeded or met have for the most part been met
for many years and this does not show a “strengthening” but shows a maintenance of good or
excellent performance. To measure “strengthening” the Council would need to identify areas of
weakness, measure the baseline and show improvement after one or more years.
Two of the targets relate to Council administration (704 report and audit). These have
been fully met for many years. One target related to Council operation – the number of meetings
with a quorum – has also been met for many years. It was an issue several years ago, and did
show a strengthening, but since it’s been met for several consecutive years, it no longer shows a
strengthening, but a maintenance.
Two other targets related to Council operation (number of people actively serving on
committees and number of issues addressed) could be a basis for showing strengthening.

Number of people actively serving on committees – the Council reports a very high
number (45), but each time I have participated in committee meetings, the numbers
have been extremely low. The definition of “active” participation may need to be
clarified. I have difficulty accepting the Council’s report on this target based on my
experience, but without more complete data, it should be counted as the committee
identified it, as being exceeded. However, perhaps it is a matter of understanding how
this number was counted toward the outcome. Does it reflect the members who just
attend the committee meetings, or should it be adjusted down to just those
individuals who actively participate during the meetings or to those who attend on a
regular basis? The Council will be now be using Outlook invitations as a way to ensure
good attendance at each committee meeting. They may wish to additionally develop
a definition of what constitutes an “active” committee member.

The “number of issues addressed” was a vague term that the committee has worked
on to clarify and now refers to the fact that the Council took some affirmative action
in relation to the issue and in many cases, the Council has recorded the impact of
those actions. This third target shows real improvement in Council operations and
impact.
Two other targets relate to the overall accomplishment of SPIL objectives and targets.
One was substantially met (#of targets achieved) and the other was only partially met (# of
objectives achieved). The Committee discussing this felt that since this was out of the Council’s
direct control, it should not be counted against the Council if the numbers fell short. This has
been dropped from the upcoming SPIL. They also felt that the group of outcomes should be
specific to the operations and activities relevant to the council. I disagree. The Council sets the
-4-
NYSILC SPIL Evaluation Report, FY 2015-2016
targets and while it does not select the contractors, the Council is allocating funds for these
projects and is ultimately responsible for the administration of these funds. I think the Council
should be accountable for this, perhaps in partnership with the DSU. I have indicated repeatedly
over the years that the targets the Council has set (often in the 80-100% range) is unrealistically
high, and the Council has never met these targets. Ambitious outcome measures are rarely met
at such a high level. If the targets were set at 50% fully achieved and 20% substantially achieved,
this would be a strong result and in many years, the Council could have met this target. I would
encourage the Council to reinstate this as a measure of success in future years.
The final two targets relate to youth participation. One in the broader community and one in
terms of participation on the Council and committees. The one for the broader community
involves providing scholarships for youth to attend leadership development programs. While
only 5 did so this year, the Council also helped support an event where 6 individuals from NY
attended. Therefore, it seems to me that this target was substantially met, not partially met as
the committee concluded. One great addition to this target is a follow up survey that the
Council developed to assess the value of the experience. This helped move this target from an
activity – youth attending an event; to an outcome/impact measure – youth increasing their
leadership skills and active involvement in the community. I have recorded this as being
“substantially met”.
The second youth target related to youth involvement with the Council. Five new
members were appointed to the Youth Leadership Subcommittee and one was appointed to the
Council. The committee is becoming very active including working on a strategic plan for young
adults. Now that the Council has a full complement of young adult members and a strong youth
subcommittee, it may make sense to shift this target to “maintaining strong representation of
young adults on the Council (minimum of 4 representatives), and maintaining a strong and active
youth leadership subcommittee (minimum of 6 members with an active agenda)”. For now, this
can go in as “value added”.
`
Summary:
This objective had nine targets. The committee reported that three were exceeded, three
were met (67% met or exceeded), one was substantially met, and two were partially met. My
assessment is that three were exceeded (although the number of active committee participants
is not clear, based on my experience); three were met; two were substantially met and one was
partially met. Overall, in either case, the objective was substantially met based on the number of
targets met.
As the Committee reported their findings: the six of the nine targets that were fully met
or exceeded include: (numbers in parentheses are # accomplished/ # of original target)
(Note: The first four bullets below are ways to show the Council is operating effectively
administratively. They don’t show mission related impact, but do show competent
administration.)

Number of full council meetings held during the year with a quorum (4/4):
-5-
NYSILC SPIL Evaluation Report, FY 2015-2016

Number of people actively serving on NYSILC committees (duplicated count): 66/45
(104 listed, but 66 actively serving).
Active service is difficult to measure, but would be a meaningful measure. This
target was not carried over into the next SPIL since as it currently stands, since the
committee felt it is not a meaningful measure as is.

Financial audit completed “unqualified” and 990 forms filed fully, accurately, and on
time: 100%.

704 Report completed with DSU to RSA (ACL) fully, accurately, and on time: 100%.

Number of issues addressed by NYSILC committees (23/12).

Number of young adults appointed to NYSILC or joining youth leadership
subcommittee: (6/2). Five on youth subcommittee and 1 on Council.
The following were substantially or partially met:

Number of young adults participating in a training sponsorship: (5/12).

Percentage of SPIL performance targets that are met on an annual basis
(64%/100%). In 2016, there were 5 active objectives with a total of 28 active targets
related to them. 13 of these targets were exceeded, 5 targets were met, 2 targets
were substantially met, 5 targets were partially met and 3 targets were not met.
Totaled, that results in 18/28 targets met or 64%.

Percentage of SPIL objectives that are fully completed on an annual basis: 20%/100%
(partially met). There were a total of five objectives identified by NY SPIL. One of
those five objectives were fully accomplished; the Statewide Systems Advocacy
Network (SSAN) exceeded all seven of its targets.
While much of the Council’s impact is measured in the accomplishment of the SPIL each
year, including the nine targets for objective 1 and the other major objectives that follow, there
are several additional impacts worth noting. These include: maximizing cooperative working
relationships; developing, distributing and posting quarterly newsletters and an Annual Report;
maintaining a website with timely information; developing mechanisms to solicit and recruit
new council members; and conducting a statewide needs assessment. These types of
outcomes should be reflected in any full evaluation of the Council.
-6-
NYSILC SPIL Evaluation Report, FY 2015-2016
Objective 2: Support a Statewide Systems Advocacy Network (SSAN)
New Yorkers with disabilities will be active and will organize to promote disability rights working
with priorities communicated by NYSILC to address issues identified by the statewide needs
assessment through the support of a Statewide Systems Advocacy Network (SSAN) and statewide
coordinator.
There were seven outcomes related to this objective and all seven were exceeded. This
objective therefore was fully met and exceeded. This objective has also been refined over the
years to become more outcome oriented and it appears that this project has produced some
strong outcomes. The most impactful outcome is the number of successful system changes the
network worked on. While the changes can’t be tied specifically to the network’s efforts, the
combined efforts of the network are substantial and it’s reasonable to assume that they had
some impact on the system change efforts along with those of other advocates.
o Number of successful system changes (9; target was 2). Exceeded. A strong definition
was developed for what constitutes a “systems change” and once again there were
several successful system changes over the past year that the SSAN network was
actively involved in. The definition includes that the SSAN coordinator will take a
leadership role in the process and that SSAN providers would be substantially
involved. Another element is that the change must be statewide or lead to statewide
change. It appears that three changes the network worked on meet this enhanced
definition, including getting new laws passed and securing funding for key initiatives.
In addition, six unwanted changes were blocked in the Executive budget and
legislative session. This target was exceeded with some significant impacts.
o Number of partnerships established (140/70). Exceeded. As noted last year, there is
no standard or criteria for what constitutes a meaningful or substantial partnership.
Four impressive examples are cited, but it’s not clear how many of the 140 new
partnerships meet this degree of involvement. However, the SSAN reports on
partnerships do include the activities the partners participate in. I have never received
a copy of these reports. The Council may want to have the Monitoring Committee
verify this information in the future. Having a clear and substantial standard for this
would make this number more impressive and significant. Strengthening this
standard might require a lower overall number as a target. Actively carrying out a
number of activities would be a more meaningful measure.
o Number of education alerts (752/616) Exceeded.
o Number of public education activities (470/280) Exceeded.
o Number of grassroots organizing activities (197/84) Exceeded.
o Number of written/oral testimonies (144/84), Exceeded.
-7-
NYSILC SPIL Evaluation Report, FY 2015-2016
As noted in prior years, the number of education alerts, public education activities,
testimonials, etc. is impressive. It does show that the network is active, engaged in
local and state advocacy and education. The large numbers show that at least some
of the partnerships are meaningful and taking on active roles. It would be very
interesting to be able to show some impact in terms of getting more people involved
or successful change of policies.
This is difficult to measure, but worth continuing to consider. With some email
systems, it is possible to count how many of the alerts are “read” or “opened” and
that would be a good place to start. If the alerts ask people to take action, it could
also include a request to “reply to let us know” that the action was taken. This does
ask the reader to do an additional step and requires the network to undertake
additional record keeping, so instead of doing this across the board for all 906 alerts,
perhaps it could be done with some of the more critical alerts as a small sample (5-10
per year) to get a sense of how effective this network is. With public education
activities and grassroots organizing activities, it would be helpful to estimate the
number of people in attendance. Even better would be to do an email follow up to
those in attendance asking for a brief report on what they might have done as a follow
up. In both cases of asking for a response, only a small percentage of those who
actually do something will respond, so it’s important to keep the targets small and
reasonable, but generating even a small response shows that there is a real impact
from these activities.
o The number of centers without a SSAN contract who are involved in the network
(3; target 2). Target exceeded.
This shows the importance of the funding for centers to take an active role in the
network. While it seems easy to pull in many volunteer partners, it is not as easy to
bring in Independent Living Centers. As part of strengthening the evaluation process,
the committee developed a definition of what it means for a Center to “participate”.
This has been defined as “participating in the action alerts and receiving training to
work on the outcomes”. It may be worth exploring what the barriers are to Center
involvement at this level and see if there is a way to engage more Centers in the
network without additional funding. It would seem reasonable that this number
increases each year, since once a Center is involved, it’s likely to stay involved and
bringing in an additional Center each year might be a good target.
Objective 3: IL Conference - there was no conference held this year, so this objective was not
active.
Objective 4: NYS CB (formerly CBVH) will provide IL services on a fee-for-service basis to
-8-
NYSILC SPIL Evaluation Report, FY 2015-2016
eligible individuals who are legally blind.
For eligible individuals who are legally blind, increase improved access at home or in the
community and/or independence in their own lives, by supporting NYSCB to provide IL Fee-ForServices (FFS) and increase the number of CILs or SCILs in the network providing NYSCB FFS.
There were five outcomes related to this objective. Two outcomes (or targets) were
exceeded, one was met, one was substantially met and one outcome was not met. Overall, this
objective was substantially met.
One target was substantially met:
o Number of legally blind consumers receiving NYSCB Fee-For-Service (FFS):(305; target
was 350; 87%), very similar to last year. At this point, if this were to continue, it would
be helpful to ascertain why the leveling off at just over 300 and if the 350 target was
not realistic. However, this objective will not be included in the next SPIL.
One was met:
o Securing two additional CILs or SCILs to provide NYSCB IL Fee for Services. Two
additional centers signed on, meeting the target. However, a follow up survey of
involved centers showed that there was little actual engagement, so while this
target was technically met, it fell short of the intention of building a network of
providers.
Two targets were exceeded were:
o Number of consumers experiencing improved access at home or in the community
(163; target - 100)
o Number of consumers experiencing increased independence (163; target - 125).
The targets set were 100 (32% of people served) and 125 (40%) respectively. For both
targets 163 people (53% of people served) achieved this, exceeding both the raw
numbers and the percent of accomplishment. These seem to be the most substantive
targets and since both were exceeded, I’d say the objective overall was achieved.
The final target was to ensure that at least 85% of legally blind people receiving the services
and responding to the survey reported that they were satisfied with the services they
received. Unfortunately, the survey was not successfully completed. As a result, this
outcome was not met. However, if in fact 53% of the people served showed significant
changes, it’s likely that at least that many were satisfied with the services received. So,
while technically this target was not addressed, in reality, it’s safe to assume that its
intention (consumer satisfaction) was at least partially and most likely substantially met.
Objective 5: Capacity Building
-9-
NYSILC SPIL Evaluation Report, FY 2015-2016
Improve the capacity of the IL network to address priority unserved and underserved populations
from the statewide needs assessment.
There were a total of five targets for this objective, but only three were active this year. One
was exceeded and two were partially met (but both over 50% putting them close to “substantially
met). Overall, therefore, this objective was substantially met.
The SPIL called for eight funded projects to serve unserved/underserved populations. There
were unexpended funds that allowed an increase so that ten centers were awarded contracts.
As a result, the targets were increased proportionately.
o Number of people served (target 480; adjusted to 570; actual: 317 or 56%). Partially met.
There was some question about how to count the “number of people served”. It was
agreed that establishing a Consumer Service Record (CSR) and providing services to these
individuals is what would be counted, not the number of services provided. On that basis,
the project met 56% of the target. The data included a detailed list of the kinds and
numbers of specific services provided. Providing a service, doesn’t mean it was done well
or had the hoped-for impact or benefit. The committee had decided to conduct a follow
up survey at the start of this year (fall 2016) to get a better sense of how these services
helped consumers in these unserved/underserved populations. This is a good example
of moving from measuring activity (services provided) to impact (benefits of services).
This survey was not conducted in 2016 and has been rescheduled for 2017. It will be
combined with an assessment of self-sustainment.
o Developing a network of community organizations (target 40; adjusted 50; actual 67).
Exceeded
Part of this project was developing a network of community organizations to support the
service delivery and/or assist with recruitment. Again, the question is what value does it
represent to have an organization listed as a contact? The committee discussed trying to
identify some of the benefits, but due to the limited size of the grants didn’t want to make
this administratively cumbersome. One suggestion was for the grantee to indicate when
each agency was contacted what kinds of services/support the agency was willing to offer.
The Council is considering developing a checklist of these kinds of services to make it
easier to collect and report the data. As it currently stands, we know that 67 organizations
were contacted, technically exceeding the target, but telling us nothing about the value
of these contacts. Since the number of consumers served is well below the target, it does
not seem that these contacts provided the desired value. This concern was communicated
to the Outreach Subcommittee, who was responsible for establishing the guidelines for
the CBILCO “How To” manuals. They asked for additional information in the partnership
section of the manual to help ascertain the quality of the relationship. This will help
determine the value of these partnerships.
The ultimate goal is for these projects to become self-sustaining with consumers receiving
- 10 -
NYSILC SPIL Evaluation Report, FY 2015-2016
services after the funding ends, and to develop and document best practices for other centers
wishing to serve these populations. There were three targets related to this:
o Raise funds to support continued operation after the funding ends. (target: $66,666;
adjusted $83,332; actual 43,297 has been raised; 52%) partially met. Some of this is “one
time money” – a donation or grant which may not help with long term sustainability.
Other funding is multi-year or fee for services which have a better chance of moving into
self-sustaining services. Also, this is a capacity building project and if the capacity to
provide these services is fully developed, some of that can be integrated into the ongoing services of a center, supported by the network developed, which could allow for
some self-sustainment without the same amount of funding. As the committee noted,
the real test of this target will be one year after the funding ends. For now, this target
was partially met.
o Have several these programs become self-sustaining. None were expected to achieve
that so early in the program, so that target is not applicable for this year. The Committee
considered this “met” – the goal was zero and we achieved zero. I consider this “n/a” for
this year and excluded it from the overall calculations.
Production of “how to” technical assistance manuals to help other Centers develop
services for these unserved/underserved populations. Again, this was planned for at the
end of the project, so is also not applicable at this stage. The Outreach Subcommittee
sent out an email to the CBILCO projects in October 2016 to inform them about the
guidelines and protocols related to the “How To” manuals. A progress conference call
took place in April 2017. Encouragingly, several centers reported that they have already
begun to develop these manuals. The Committee again considered this “met” – the goal
was zero and we achieved zero. I consider this “n/a” and excluded it from the overall
calculations.
Objective 6: Voting Rights Network - This objective was only funded for the first two years of
the SPIL. It was not active this year.
Objective 7: Coaching and consulting for Centers his objective was only funded for the first
two years of the SPIL. It was not active this year.
Objective 8: Capacity Building – Carry Over Not active in 2016
Objective 9: Develop a Database
Designate funds for the purposes of developing technical specifications and establishing a
database to compile, analyze and interpret data from the statewide network.
- 11 -
NYSILC SPIL Evaluation Report, FY 2015-2016
There were four targets related to this objective. One outcome was met, one partially met
and two were not met. These targets are difficult to understand and measure. They use a
percent to show completion of activities where in some cases there is no framework to
measure percent of completion.
o Percent completion of database design (100%; target - 100%). Met.
The committee selected a vendor and agreed upon a design, so this target was met.
o Number of centers contributing to the design (9%, target - 60%). Partially met.
Four centers actively participated in discussions and online feedback, which
contributed to the database’s design and development. The target was unclear, but
the committee interpreted it as 60% of the 44 centers, or 26 centers. Four centers
constitute 9% of the network of 44. The committee has indicated that for successful
implementation it must do outreach to engage the majority of centers going
forward.
The last two targets were contingent on the data base being ready for testing and reporting.
Since it’s not at that stage yet, neither target was met. Overall this objective was partially met.
Objective 10: Support Service Provider (SSP) Not active in 2016
Overall Summary
Overall the objectives had very strong results even if only one was fully met. The
committees continue to do a good job of clarifying and defining the terms used for evaluation
purposes. As noted above, there are additional areas still in need of definition and specificity.
On the other hand, some of the targets had unrealistically high targets (see objective 1 – SPIL
outcomes).
When setting targets, it would be helpful to think carefully about the intention of the objective
and what are some meaningful and measurable indicators that could show (indicate) that
progress is being made for the objective. Objectives and targets should indicate progress
towards a desired change which is stated in the goal statements. The Council has come a long
way from the start in developing outcome based objectives and indicators. Continued
attention to this will create better and better plans and evaluations that measure meaningful
results and impacts.
Formative Analysis
The above analysis of the results of each objective focuses on “summative” evaluation,
or evaluation strictly of numerical results. Formative evaluation looks at using the data to
inform and improve the operation of the projects. In the current state plan, the Council made
- 12 -
NYSILC SPIL Evaluation Report, FY 2015-2016
an effort to shift from objectives that were more activity based (measuring numbers served,
numbers of events) to those that are more impact or outcome based (how the situation was
overall improved). Some targets in the old plan were overly specific and detailed and were
changed in the new plan to focus more on outcomes rather than specific outputs or activities –
they indicate how things might change as a result of the plan’s efforts, not just what was done
or what was produced.
Several the objectives show a clear shift in the direction of impact based performance
targets and/or outcomes. This is a very challenging shift to make and the Council should be
applauded for the progress made. From initially having no objectives, to in an earlier SPIL
having 14 objectives, each with multiple performance targets that were mostly activity
oriented, to now having a smaller number of objectives that are more outcome focused, is a
large step forward. In the next SPIL the Council has taken this further, further prioritizing the
objectives to a smaller number. This reflects a better sense of the Council’s role in the SPIL, an
effort to maximize impact in light of the reduced funding due to the sequester. Continuing to
include “value added” elements that focus on outcomes is a good strategy for continuing this
development to more outcome based evaluation.
Developing a SPIL is a very large and challenging undertaking. The process rightly focuses
primarily on how resources should be allocated. The time for writing objectives and
performance targets is at the end when there is little time for reflection and revision.
Committees may wish to begin the evaluation process on the first day of the plan
implementation by looking at each objective and performance target and discussing upon what
they would base their evaluation. If it’s not clear in the objective itself, the committee may
want to develop a more specific set of targets and negotiate these with the contractors
involved, so everyone is clear what will be evaluated right from the start.
The Council’s evaluation process focuses more on assessing “yes” or “no,” was the
objective met or not? It could be helpful to also look at “why” an objective was met, exceeded
or fell short. Was failure due to poor planning, poor implementation, or unexpected factors, or
influences outside the control of those running that program? What was the reasons for success?
Were the targets too low, were the providers exceptionally effective, or were additional
resources made available? Whatever the reasons, the Council should look at what can be done
in the following year to address these issues and continue to build on successes.
As these questions are answered, successes and failures can be analyzed and future plans
can have more accurate targets. This can include revising unrealistic targets or objectives down,
or increasing those that were understated. It can also include looking to reallocate resources or
change program designs based on results of the prior year.
In several of these cases, adding quality criteria to the target (e.g. action alerts that are
responded to), will bring down the number of “successful” outcomes, but will show outcomes
that have a measurable impact, not just an activity.
- 13 -
NYSILC SPIL Evaluation Report, FY 2015-2016
The Council’s plans to have an increased role in monitoring project implementation will
help answer some of these questions.
Value added: a number of objectives had “value added” components. In some cases,
these components measured additional outcome results, and in some cases, they dug deeper to
measure impact oriented outcomes. As these questions were answered, more clarity could be
seen as to the true impact of the programs and initiatives. This is the primary area for continued
growth for the Council in terms of evaluation: how to evaluate whether the programs and
initiatives they support are making a difference in furtherance of the mission.
Learning from the prior SPIL, the Council has built in more development time for some of
the objectives. This may need to be expanded to all the objectives. All new initiatives need time
not only for the RFP process, but for establishing the program, developing tools and procedures,
building networks, etc. One strategy discussed earlier is to consider year 1 of the plan a startup
phase and then extend the project into the following plan to allow it time to gear up and operate
for 3 years.
The Chairs of each of the committees serve on the Executive Committee. Therefore, it
was decided last year to use the Executive Committee to help evaluate the evaluation process
overall and the work of the evaluation consultant. That was not done this year. However, this
will change next year as the council transitions to the new Monitoring and Evaluation Committee.
- 14 -
NYSILC SPIL Evaluation Report, FY 2015-2016
Recommendations
As the NYSILC committees continue to analyze and evaluate each of the objectives, they
can continue to assess whether the targets are properly set, resources appropriately allocated,
and systems effectively in place to ensure the best use of funds and the greatest impact on plan
goals. Continuing to work to increase targets that focus on the impact of these objectives will
help with this process, as will conducting conversations and/or surveys with providers and
consumers to gather more detailed data in areas where performance fell short.
In some cases, where an objective or target was “substantially met” they may want to
consider whether the target was overly optimistic and needs to be scaled back or if there are
strategies for improving outcomes in future years.
ATTACHMENTS FOLLOW: SPIL Committee report: detailed evaluations of each objective
- 15 -
NYSILC SPIL Evaluation Report, FY 2015-2016
SPIL COMMITTEE REPORT
NYSILC 2016 SPIL Objectives/Evaluation Summaries
Executive Committee
Objective # 1 – NYSILC Operations
Objective 1: In order to effectively coordinate, monitor, and evaluate the SPIL, NYSILC will
strengthen its operations and capability.
There were nine outcomes related to this objective. Preliminary results indicate that NYSILC
exceeded three outcomes (or targets), met three, substantially met (60%) one, and
partially met two.
2016 Performance Targets:

Number of full council meetings held during the year with a quorum (Target: 4): NYSILC
successfully held all four full council meetings with a quorum during the year at the Troy
Hilton Garden Inn on the following dates: November 13, 2015, April 15, 2016, June 17,
2016, and September 9, 2016. Outcome met.
o This is verified by the summary of minutes approved by the full council for each
meeting. These summary minutes will be archived on the NYSILC website with audio
clips for the meetings: http://www.nysilc.org/index.php/full-council-minutes-audioand-minutes
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target met.

Number of people actively serving on NYSILC committees - duplicated count (Target: 45):
Based on the NYSILC 2016 Organizational and Committee document/listing, the following
active NYSILC committees had the corresponding number of people on their committees
during the past year (duplicated count): Executive Committee (7), Public Policy (13),
Finance Committee (5), Recruitment (6), State Plan (SPIL) Committee (13), Emergency
Preparedness (16), Youth Leadership Subcommittee (7), Development Subcommittee (9),
Consumer Satisfaction Survey Subcommittee (9), Outreach Subcommittee (10), and
Database Work Group (9). Total listed on committees: 104. Total actively serving: 66.
Outcome exceeded.
o This is verified by the NYSILC 2016 Organizational and Committee Materials
document/listing.
o Value added:
- 16 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target exceeded. This is an example of how a target was established
and “value-added” input was received to clarify how it could be improved. Rather
than just count the number of NYSILC committee members on each committee (this
is instead the starting point), the list is reviewed to determine how many of those
individuals are actually “actively serving” on the committees. This number is used.

Number of issues addressed by NYSILC committees (Target: 12): Based on the information
from the July-September 2016 NYSILC quarterly contract report (which also reflects the
end of the contract year), twenty-three issues where addressed by NYSILC committees
during the past year. Total issues addressed by committees for the year: 23. Outcome
exceeded.
o Some of the issues addressed/discussed by committees during the past quarter
include:
 For the Executive Committee: the budget modification and spenddown plan,
restitution from the former employee, the NYSILC 3-year contract,
recruitment of new members, SPIL submittal, the LDCEP program,
consumer satisfaction survey, and database project.
 For the Outreach Subcommittee: the communicate needed to be sent to the
ten CBILCO projects as they enter the last year and coordinating an ad hoc
group to determine the general guidelines for the “How To” manuals for the
CBILCO manuals.
 For the Public Policy Committee: review of the 2016 Presidential postelection survey.
 For the Consumer Satisfaction Survey Subcommittee: review of all the
updates to the process for the upcoming statewide CIL consumer
satisfaction survey process.
 For the Database Work Group: review of options provided by the Center for
Health and Social Research (CHSR).
 For the Emergency Preparedness Subcommittee: an update on Dave
Whalen’s DDPC grant and Core Advisory Group (CAG) meetings.
 For the Recruitment Committee: review and recommendations of new
council members to fill vacancies for 2017.
This target also includes two sub-targets:

Issues successfully address by committees (Target 4): The council
successfully address nine issues during the year. Total issues successfully
addressed by committees for the year: 9. Outcome exceeded. A few of the
issues included:
 The Public Policy Committee completed the NYSILC 3-year Public Policy
agenda. It is posted online on the NYSILC home page at:
http://www.nysilc.org/images/NYSILC_Three_Year_Public_Policy_Agend
a.doc.
- 17 -
NYSILC SPIL Evaluation Report, FY 2015-2016





Committees and council with consultant complete NYSILC 2015 NY
SPIL Evaluation Report:
http://www.nysilc.org/images/Final_SPIL_Evaluation_Report_2015.docx.
The Outreach Subcommittee, with the assistance of the NYSILC office,
completed an Excel spreadsheet, which will be used to monitor the
quarterly and annual outcomes and results of the ten CBILCO projects. It
will serve as a model monitoring tool.
SPIL Committee efforts end up facilitating an approved SPIL for 20172019:
 http://www.nysilc.org/images/NY20172019SPILApprovalLetter.pdf.
 http://www.nysilc.org/images/Clean_Copy_Draft_NY_SPIL_fifth.d
oc.
The NYSILC Needs Assessment Committee (NAC) reviewed the final draft
provided by a consultant that resulted in the 2015 NYSILC Statewide
Needs Assessment: http://nysilc.org/images/NeedsAssesFinalRep.docx.
Unresolved Issues (Target 4): Many of the unresolved issues identified during
the year were eventually resolved. Only two unresolved issues were carried
forward to the next contract year: the database project (SPIL objective # 9)
and the 3-year NYSILC contract and related MWBE issue. During the year,
five unresolved issues were resolved. Total unresolved issues addressed by
committees during the year: 5. Outcome exceeded.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target exceeded. See recommendations for next (this) year for
comments.

Annual financial audit completed “unqualified” and 990 forms filed fully, accurately, and on
time (Target: 100%). Based on the information from the January-March 2016 NYSILC
quarterly contract report, the NYSILC independent fiscal audit was successfully conducted
and presented to the Finance Committee on 3/4/16. CPA Tom Hosey of Lubbe and Hosey
reviewed the statement on the call with members and answered questions. He noted that
NYSILC had an “unqualified” audit, which is the desired outcome. The Finance Committee
formally accepted the audit and presented it at the full council meeting on April 15, 2016.
The
NYSILC
2014
990
was
posted
online
at
the
following
link:
http://www.nysilc.org/images/NYSILC_Filing.pdf. Outcome met.
o This is verified by NYSILC quarterly contract reports which were shared in the full
council meeting packets.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
- 18 -
NYSILC SPIL Evaluation Report, FY 2015-2016
progress, dropped, inactive.
o Comments: Target met.

Annual 704 Report completed with DSU to RSA fully, accurately, and on time (Target:
100%). Based on the information from the January-March 2016 NYSILC quarterly contract
report, NYSILC worked with its partners to help complete the annual 704 report. The NYS
2015
704
Report
is
posted
online
at
the
following
link:
http://www.nysilc.org/images/FINAL_SevenFour.doc. Outcome met.
o This is verified by the 704 Report posted on the NYSILC website.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target met.

Percentage of SPIL objectives that are fully completed on an annual basis (Target: 100%).
In 2016, the NY SPIL had five active objectives. Based on NYSILC new evaluation criteria
for its objectives (exceeded, met, substantially met (60%), partially met, not met – with the
option of inactive or discontinued), this is an assessment of how many objectives were fully
completed (i.e., either the objective reaching full completion or having all of their active
outcomes exceeded or met during the year). Objective # 2 Statewide Systems Advocacy
Network (SSAN) exceeded all seven of its targets. Overall, 1 of 5 objectives were completed
or 20%. This is a 13% decrease from the previous year. However, objective # 1 NYSILC
(this objective) was very close to meeting the criteria to being fully completed. Targets 6 and
7 represent the two measures that pose the greatest challenge for the council. They were
discussed last year and were subsequently addressed going into the next plan so that it is
expressed in a way that is realistic and reflects the council’s responsibilities. Percentage of
SPIL objectives that are fully completed on an annual basis: 13%. Outcome partially met.
o This is verified by a variety of documents available to NYSILC and mentioned in this
report.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target partially met.

Percentage of SPIL performance targets that are met on an annual basis (Target: 100%).
In 2016, the NY SPIL had five active objectives with a total of 30 outcomes or targets related
to them. NYSILC developed new evaluation criteria for its targets. It includes exceeded,
met, substantially met (60%), partially met, not met – with the option of inactive or
discontinued. Targets were evaluated according to these new criteria. Since the number of
active objectives have decreased almost in half, it doesn’t make sense to compare them to
last year (they will be less for each number by comparison). Based on preliminary
- 19 -
NYSILC SPIL Evaluation Report, FY 2015-2016
information, 13 targets were exceeded, 7 targets were met, 2 targets were substantially met
(60%), 5 targets were partially met, and 3 targets were not met (increase from last year).
When the first three are combined (exceeded, met, and substantially met) it amounts to 20
targets with significant results. So, 22/30 or 73% which represents. This is a very good result
and represents a slight decrease (74%) from last year. Outcome substantially met: (73%).
o This is verified by a variety of documents available to NYSILC and mentioned in this
report.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target substantially met.

Number of young adults participating in a training sponsorship (Target: 12). Based on the
information from the July-September 2016 NYSILC quarterly contract report (which also
reflects the end of the contract year), five applications were received for Pat Figueroa
sponsorships. This reflects a decrease in activity compared to previous years. All five
applications were approved. Four of the individuals attended the NCIL conference during
the last week of July in Washington, DC. Two individuals went for the entire event and
accompanied Zach Garafalo and Lyndsi Wickert who facilitate the NYSILC Youth
Leadership Subcommittee. Two additional peers attended the one-day NCIL rally and
participated in visits. They were supported by Aaron Baier and others from ATI Cortland.
The newly elected youth representative to the NCIL board voiced her need to obtain support
to attend future board meetings. She will get reimbursement to attend future NCIL board
meetings. In addition, the council supported the ADAPT youth summit this year which was
held in Rochester instead of Chicago. Twenty youth from across the country attended the
youth/young adult leadership training. NYSILC saw this as a unique opportunity to support
an event so that several youth/young adults could receive leadership training that was taking
place in New York. As a result, only five individuals attended trainings this year. In addition,
six individuals from New York attended the ADAPT summit. NYSILC helped to sponsor the
event. This represents 42% of the target. Total youth and young adult sponsorships of: 5.
Objective partially met.
o This is verified by NYSILC quarterly contract reports which were shared in the full
council meeting packets.
NYSILC created a survey monkey link to consolidate the results for 2013 to 2016 for Pat
Figueroa Sponsorship recipients. This allows for a review of the collective responses to
date. Most respondents felt that the trainers were knowledgeable about their topics (85%
yes, 15% N/A), the training was useful to them (85% yes, 15% N/A), and the materials
were useful (85% yes, 15% N/A). When asked if the training materials would help them
to increase their leadership development skills, 92% replied yes and 8% said no. When
asked if accessible accommodations and or formats were provided, 69% said yes, 8%
no, 8% not sure, and 15% N/A. When asked for comments about their experiences, 2016
comments included:
- 20 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o I had an amazing time getting the chance to advocate for people with disabilities
and help stand up for the rights that they deserve. I also had fun getting the
chance to tour the Capitol.
o Seeing so many people with disabilities gather together all at once was very
emotionally moving for me as it showed how powerful working together can be
and how so many people care about the cause of independent living.
o Being on the NCIL board will be a rich and rewarding experience.
When asked how the event/training changed their outlook on youth/young adults
with disabilities,
2016 comments included:
o I was surprised at how many youths independently attended the event and stood
up for their rights despite the social pressure from the many people around them
who do not understand.
o That the differences of youth with disabilities do not make them less capable, and
they will always have a bright future, no matter what obstacles they go through.
o As the first youth-at-large on the NCIL board, I have been empowered to continue
to amplify the voices of youth with disabilities.
When asked are you willing to participate in activities in your community, sharing what you
learned, and learning more yourself: (60%) were willing to join a youth advocacy group
in your area, (60%) were willing to volunteer at their local ILC, (60%) were willing to join
or start a youth peer support group, and (50%) were willing to present on disability issues
at an event in their local community.
At the September 16, 2016 full council meeting, the council held a youth forum. It was done
to help evaluate the program and hear directly from Pat Figueroa recipients. In the end,
it was felt that the Pat Figueroa sponsorships needed to be better promoted and more
consideration should be given to applications with greater financial need or to events
that have a significant cost to attend. The council will discuss ideas to better promote
and expand the program.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target partially met.

Number of young adults appointed to NYSILC or joining youth leadership subcommittee
(Target: 2). The NYSILC Youth Leadership Subcommittee became active during the past
year. Zach Garafalo and Lyndsi Wickert facilitated the group. There are nine members
active on the committee. They worked on a strategic plan for the committee and four
members attended the NCIL conference. Five new members have been added to, and are
participating in, the committee. Daniel Fuster was recommended to the council and will be
appointed. This will add one youth/young adult member to the council. Total youth: 6 (5
youth subcommittee and 1 NYSILC). Outcome exceeded.
- 21 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o This is verified by NYSILC quarterly contract reports which were shared in the full
council meeting packets.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments: Target exceeded.
o Recommendations for Next Year:
Before the recommendations are provided, the Executive Director should have framed the
2016 SPIL evaluation before engaging in it. First, this is the 2016 evaluation, so it
focusses in on the last (third) year of the previous state plan. The objectives and the
outcomes were discussed and improvements were made going into the new state
plan. Are they perfect? No, but they are improved based on input from the evaluation
process we have.
As a result, after a lengthy discussion, it was recommended that the council utilize the
presentation segments of its full council meetings in 2017 to review the current SPIL
objectives and outcomes and look to improve the language – especially the outcomes.
At a minimum, it would develop the “value-added” language to improve what should be
in the SPIL. The SILC should seriously decide whether it should amend an active state
plan and everything that is entailed.
Public Policy Committee
Objective # 2 – SSAN
Objective 2: New Yorkers with disabilities will be active and will organize to promote
disability rights working with priorities communicated by NYSILC to address issues
identified by the statewide needs assessment through the support of a Statewide
Systems Advocacy Network (SSAN) and statewide coordinator.
There were seven outcomes related to this objective. Preliminary results indicate that all
seven outcomes (or targets) were exceeded.
2016 Performance Targets:

Number of SSAN significant statewide system changes (Target: 2). The SSAN participated
in the following significant systems changes were achieved during the past year:
o Community First Choice Option (CFCO) Approved: On October 23, 2015, the State
Plan Amendment was approved at long last, retroactive to July 1, 2015. This means
NYS will finally begin implementing CFCO. It also means that the State is eligible for
an additional 6% FMAP on CFC services, which will result in a significant amount of
savings to the State. It should be noted that CMS approved the SPA without the
amendment to the Nurse Practice Act to allow for Advanced Home Health Aides
(AHHAs) because they consider this issue a state matter. However, the State still
needs to amend the Nurse Practice Act to allow for AHHAs for full and robust CFC
- 22 -
NYSILC SPIL Evaluation Report, FY 2015-2016
implementation. The legislature did pass AHHA legislation (see below for more
detail) which we expect Governor Cuomo to sign. NYAIL and the SSAN have
advocated for CFCO implementation for several years. Having CFCO means many
additional long term services and supports are now available as State Plan services
through Medicaid. The State has also committed to allocating savings from CFCO
implementation to funding Olmstead implementation. NYAIL and the SSAN will
continue to advocate for AHHAs as well as how the State should allocate savings
realized through CFCO.
o NY ABLE Approved: Throughout the 2015 legislative session, NYAIL advocated for
the New York achieving a better life experience (NY ABLE) savings account act,
A.7767B (Gunther) and S.4472D (Carlucci). The ABLE Act will allow qualified New
Yorkers with disabilities and their families to create tax deferred savings accounts to
cover disability related expenses. The main benefit of these accounts is these assets
will not count against programs such as Medicaid and SSI. Governor Cuomo signed
this legislation into law on December 22, 2015. While this bill was not on NYAIL’s
DPA, NYAIL met with both sponsors to ensure the best version possible and to
ensure its passage. Many advocates within the SSAN also met with their legislators
in support of the bill.
o Mental Health Education Bill Approved: The Mental Health Education bill, A.3887B
(Nolan) and S.6046 (Marcellino), will ensure that mental health is included in any
health education students receive. Through NYAIL’s Inclusive Education committee,
the SSAN supported this bill throughout the 2016 legislative session by meeting with
legislators, building coalitions of support, and responding to the action alerts NYAIL
distributed in support. This bill was signed into law on October 1. While we
acknowledge this was technically signed after the end of the reporting period, it was
sent to Governor Cuomo, and all of our advocacy took place, within the reporting
period.
In addition, the following systems change actions prevented negative activity from
occurring:
o Successfully delayed the transition of NHTD and TBI Waivers into managed care:
The Nursing Home Transition and Diversion (NHTD) and Traumatic Brain Injury (TBI)
Medicaid Waivers were scheduled to be transitioned this year into managed care.
Last year, advocates were successful in securing a one-year delay and a workgroup
process to provide DOH with input to ensure a smooth transition. However, despite
many workgroup meetings between DOH and stakeholders, there were many
outstanding issues which had yet to be addressed. Stakeholders felt strongly that the
draft transition plan did not protect services for participants in the two Waivers.
Throughout the Budget process, NYAIL and SSAN advocates advocated for a delay
to the waivers. As a result of this advocacy, the Budget delayed the transition to 2017,
allowing additional time for DOH and stakeholders to address outstanding concerns.
Throughout the year since, NYAIL and several SSAN centers have participated in
those stakeholder workgroup meetings and provided written comments on DOH’s
draft NHTD/TBI Waiver Transition Plan.
o Spousal and parental refusal protections maintained: The proposed Executive
budget called for a change to the Medicaid regulation which allows a parent or
- 23 -
NYSILC SPIL Evaluation Report, FY 2015-2016
spouse of an individual with a significant disability to refuse to support them
financially. This regulation allows the individual to keep their Medicaid eligibility
without forcing the parent or spouse to give up all of their income. In the proposed
Executive budget, the requirement would be for the parent or spouse to live
separately in order to refuse to financially support the individual with the disability.
This harmful proposal would force couples to divorce, or for the individual with the
disability to go into an institutional setting in order to receive the necessary services
provided under Medicaid. NYAIL strongly opposed this harmful proposal on our
budget DPA and SSAN staff advocated for the legislature to reject this proposal in
their one house budgets. Ultimately this harmful proposal was not included in the
final budget.
o Prescriber Prevails maintained in final budget: The Executive budget proposed
eliminating the crucial prescriber prevails provisions in Medicaid. This proposal
would have repealed an important patient protection in the Medicaid program which
restored “prescriber prevails” for several classes of drugs in the fee for service and
managed care programs. A prescriber, with clinical expertise and knowledge of his
or her individual patient, should be able to override a preferred drug. Different
individuals may have very different responses to different drugs in the same class.
Sometimes only a drug is effective or alternative drugs may have unacceptable side
effects. Prescribers are in the best position to make decisions about what drug
therapies are best for their patients. NYAIL strongly opposed this proposal in our
budget DPA and SSAN staff advocated against it in meetings with legislators on
NYAIL’s legislative day and throughout the budget process. Prescriber prevails
protections were included in the final budget.
o Community Spouse Resource Limit Maintained: The Executive Budget proposal
included a provision to lower the Community Spouse Resource Allowance to merely
$23,844, the minimum allowed by federal law. Currently, New York allows a
community spouse to keep the maximum allowed by federal law. As New Yorkers
face some of the highest costs of living in the country, this makes sense. Lowering
the resource limit to the lowest allowed by Federal law would have dire
consequences for the community spouse, who is not on Medicaid and who may
require assets in order to pay his or her own medical expenses. NYAIL opposed this
proposal in our Budget DPA and SSAN staff vigorously opposed the proposal on
NYAIL’s Legislative Day and throughout the Budget session. This proposal was not
included in the final budget.
o Assisted Suicide bills die in legislature: Throughout the legislative session, NYAIL
and our SSAN providers have actively worked to educate the legislature on the
particular dangers people with significant disabilities face when assisted suicide is
legal. At the beginning of the legislative session, NYAIL facilitated a webinar on the
issue to help educate SSAN providers and others in the network to prepare them to
speak to these particular concerns and dangers. NYAIL also helped to organize
meetings with legislators on the Assembly and Senate Health committees as well as
an advocacy day in opposition. Many legislators who were undecided at the
beginning of session decided to oppose the bills based on the education they
received from the disability community. While this proposal will no doubt be raised
again in future sessions, the disability community has made a real impact in the
minds of many legislators.
- 24 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o Uber bills die in Assembly: NYAIL’s transportation committee has been opposing any
bills seeking to allow transportation network companies (TNCs), such as Uber and
Lyft, to operate in New York unless they have accessible vehicles. NYAIL and many
SSAN centers participated in multiple lobby days in Albany opposing the State
allowing these services to operate unless they include accessibility. As a direct result
of these meetings, Assembly members Weprin and Fahy asked for suggested
language for a bill to include a percent of their fleet be accessible. Assemblyman
Weprin did introduce a bill to mandate a percentage of these vehicles be accessible.
While the Weprin bill did not move in the Assembly, neither did the insurance bill
seeking to allow TNCs to operate in New York.
o Total number of significant system changes: 9. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of local partnerships and coalitions established by the SSAN network (Target: 70).
The SSAN centers established 140 new partnerships and coalitions, exceeding the goal by
70 – a decrease from last year but still far exceeding the annual target. A few examples of
the partnerships and coalitions include:
o AIM worked with a local restaurant to create menus in large print and Braille.
o ILCHV has been working with Transport Troy and many other local Troy
organizations to put on an event called: The Collar City Ramble. The purpose of the
Collar City Ramble is to shine a light on Complete Street proposals and examine the
accessibility of trails along with bike roots, boating and other activities.
o FLIC re-established connections with the Tompkins County Board of Elections, as
part of their voter registration efforts, and to get an article published about the Ballot
Marking Device in the Ithaca Journal.
o WNYIL has started working with the New York State School for the Blind in Batavia.
o Total local partnerships and coalitions: 170. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of educational alerts disseminated to local volunteers by the SSAN network
(Target: 616). Distribution of alerts is one of the areas in which SSAN centers are most
active with a total of 752 alerts distributed to local volunteers. This was a decrease
compared to last year, but again exceeding the annual target. Subsequently, SSAN
advocates collectively disseminated 156 alerts to their local volunteers. Examples include:
o Support HR 5689, the Disability Integration Act!
- 25 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o ADA Alert! Fight serious attack on the ADA!
o Tell Your Senators to Support S.2680 and Reject Harmful Language in Mental Health
Bills!
o Urge Your Senators to Support Passage of S. 2680, The Mental Health Reform Act
of 2016!
o Protect CDPA - Stabilize wages and reimbursement!
o Call Today in Support of the Mental Health Education Bill!
o Get DIA into the Presidential Debate!
o Total educational alerts disseminated: 752. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of local public education activities engaged in by the SSAN network (Target: 280).
The SSAN Centers exceeded this goal by engaging in 470 public education activities in the
past year. This exceeded last year’s activity. Examples of local public education include:
o WDOMI hosted a meeting with Yonkers City Council members to discuss their
proposal for the City of Yonkers to amend its Affordable Housing Ordinance to set
aside 10% of housing units which have been designated as affordable housing in
any new or renovated housing developments for individuals living with mobility and
sensory disabilities.
o RCAL provided local Disability Sensitivity presentation to New York Connects and
OFA employees at Ulster County Business Resource Center (BRC).
o AIM met with a business council to discuss accessibility concerns in the district. The
council agreed to conduct outreach to businesses and to work with businesses to
address the accessibility issues, including fixing broken door openers. Also, an article
about the importance of accessibility was included in the council’s newsletter.
o The Systems Advocate from Independent Living, Inc. attended a workshop on
Americans with Disabilities Act Transition Plans and Other General Requirements
provided by NYS Department of Transportation (DOT) and the Federal High Way
Administration. The Systems Advocate presented on the role that ILCs can play in
transition planning.
o Total public education activities engaged: 470. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:
- 26 -
NYSILC SPIL Evaluation Report, FY 2015-2016

Number of grassroots organizing activities engaged in by the SSAN network (Target: 84).
SSAN sites reported a total of 197 grassroots organizing activities for the year. This
exceeded last year’s activity. Examples of grassroots organizing activities include:
o ATI organized local advocates to attend and participate in the NCIL March, Rally,
and Capitol Hill Visits in Washington, D.C.
o STIC met with the Education Advocacy Team at STIC (parents, teachers, service
coordinators, advocates, behavior specialists, professors) to review NYSED’s
proposal for Superintendent’s Determination (affects high school students with
disabilities). NYSED also requested public comments for numerous mandate relief
proposals. EAT met to consider these proposals, too.
o WNYIL held an ADA Education Event at Delaware Park in Buffalo NY, where over
600 persons attended.
o ILCHV has established an Advocacy group to address disability legislative issues.
They are working on developing a political candidate questionnaire for the upcoming
election. They are contacting their local political officials and/or those running for
political office to invite them to participate in a “Greet and Meet” seminar to be held
in early October.
o ARISE organized an ADA 26th Anniversary Celebration in Syracuse. Over 400
people attended the program with speakers and then parade through downtown to
celebrate the ADA.
o Total grassroots organizing activities engaged: 197. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of oral or written public testimonies statements or letters provided, in response to
a documented request, by the SSAN network (Target: 84). SSAN sites collectively provided
144 public testimonies during the past year. This exceeded last year’s activity. Examples
of public testimony include:
o WDOMI submitted written comments regarding HUD’s proposed Small Area Fair
Market Rent (SAFMR) proposed rule.
o STIC submitted written testimony to NYSED, in response to a request for comments
on the field advisory on Least Restrictive Environment – School District
Responsibilities.
o ARISE submitted written comments to Social Security Administration opposing
proposed rule Implementation of the NICS Improvement Amendments Act of 2007.
SSA-2016-0011 Regulations.gov. Social Security should not automatically submit
names of people using a representative payee for their SS benefits to the FBI’s
National Instant Criminal Background Check System.
o RCAL send a letter to Governor Cuomo in support of the visibility tax credit bill which
passed the legislature this year.
o Total number of public testimonies: 144. Outcome exceeded.
- 27 -
NYSILC SPIL Evaluation Report, FY 2015-2016

This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of CILs or SCILs without a SSAN contract voluntarily involved in the participation
of the network (Target: 2). Three non-SSAN centers Taconic Resources, Bronx
Independent Living Services, and the Center for Disability Rights actively participated in
SSAN activities throughout the year. Advocates from these centers participated in one or
more NYAIL public policy subcommittees, attended at least one of NYAIL’s many training
opportunities, and all three centers participated in NYAIL’s Legislative Day.
o Total number non-SSAN centers involved in network: 3. Outcome exceeded.
 This is verified by the 2016 SSAN Yearend Report.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:
o Formative Evaluation (what have we learned):
- 28 -
NYSILC SPIL Evaluation Report, FY 2015-2016
Consumer Satisfaction Committee
Objective # 4 – NYSCB IL FFS
Objective 4: For eligible individuals who are legally blind, increase improved access at
home or in the community and/or independence in their own lives, by supporting
NYSCB to provide IL Fee-For-Services (FFS) and increase the number of CILs or
SCILs in the network providing NYSCB FFS.
There were five outcomes related to this objective. Preliminary results indicate that two
outcomes (or targets) were exceeded, one met, one outcome was substantially met,
and one outcome was not met.
2016 Performance Targets:

Number of legally blind consumers receiving NYSCB Fee-For-Service (FFS) (Target: 350).
Based on information received from NYSCB and their central database, 305 individuals
received NYSCB IL FFS during the past year. This represented almost 87% progress
toward the outcome. It slightly increased from the year before (compared to 301 individuals
and 86%). The service level seems to have leveled off at around the 300 mark. Total number
of legally blind consumers receiving NYSCB FFS: 305. Outcome substantially met.
o This is verified by information gathered from the NYSCB Computerized Case
Management System (CIS) and or Central Office/District Office records.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Increase in the number of CILs or SCILs providing NYSCB IL Fee-For-Service. (FFS)
(Target: 2). During the past year, two CILs or SCILs were added to the existing group to
increase the number providing NYSCB IL Fee-For-Services as well as other services. The
Regional Center for Independent Living ((RCIL) in Rochester began providing sign
language interpreter services and the Resource Center for Accessible Living (RCAL) in
Kingston began providing job coaching and sign language interpreter services. Increase in
the number of CILs or SCILs providing NYSCB FFS: 2. Outcome met.
o This is verified by information gathered from the NYSCB Computerized Case
Management System (CIS) and or Central Office/District Office records.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of legally blind consumers who receive services and experience improved access
at home or in the community (Target: 100). Based on information received from NYSCB and
their central database/district office records, out of the individuals who received NYSCB IL
FFS during the past year, 163 legally blind consumers experience improved access at home
- 29 -
NYSILC SPIL Evaluation Report, FY 2015-2016
or in the community. It represents a decrease from last year’s result (183), but still well
exceeds the target. When this impact is compared to the number of people served (305), it
makes the statement that 53% of the consumers who received this service believe they
experienced improved access at home or in the community (163/305). Total number of
legally blind consumers who receive services and experience improved access at home or
in the community: 163. Outcome exceeded.
o This is verified by information gathered from the NYSCB Computerized Case
Management System (CIS) and or Central Office/District Office records.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of legally blind consumers who receive services and experience increased
independence in their lives (Target: 125). Based on information received from NYSCB and
their central database/district office records, out of the individuals who received NYSCB IL
FFS during the past year, 163 legally blind consumers experience increased independence
in their lives. It represents a decrease from last year’s result (183), but still well exceeds the
target. When this impact is compared to the number of people served (305), it makes the
statement that 53% of the consumers who received this service believe they experienced
increased independence in their lives (163/305). Total number of legally blind consumers
who receive services and experience increased independence in their lives: 163. Outcome
exceeded.
o This is verified by information gathered from the NYSCB Computerized Case
Management System (CIS) and or Central Office/District Office records.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Percentage of satisfied legally blind survey recipients who receive services (Target: 85%).
NYSCB was working with the Center for Essential Management Services (CEMS) to
conduct the Independent Living Consumer Satisfaction Survey. Unfortunately, the survey
was not able to be successfully completed due to a variety of factors, including the
retirement of the vendor along with challenges to NYSCB staff resources. Percentage of
satisfied legally blind survey recipients who receive services: N/A. Outcome not met.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:
o Formative Evaluation (what have we learned):
- 30 -
NYSILC SPIL Evaluation Report, FY 2015-2016
Outreach Committee
Objective # 5 – CBILCO
Objective 5: Improve the capacity of the IL network to address priority unserved and
underserved populations from the statewide needs assessment.
It should be noted that the original objective was written for eight projects. However,
unexpended funds allowed for two additional projects to be awarded for a total of ten
projects. Therefore, the original targets will be stated per the SPIL, but note the
adjusted number is for ten projects. These outcomes are for year two of the objective.
The third year will be carried over to the first year of the next SPIL.
There were five outcomes related to this objective. Preliminary results indicate that one
outcome (or target) was exceeded, two outcomes were met, and two were partially
met.
2016 Performance Targets:
1) Number of people served by identified target unserved/underserved population (Target:
480, Adjusted target: 570). Collectively the ten sites reported that they established
Consumer Service Records (CSRs) with 317 new individuals over the past year. The
committee noticed that this number represented 56% of the target for the year. The
group felt that it could be due to due an aggressive target that may have been set too
high. It was felt that Brad should mention this fact in an October email communication to
the CBILCO projects and encourage them to improve CSR productivity during their final
year. Also, the issue of monitoring came up as a concern that would have to be
addressed in the new SPIL. The SILC would have to increase its involvement for SPIL
projects and develop protocols for how to communicate with them during the monitoring
process. The Outreach Subcommittee identified four projects for Brad to reach out to.
He touched base with them which added clarity to information not found in the reports.
A fourth project ended up ceasing their project due to difficulties maintaining a project
coordinator.
o Total number of people served by identified target unserved/underserved population:
Outcome partially met.
 This is verified by July-September 2016 quarterly reports submitted by
CBILCO vendors, the CBILCO Excel spreadsheet and minutes.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of community organizational contacts related to targeted populations that are
developed as a result of outreach efforts (Target: 40, Adjusted target: 50). Collectively the
ten sites reported that they made 67 community organization contacts as a result of
- 31 -
NYSILC SPIL Evaluation Report, FY 2015-2016
outreach efforts related to their target populations. The projects were active in the first
quarter, minimal in the second, and then picked up in the third quarter. Examples of selected
community contacts from the projects include: Elmira and Horseheads School Districts,
Syracuse University Parent Assistance Center, Cortland Works Career Center, Cortland
and Homer High Schools, Acacia Network, Inc., Montefiore Community Center, Niagara
Orleans BOCES, Clinton County Office for the Aging, Wiggletown Taxi, Clinton County
Department of Health, Rockland Youth Bureau, Southern Tier Veterans Support Group, and
Wounded Warriors Project.
o Total number of community organizational contacts related to targeted populations
that are developed as a result of outreach efforts: 67. Outcome exceeded. This
number exceeds both the SPIL and adjusted targets.
 This is verified by July-September 2016 quarterly reports submitted by
CBILCO vendors, the CBILCO Excel spreadsheet and minutes.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Amount of new funding secured toward self-sustaining programs (Target: $66,666, Adjusted
target: $83,332). Collectively, six sites have worked on developing self-sustaining
opportunities, while two sites successfully secured funds during the year. Two projects
attracted $43,297. WNYIL attracted $200,000 from the Tower Foundation for its youth with
disabilities project, with a yearly disbursement schedule. They have access to
approximately $41,047 annually. ATI Cortland obtained $2,250 in grants to support their
youth with disabilities project from two different sources. The committee noted that while
this target looks at funds attracted to the project, an assessment of sustainment won’t be
made until a year after the projects end – as noted in the next target.
o Total amount of new funding secured toward self-sustaining programs: $43,297. This
represents 52% of the adjusted target. Outcome partially met.
 This is verified by July-September 2016 quarterly reports submitted by
CBILCO vendors, the CBILCO Excel spreadsheet and minutes.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:

Number of self-sustaining programs (Target: 0) The measure for this target is zero and no
project has yet to attain a “self-sustaining” status after the completion of the second year of
activity. Remember that while this is the third year of the SPIL, these projects are in their
second year of implementation. Total number of self-sustaining programs: 0. Outcome met.
o This is verified by July-September 2016 quarterly reports submitted by CBILCO
vendors, the CBILCO Excel spreadsheet and minutes.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
- 32 -
NYSILC SPIL Evaluation Report, FY 2015-2016
o Comments:

Number of “how to” technical assistance manuals submitted to NYSILC (Target: 0). Just
about every project noted progress in their quarterly reports on their “how to” technical
assistance manuals. The group felt that NYSILC should convene a group of the committee
the content of the “how to” manuals for the CBILCO projects. This should include how to
evaluate the manuals and best distribute and post them for the benefit of the network. The
council must send out an October communication to the projects that provides guidance
related to the manuals. Total number “how to” technical assistance manuals submitted to
NYSILC: 0. Outcome met.
o This is verified by July-September 2016 quarterly reports submitted by CBILCO
vendors, the CBILCO Excel spreadsheet and minutes.
o Summative Evaluation: Exceeded, met, substantially met (60%+), partially met, not
met, no progress, dropped, inactive.
o Comments:
o Formative Evaluation (what have we learned):
SPIL Committee
Objective # 9 – Database
Objective 9: Designate funds for the purposes of developing technical specifications and
establishing a database to compile, analyze and interpret data from the statewide
network.
There were four outcomes related to this objective. One was met, one partially met, and
two were not met.
In hindsight, a more realistic timeframe should have been foreseen related to the
development of database product.
2016 Performance Targets:
1) Percentage of completion of a multi-purpose, interface database design (Target: 100%).
The Database Work group met four times during the year. At the 3/31/16 meeting, the group
met to review three vendor proposals for the database and interface based on the project
description. In the end, the group felt that the proposals required additional information.
However, the group was only interested in interviewing the state university and Albany
based vendors. Brad thanked the third vendor for their time and arranged dates for follow
up discussions with the two vendors in early May. The group is generating questions to ask
each vendor in advance of the calls. During the 5/2/16 meeting, the members met with a
local vendor to follow up on questions related to their proposal. They were pleased with the
details of the proposal including the scope of work. The cost was within the $75,000 budget.
Ongoing maintenance was $15,000 a year and including hosting costs and revisions to the
database. Brad will follow up with credentials and work examples. At the 5/3/16 meeting,
- 33 -
NYSILC SPIL Evaluation Report, FY 2015-2016
the group met with a statewide vendor to address questions related to their proposal.
Members valued that the vendor had advanced knowledge of social research. However,
they identified instances in the proposal that addresses only minimal requirements. In order
to achieve what members felt was four times during the year, appropriate in terms of an
interface, mobile devices, and a mapping option will require a significant amount of
additional funds. Ongoing maintenance would cost $15,000 a year. It did not include hosting
(extra) or changes to the system. The vendor indicated they will follow up with additional
materials. During the 8/4/16 meeting, the group met Center for Health and Social Research
(CHSR) staff who provided a live demonstration of a potential online product. The following
link
provides
audio
to
the
webinar
session:
http://www4.buffalostate.edu/centers/chsr/media/2016-0804%2010.12%20NYSILC%20Webinar.mp4. The workgroup ultimately felt the proposal
submitted by the local/regional vendor Jeremy McGowan Enterprises Corp had the best
potential and discussed it at a meeting with ACCES-VR on September 20, 2016. Next steps
were then discussed related to the project. The conceptual design of the project was
complete. It would need to be addressed as a network along with confirmation of data fields
and current versions of center databases.
o Percentage completion of data interface design: (100%). Outcome met.
 This is verified by the Database Workgroup meeting minutes and materials.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments:
2) Percentage of CILs/SCILs contributing to the multipurpose, interface database design
(Target: 60%). Four members of the work group are CILs/SCILs and participate actively in
discussions and online feedback that contribute to the database’s design and development
(STIC, WNYIL, ATI Cortland, BILS, and CDCI). It will be crucial for the project that and
education piece be developed and centers understand the nature of the project as
confirmation of the data fields and center databases occur going forward.
o Percentage of CILs/SCILs contributing to design: 16%. Outcome partially met.
 This is verified by the Database Workgroup meeting minutes and materials.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments:
3) Percentage of 21st Century Data and Collection Model Workgroup CILs/SCILs that will
engage in testing the new database design and framework (Target: 75%). No work has
- 34 -
NYSILC SPIL Evaluation Report, FY 2015-2016
taken place on this step of the process. It is premature. The design of the product and
identification of a vendor was recently achieved.
o Percentage of 21st Century Data and Collection Model Workgroup CILs/SCILs that
will engage in testing the new database design and framework: 0%. Outcome not
met.
 This is verified by the Database Workgroup meeting minutes and materials.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments:
4) Percentage report completed by NYSILC and the DSU (ACCES-VR) based on statewide
2012 CIL/SCIL data (Target: 100%). No work has taken place on this step of the process. It
is premature. The design of the product and identification of a vendor was recently achieved.
o Percentage report completed by NYSILC and the DSU (ACCES-VR) based on
statewide 2012 CIL/SCIL data: 50%. Outcome not met.
 This is verified by the Database Workgroup meeting minutes and materials.
o Value added:
o Evaluation: Exceeded, met, substantially met (60%+), partially met, not met, no
progress, dropped, inactive.
o Comments:
o Recommendations for next year:
- 35 -