Number 2, August 2008
THE PARTNERSHIP IN EVALUATION
TOWARDS RECOVERY (PER) TEAM
‘The evaluation provided a 360 degree view of the service; the team engaged with people.
providing real accounts of the service provision and what needs to be improved”
Feedback from Inpatient Service Manager
at a glance
What:
Why:
How:
Target:
Where:
Partnership in Evaluation towards Recovery (PER) is a service user-led team that evaluates
clinical mental health services within the Counties Manukau District Health Board (CMDHB)
provider arm.
Mental health improvement initiative - improving of the mental health care culture for and with both
providers and service users.
Multi-faceted approach, inductive, qualitative evaluation method, involving staff, service users and
family/Whanau, NGO’s and other associated services.
Mental Health service providers and service users.
Counties Manukau District Health Board area
the profile
The Partnership in Evaluation towards Recovery (PER) team is a service user-led team that evaluates services in the
Counties Manukau District Health Board (CMDHB) provider arm. A consistent evaluation criterion is used for all
stakeholders, such as service users, family/whanau and staff, in order to assess a service against a set of agreed
standards and indicators. Information gathered enables service improvements that support recovery and enhance
service delivery.
the beginnings
This initiative arose as the result of short-comings that surfaced in the service user satisfaction surveys. The survey
results were not clearly identifying unique issues about each of the different mental health teams. In order to obtain a
deeper understanding of the dynamics of service delivery, an alternative method for gathering information was
necessary. The idea of evaluation was championed by the Senior Consumer Advisor, Shirl Corles-Davis.
In 2006, the PER team was established and SAMS (Standards and Monitoring Services) trained the team members
using an Multi-Perspective Approach (MPA) which enables service users, families and providers to constructively
work together to provide a forum for improved service delivery. Consultation meetings with stakeholders led to the
development of seven standards (choice, valued surroundings, treated as an individual, participation, belonging,
privacy, health and wellness). Each standard has a set of key indicators that are used to measure the service
effectiveness from all of the stakeholders perspectives.
Ian McKenzie, the General Manager Mental Health, Community and Intermediary Care Services for CMDHB, has
fully supported the development of this initiative and continues to support the ongoing work of the team. The PER
-1-
team, consisting now of four team members, is part of the mental health research team (REAMHS) at the mental
health management level of the DHB and operates independently of the clinical services.
the process
The inductive qualitative evaluation approach includes inviting all stakeholders who have had contact with the
service over the previous six months to be interviewed. The PER team also conduct focus groups, observations and
attend clinical team meetings, review of clinical files and service core documents,. Evaluators assess how well mental
health teams meet the Standards and identify areas that might need a different approach. The team also assess
whether the environment supports staff and enhances service delivery.
The process is “like putting together the pieces of a jigsaw puzzle” to build a picture of what is working well and what
could be improved.
the unique approach
•
•
•
•
At the development stage of the PER team broad and diverse views from the local community, managers,
professional leaders family/whanau and service users were sought on how the initiative should evaluate services
and what standards and indicators were considered most valuable to the local community.
The PER team is unique being the only service user-led evaluation team operating within a DHB
“We talk to everyone”
in New Zealand.
During the evaluation interviews there are no notes taken. To maintain objectivity, in-depth interviews with
participants are conducted ideally by three members of the PER team. One facilitates and the other two support
and listen. After the interview the PER members discuss the content and identify and document emerging
themes.
The PER team uses positive reframing techniques to focus on celebrating the strengths and positive aspects of a
service’s work and areas of improvements are presented within the context and dynamics of each team. Using
positive reframing techniques differs from many audits methods which often highlight only negative aspects of
work and provide respective suggestions.
the results
•
•
•
•
•
•
•
The initiative has established a strong service user presence in CMDHB service development and improvement.
The Team provides a structured way of giving and receiving feedback from all stakeholders.
Trusting relationships enable staff and service users to come forward with issues and talk
“It is a useful tool to support
about the treatment and service environment freely.
some of the ideas staff were
Evaluations help to highlight intangible aspects of the service including the quality of
putting forward in terms of
relationships between service users and clinicians and between clinicians and their work
quality of service, thus
environment.
leading to mutual goals for
At the conclusion of each evaluation the PER team write the report and final
improved mental health
recommendations are made including information on how to resolve the issues highlighted
care.”
and what outcomes can be achieved. This approach encourages services to implement the
changes in a relatively short timeframe.
The PER team undertake regular return visits to services previously evaluated to discuss the progress of
achieving the recommendations made to that particular service.
Recommendations are also able to be taken to a higher level in the organisation through a number of different
avenues to ensure they are implemented by the service concerned.
Evaluations also add a supportive voice for staff who identify changes required but have not had the means to
bring these about.
the lessons learnt
•
•
An evaluation team requires the full support from management, professional leaders and human resources
within the DHB. It must be incorporated in financial planning and quality systems (funding, recruitment,
training and scheduled service audits).
It requires a mindful evaluation team to enable a healthy and reflective culture that encourages and supports
qualities such as respect, confidentiality, openness to diversity and professionalism and a positive outlook.
-2-
•
•
•
Good recruitment plays a vital role.
Quality relationships are key to providing a good service. The way the evaluation team is treated by a service can
often reflect how the staff work in practice with service users.
This evaluation model has been flexible enough to be applied to all Community Mental Health Centres
(CMHC), specialist services including a Child and Adolescent Mental Health Services (CAMHS). The evaluation
team have the ability to bring onboard further expertise to assist with evaluations if needed.
The PER team, 2008
more information
Contact
• Claire Moore at [email protected] or 09 2760044 ext 8697
Documents/Links
• CMDHB Mental Health Services webpage http://www.cmdhb.org.nz/funded-services/mental-health/default.htm
•
Standards and Monitoring Services webpage
http://www.sams.org.nz/
•
Partnership in Evaluation Pamphlet
available by visiting stories of change at www.tepou.co.nz/knowledgeexchange
-3-