Centre for Behaviour Health Sciences
Executive Functioning
A Newsletter for Senior Leadership in Organizations Providing Human Services
June 2014
Executive Functioning and Evidence-Based Decision Making
An Introduction to a New and Exciting Newsletter
By: Dave Hingsburger, Jim Hughes, Angie Nethercott
It took Google 22 seconds to take two words that were typed into the search engine, those words being
‘disability’ and ‘journal,’ to complete the search. In those 22 seconds, they came up with ‘about
17,000,000 results.’ Any reasonable person would be overwhelmed by this figure in about .75 seconds.
Google was a bit faster to search for the words ‘conferences on disability,’ taking only 15 seconds to come
up with ‘about 1,220,000 results.’ Perhaps this is what they mean by ‘information overload.’
Our sector, serving people with intellectual disabilities, has really grown up. In fact, human services, no
matter which population is served, has really come into its own. A mere twenty years ago, it was hard to
find much research into questions whose answers would lead to good evidence-based decision making, a
solid set of practices to guide staff, and a high quality of life for people who live in, or receive service from
human services.
Executive Functioning is a newsletter that will be publishing information for those in senior leadership
positions in human services. The goal is to provide timely, easy to read, practical information that will
make information accessible. The editorial team will be publishing articles that deal with current issues
and ‘hot button topics.’ In the information age it can be, ironically, difficult to find the kind of information
you need, when you need it. That is where this newsletter comes in.
Evidence-Based Decision Making
All the articles that will be published will look at evidence-based decision making. We hear a lot about
‘evidence-based practice’ these days and it might serve well as an introduction to the new endeavour to
review exactly what that means. Oh and, by the way, Google found 22,100,000 results for ‘evidence-based
decision making.’ We’ll try to make this a whole lot less painful than that. In sifting through those results,
one definition of ‘evidence-based decision making’ stated fairly clearly what it was:
“Evidence-based decision making requires a systematic and rational approach to researching and analysing
available evidence to inform the policy making process. It helps people make well informed decisions
about policies, programmes and projects by putting the best available evidence from research at the heart
of policy development and implementation.” (Davies, 2004: 3).
Volume 1 - Issue 1
1
By the way, this quote is from the Jerry Lee Lecture, given at the Campbell Collaboration Colloquium, the
magic of the internet means that you didn’t have to be there, didn’t have to nap through it.
So clearly, decision making at a senior level has far reaching effects. It affects the life of an agency through
the establishment of policy and protocols, it affects the life of the staff working within the organizations
through the programmes and projects that they will be required to carry out, and this all, without
question, affects the lives of people who receive support.
It doesn’t take much to remember back a few years when the standard practice of many residential based
agencies to disallow the formation of consenting sexual relationships was the norm. Decision making,
then, is different than decision making now. Before the advent of evidence-based practice, opinions and
points of view of individuals in many situations led to policy and practice. As the research grew, as
evidence became more rigorous, past practice was challenged. But it wasn’t just research that lead this
change was it? Other things were beginning to happen. Voices were raised in opposition to these policies,
primarily by people with disabilities who were discovering the power of self advocacy. Clinicians began
providing ‘experiential’ information from what they were seeing in their direct work with people who were
experiencing problems as a result of the policies. Direct support staff were joining in with those they
served to protest unfair restrictions on the right to relationships. Recommendations began to be made
that had growing acceptance into the culture of organizations, in other words, the idea of having policies
that promote healthy relationships began to fit into the organizations’ mindset. Change happened because
decisions were made.
Three Kinds of Evidence
But when we review this bit of past history, it’s easy to see that the traditional thought of ‘evidence-based
practice’ and ‘evidence-based decision making’ took into consideration more than simply research. The
trouble with research is that, no matter how solid it is, for change to happen, it can’t go alone. In fact The
Center for Disease Control and Prevention (2010) suggested that there were three kinds of evidence that
need to be considered when making evidence-based decisions:
Best Available Research Evidence: It’s hard to define this but let’s give it a go – this is ‘the best available
research evidence.’ Surely that is a bit clearer for you. As clear as this is, we need to look at this more
closely later.
Experiential Evidence: Some might quibble at this being here, but one guesses the Center for Disease
Control and Prevention must have some pretty powerful experiential evidence indicating that experience
can be evidence. Here the evidence comes from the experience of the primary professionals involved with
the situation who need to be part of the decision making practice. The expertise and skill gained from the
daily practice of supporting someone with a disability, either as direct support, or from a supervisory role,
or from a clinical role is important and this kind of intuitive understanding and knowledge shouldn’t be
lost.
Volume 1 - Issue 1
2
Contextual Evidence: This one’s a little murkier. This one involves whether or not an idea, a strategy or a
policy would be acceptable to the culture of the organization and the values of the people in it. The
strategy needs to be evaluated on its ‘workability’ in a certain setting or with certain staff. Let’s take
something fairly controversial - contingent electric shock for self-injurious behaviour. There is research
evidence that this approach works. Now comes the first question, is it the ‘best available research’ ... now
comes the second and, ultimately, the most important question: even if it was the best research, is that
practice going to be practical and feasible for administrators, for staff and for people in service? In
answering this question with a resounding ‘no!’ community based organizations in Ontario stated that
there needed to be new evidence about new ways of providing support.
Qualitative and Quantitative Research Approaches
OK, let’s go back and take a closer look at Best Available Research Evidence:

Many of the research studies that focus on human services fall into one of two categories: Qualitative
and Quantitative research. Both provide information but both approach research differently.

Quantitative research looks at numbers, data, and things that can be measured precisely.

Qualitative research looks at data that can be observed, but not precisely measured.

Maybe it’s best to use an analogy (which you may be surprised to know, is a form of evidence).
Look at this picture and then we’ll look at how Qualitative and Quantitative Research would describe what
you are seeing.
Quantitative Research Findings:
Qualitative Research Findings:
Dog
Dog
34 kilos
Brown and golden fur
61 cm tall
Fur primarily straight but with some waves
12 years average life span
Named ‘Big Ben’ because he’s never late for dinner
Produces poop, 3 times per day
Chews shoes, fears squeaky toys
Will drool to a bell if paired with food
Thinks squirrels are demons from hell
Volume 1 - Issue 1
3
While, admittedly, this is a lighthearted look at two important aspects of research, it does point out the
value in each. Depending on the question you are asking, or the decision you are making, one type of
research may be more valuable than the other. However, it may not be possible to be fully informed when
only one kind of research is valued and promoted.
Qualitative Research has been used very successfully with people with intellectual disabilities. A recent
Ontario study looked at the responses of people with disabilities who attended abuse prevention training
sessions with trainers who also had intellectual disabilities. Using qualitative methods it was possible to
capture the ‘sense’ of the responses and the ‘strength’ of the opinions held by those attending. In the field
of disabilities, it’s only recently that research like this has brought the ‘voice’ of people with disabilities
into focus.
Quantitative Research has a long history in providing information based on data that shows whether or not
a certain approach has been effective. The statistics gathered from these studies have radically altered our
understanding of people with intellectual disabilities and their capacity to learn, change and grow. Early
studies demonstrated that, if taught using a systematic approach, people with intellectual disabilities could
learn more than basic life skills. It’s hard to believe that this was ever questioned, but it was. Behavioural
research can take a bow for proving that intellectual disability did not mean a lack of intellect or a lack of
the ability to learn and grow.
So What Does This All Mean?
As we move towards more accountability and more scrutiny, there will be a growth in calls for
organizations to move towards evidence-based decision making. This will ultimately benefit people who
receive care, but it will also place additional demands for those who make decisions within organizations
to keep current with the evidence.
This takes time and will be difficult at first to simply remember to check what the literature says. Google
Scholar, a search engine that culls research, will become a really good friend. But, it will also be important
for those in leadership positions to ask the professionals who consult to their agency, or for their agency,
about their use of clinical information – is it recent; did it take into consideration the experiences, skills
and knowledge of others; does it fit well into the environment – is it practical and usable? To the work that
was done by The Center for Disease Control and Prevention, we’d add one more: Has the individual
receiving support had an adequate opportunity for input, review and consent? (This will be the subject of
an upcoming newsletter.)
How Does the Newsletter Fit In?
I’ll bet you’ve got some questions:



What do you do when there is no available research?
What do you do when the people and families you are serving are advocating for a practice that is not
evidence-based?
How do you compare two contradictory pieces of qualitative empirical data?
Volume 1 - Issue 1
4
Well, don’t worry; this is going to be the subject of another upcoming newsletter. We are going to tackle
the kind of issues that arise from evidence-based decision making.
Who We are and What We’d Like to Achieve
The three editors, Dave Hingsburger, Director of Clinical and Educational Services, Vita Community Living
Services; Jim Hughes, Program Manager, The Centre for Behaviour Health Sciences, and Angie Nethercott,
Senior Behaviour Consultant, Hands TheFamilyHelpNetwork.ca, all work in providing clinical supports to
organizations, staff, and people with disabilities. Executive Functioning will seek to provide evidence-based
(of both kinds … the dog is brown and 61 cm tall …) articles. We will be drawing topics from our
collaborations with senior leadership in a variety of different organizations.
In Closing:
A new newsletter has begun. It will publish three to four times per year and will focus on the needs of
senior leadership. Like Service, Support and Success, the newsletter for direct support professionals, we
want to publish and highlight excellent work that gives practical advice. This introductory article was our
way of announcing the beginning of this new venture and to ask you to think about contributing an article.
If you’ve worked your way through an issue, or developed a new approach to supporting staff and
members or dealing with staff burnout or turn over … or anything … this is the forum to share those
approaches. We will work with you to develop the article and, you can submit your idea to any one of the
editors and we’ll go from there.
References
Cook, M.A. & McVicker, A.P. (2013). Bringing Up the Roof: An evaluation of the Advocates Against Abuse
Project (Executive Summary). Windsor, Ontario: Author.
Davies, P.T. (2004, February). Is Evidence-Based Government Possible? Jerry Lee Lecture, presented at the
4th Annual Campbell Collaboration Colloquium, Washington, DC.
Puddy, R. W. & Wilkins, N. (2011). Understanding Evidence Part 1: Best Available Research Evidence. A
Guide to the Continuum of Evidence of Effectiveness. Atlanta, GA: Centers for Disease Control and
Prevention.
Centre for Behaviour Health Sciences
Volume 1 - Issue 1
5