Process Use:
Intentional Practice or Just Good
Practice?
anzea 2013 Conference
22–24 July 2013
Alexandra Park, Epsom, Auckland
Michael Blewden
Massey University
Overview
• Background
• Research question/approach
• Case study of findings
• Implications for practice
Process Use
• Learning and development from
stakeholder participation in evaluation
• Influence/consequence of evaluation
processes
• Distinct from/independent of findings use
e.g. Stakeholder participation…
…enhances willingness to use findings
…develops evaluative thinking or action
…develops shared understanding
…influences the evaluand
Patton (1997) says:
“Evidence of process use is represented by the
following kind of statement after an evaluation:
“The impact on our program came not just from
the findings but also from going through the
thinking process that the evaluation required”
Shaping this study
• Process use:
- enhances value and utility
- often an unintentional side-product
- more likely if we purposefully seek it
Patton (1997) again
“...the possibility and desirability of learning from
evaluation processes as well as findings can be
made intentional and purposeful…”
“…instead of treating process use as an informal
offshoot, explicit and up-front attention to the
potential impacts of evaluation logic and processes
can increase those impacts and make them a
planned purpose for undertaking the evaluation…”
The question of ‘intent’
Patton infers evaluators may
• choose to deliberately seek process use
• adopt specific practices to achieve it
• increase the value and utility of evaluation
The question of ‘intent’
• …historically, process use more typically
regarded as “...an informal offshoot”
(Patton, 2007)
• …few methodologies intentionally seek
process use…rarely an integrated goal of
practice (Morabito, 2002)
• Observations of NZ practice and context
The (research) question
• Why and for what purpose do evaluators
seek process use?
• Why do they choose the practices they do
to achieve it?
Research Approach
• Process use as ‘sensitising concept’
• Process use as a ‘construction’
• Interpretivist explanation
• Importance of context
Interpretivist explanation
Meaning + beliefs + desires = behaviour
e.g. action X was done because person held
belief Y according to which doing X would
fulfil desire Z
Journey to practice
Practice setting
Traditions
Beliefs about evaluation
Evaluation
practice
Values
Beliefs
about
outcomes
Evaluation theory
Process use
intent and
practice
Beliefs about
role
Beliefs explaining
beliefs
Beliefs about
practice
Project setting
Cultural context
Meaning
Why
important
Embedded
‘rules’
Intent and
practice is
‘understandable’
Justifications,
reasons
Point and
purpose
Expectancies
General
awareness
and
experience
of process
use
Process
use
examples
considered
important
and
intentional
Participants
• 24 practicing evaluators
• Eligibility criteria
• In-depth face to face interviews
• Auckland and Wellington location
Assumptions
• Desirable for evaluators to seek process
use but not necessarily always
• Pursuing process use may have risks
• Understanding, use, relevance or
appropriateness of the term not assumed
Evaluation
as process
Evaluation
as
capacity
building
Evaluation
as
development
Evaluation
as findings
use
Beliefs about evaluation
Social
betterment
Evaluation as
intervention
Intent and practice
Enabling
Equality
Knowledge is
experiential
and
constructed
Beliefs about practice
Collaborative, transparent,
understandable, trustful
Accountable to
relational ethics
and morals
Intent and practice
Should address
issues of power and
inclusion
Tools and
procedures
as learning
Beliefs about role
Should facilitate mutual learning,
development, improvement
Intent regarding
process impacts
Intent and practice
Should act in the interests of
those with less power
Responsibility to give
back/return value
Beliefs about outcomes
Accept evaluative
conclusions/findings
Data quality
Critical
engagement
Intent and practice
Equality
Attitudinal/affective
change
Capacity development
and learning outcomes
Explanations
• Findings are ‘ideographic’ - however….
• PU integral and inevitable
• Intent/practice understandable when
evaluators are understood as thinking,
‘meaning makers’
• Enhancing process use about debating the
evaluator’s mandate, role, responsibility
Reflections
• Do these evaluator beliefs have
implications – positive or negative?
• Are there risks to evaluation?
• Could there be process use misuse?
• How should we respond?