School leadership under the microscope –
do they know what they are doing?
Univ.-Prof. Dr. Stefan Brauckmann
International Conference on EVALUATION
Erasmus + Project “EVALUATION OF SCHOOL LEADERS AND
TEACHING PRACTICE”
20th-24th March 2017
Bolzano, South Tyrole, Italy
“Father, forgive them, for they do not know what they are doing!”
(Luke 23:34)
Leitmotiv (Leading motif)
• What is the subject of school leadership evaluation?
• What are we doing it for ?
• Based on what kind of criteria
• Who is doing it?
• How is it done?
are we doing it?
Prélude
5
6
Evaluation in my private life
7
Evaluation in my professional life
8
Common understanding of evaluation
• Evaluating an action and/or the consequences of an action
• Evaluation based decisions and/or decisions which might
lead to further evaluation
9
Evaluation as an instrument, in order ….
• to assess organsations or units of organizations
• to provide information for decision making within
organisations
10
But…(un-)fortunately
- It´s complex
- It´s complicated
- It depends
11
Systematic evaluation procedures
… to anchor them in the structure and culture of a governance system
and to use them permanently
Systematic evaluation
procedures
(introduction and
shaping)…
… initiating developmental and learning processes and to sustain them
over time
… creating synergies of different datasets
… use of a theoretical framework which aligns Context-, Input-,
Output- und Process-elements
12
Structuring and optimizing the use of
evaluation
(1) what √
(2) what for
(3) based on what kind of criteria
(4) by whom
(5) how
13
What is
evaluation?
Evaluation tree
Standards and
models
Guiding
questions
Condition
Changes
Impact
Methodology
Evaluationdesigns
Gollwitzer & Jäger, 2014
Instrument
development
Use of data
Data collection,
-structuring, analysis
What exactly do you want to
evaluate?
15
Evaluating school leadership actions
• Achievement of one person?
• Achievement of the school leadership team?
• Achievement of the organisation as a collective actor?
16
Evaluation of school leaders,
what for?
17
Leading functions of evaluation
Knowledge
Control
Evaluation
Development
Stockmann & Meyer, 2014
Legitimation
Relationships and potential conflicts
Contracting
entity
Fears of the evaluated
person
Object of
evaluation
Evaluator
Mental reservations
Drop-outs, Biases
Gollwitzer & Jäger, 2014
Credibility of the evaluator,
lack of support provided by
contracting entity
Evaluation…
Quality
development
assurance
..both sides of a coin!
20
Criteria?
21
Quality of school leadership
Quality leadership
Reference norm
approach
domains
multilayered
normative
empirical
input
process
output
Criterion
Social
Individual
Pashiardis & Brauckmann 2016
22
Who?
23
Who is evaluating school leaders?
Centralized agency
Inspectorates
Businesses
Teachers
Students and/or
parents
24
How?
25
School leadership
External Evaluation
External Evaluation
Evaluation procedures
Communalities
Differences
Quality improvement & quality
assurance
Interests (behind those
procedures) are quite different
Internal and external evaluation
Starting point
Internal Evaluation
External Evaluation
- vague, unstandardized, no clear
roles and responsibilities matrix,
no systematic qualification
-clear, structured, organized,
standardized,
Trained inspectors
28
Complementary effects?
External evaluation
Accountabilty function
1. Unbiased observers
External
Evaluation
2. Focus on the blind spots
3. Can support results of
the external evaluation
Internal evaluation
Impulses for school
development
1. Can deal with
unstandardized
leadership activities
(under the radar)
2. Can help to understand
better the results of
the external
evaluations
Internal
Evaluation
29
Three possible constellations
parallel,
Independent existence
sequential,
Inspectorate is building
on the results of the
internal evaluation
Cooperative,
External evaluators are
designing together with
the school leader(s) a
context-specific
evaluation approach
30
Conceptual framework for teacher appraisal
Evaluation and
Assessment framework
Student
assessment
Procedures
Frequency
Aspects and criteria
Reference standards
Instruments
Goals for the
Education system
Evaluation and
Assessment framework
Governance
Design framework
Purposes
Requirements
Reponsibilities
Funktions
Teacher
appraisal
Evaluation and
Assessment framework
Teacher policies
School leader
appraisal
Capacity
Evaluators
Preparation
Competencies
Use of results
Feedback,Professional
Development
Employment status
Career progression,Rewards
Education system
evaluation
Goals for the
Education system
School
evaluation
Goals for the
Education system
OECD 2013
Goals for the
Education system
31
To sum it up…
Future
directions
• A regional educational management
• Is influencing the contextual predispositions of school
leadership initiatives
• Should carry out meta-evaluations (local monitoring effort)
• Should evaluate its own work
• Developing empirically sound indicators
32
The positive swing…
33
• Thank you!!! Have a safe trip back home!!!
34
35
Formative versus summative Evaluation
Formative Evaluation
Scriven: „When the cook tastes the soup it
is formative evaluation…“
Summative Evaluation
Scriven: „…when the guest tastes the soup
it is summative.“