Sigried Caspar
European Commission,
DG Employment & Social Affairs
Moderator
Workshop A7: how to evaluate
your communication activities
2014-2020
December 10, 2013 – Brussels
Céline Mas
Occurrence est
certifiée
ISO 9001
depuis 2004
"Half the money I spend on
advertising is wasted;
the trouble is I don’t know
which half."
John Wanamaker
Inventor of mass
retailing in the United
States
Occurrence - Workshop A7 - 10/12/2013
Key issue
> Communications are under pressure:
-
How much does it cost?
How much
does it contribute?
€
What
does it contribute?

> View it as an investment and not as an expense
> Provide the resources to prove your effectiveness: 5% to evaluation.
> "How much does overlooked inefficiency cost?“ : if you can not assess it, you can
not improve it!
Occurrence - Workshop A7 - 10/12/2013
4
The missing link in a virtuous circle
Communication plan
Most of the time,
evaluation is occasional
or partial.
A goals and resources « contract »
with institution or company management
It rarely shapes dialogue
between the
communications team
and the other decision
makers.
Assessment
Reporting on the achievement of the targets,
or on the progress and effectiveness
of the implementation
Occurrence - Workshop A7 - 10/12/2013
Actions
A goals and resources « contract »
with company management
Behaving like any other function
> Communications must not exclude
itself from Quality and Operating
Excellence systems
> Communications is a job and a skill;
it must include ongoing improvement
procedures
The Deming Wheel
Quality management system
Occurrence - Workshop A7 - 10/12/2013
6
The 4 main benefits of an evaluation
1. PROMOTING
2. MANAGING
• Circulating results and performances
• Allocating resources in accordance
to other departments/teams
with performance indicators
• Achieving investment choices that are
• Identifying the most effective initiatives
based on targets, and not on expenditures
on resources
for achieving your various objectives
4. SHARING
3. SAVING TIME
• Prioritizing/Sorting initiatives by order
• Gathering all the activity and
of effectiveness
• Concentrating your efforts and budget on the
most effective initiatives
•
effectiveness data
• Highlighting best practices
AND SPECIFIC TO THE PUBLIC
SECTOR
Giving evidence of a sound use of public
money
• Reinforce citizen’s trust
Occurrence - Workshop A7 - 10/12/2013
7
2 main categories of performance (KPI)
indicators
Activity
including
Resources
(What?)
Effectivenes
s
including the
Audiences
(For what
purpose?)
e.g. Number of initiatives/tools,
type of initiatives/tools, assessment of the content issued
(Press releases, and internal communications),
Ressources: Who? How much? How long?
e.g. Memorization, Understanding, Buy-in, Incentive, Transformation,
Satisfaction, Improving the brand's image, and satisfying internal customers
Audience: How many people attended? How many Likes ? How many
readers? …
> A third, highly practical approach is possible: assessing the satisfaction of
(internal) customers
> Defining performance thresholds for each indicator
Occurrence - Workshop A7 - 10/12/2013
8
Communications Dashboard communications plan
Process : global picture
Target audience
Communications objectives
Segmentation


1. Define
2. Count
3. Contact details
Opinions/perceptions to share
Do you agree with the following assertion ?
Surveys, research - "ABC respects environment”
media analysis, - …
Yes
observations
Indicators
-
x 1
[File qualification]
- "ABC respects environment”



60%
Current
score:
60 %
Occurrence - Workshop A7 - 10/12/2013

Targeted
score :
90 %
x N
[Sample survey]
Gap between results and objectives
!!
!
OK
Yes :
No : 
Don’t know :
Management
•
•
•
•
•
Allocate
Understand
Adjust
Maintain
Etc…
Advice
> Efficiency means producing the desired effects on the desired target audiences
> Therefore, you need to define the target audiences that you want to reach with
which effect, prior to the initiative, and ideally to define the performance threshold
> As a starting point: KISS (Keep It Simple, Stupid)
-
Keep things simple at the beginning
> BUT assess them regularly
> And don't change the assessment system for each evaluation
> Do not restrict the evaluation to the activity, in order not to limit communications to
initiatives and tools
> Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools
and goals for each initiative
> Share the results and the decisions they help to take in order to enhance the value
added of evaluation
Occurrence - Workshop A7 - 10/12/2013
10
Thank you
> Contact : Céline Mas
> Partner & Research Director Occurrence
> [email protected]
Occurrence - Workshop A7 - 10/12/2013
Brussels, 9-10 December
Workshop B7: How to evaluate communication
activities 2014-2020
UK Government perspective
December 9, 2013 – Brussels
Paul Njoku
Cabinet Office, UK
Agenda
A
• Why evaluate (context)?
• Barriers to evaluation
B
• How to go about it - PROOF guiding principles
• 4 stage evaluation process - The Big IDIA
• Main performance categories - KPI indicators
C
• Top tips
Brussels, 9-10 December
A1
Why evaluate?
16
A2
•
Context
Austere times


•
Media landscape & consumption patters


•
Need to make every € count
View as an investment not an expense
Evidence of what works and what does not
Optimise use of scarce resources
The role of communications
 How it supports achievement of policy outcomes
 Business planning & activity prioritisation
Brussels, 9-10 December
A3
Strategic alignment
DEPARTMENTAL OBJECTIVE
To address specific issue
Policy development, policy delivery, reputation management
COMMUNICATION OBJECTIVE
Role that communication will play in achieving departmental objective
Overall communication strategy - how communication will achieve its
objective
Sub-objective
Sub-objective
Sub-objective
Communication plan
Specific activities, channels,
target audiences
Communication plan
Specific activities, channels,
target audiences
Communication plan
Specific activities, channels,
target audiences
A4
Barriers – stopping it happening
Lack of SMART or
unrealistic policy
objectives / targets
Culture &
entrenched
behaviours
Insufficient time /
resource / budget
Difficulty accessing
the right data / tools
Gaps in evaluation
standards & capability
19
B1
How to go about it
Five key principles
P
Pragmatic – best available within budget, not best ever
R
Realistic – prove what you can, acknowledge what you can’t
O
Open – record and share as much as possible
O
Objective – be honest & constructive about results, to inform future learning
F
Fully integrated – integral part of planning & delivery, not an add-on
Brussels, 9-10 December
B2
1
Evaluation stages –The Big IDIA
Identify
The scope of your project
2
Develop
Your evaluation plan
3
Implement
Gather data to measure
performance
4
Analyse & report
Performance against plan
Task 1: Define what you need to evaluate by asking:
• What activity am I evaluating?
• What do I know & what factors could affect the outcome?
• What is my evaluation expected to achieve?
Output: Summary of your proposed evaluation approach
Task 2: Define how you’ll measure success:
• Set SMART objectives & defining your target audience
• Map out how activity will work
• Set performance metrics (KPIs) & agreeing targets
Output: Draft evaluation plan
Task 3: Identify and gather evaluation data:
• Make most of existing data
• Gather additional data (research, feedback & proxies)
• Review data gaps (more budget ?) manage expectations
Output: Completed evaluation plan
Task 4: Assess the success of your activity:
• Analyse effectiveness & provide insights for future
• Demonstrating efficiency and value for money
• Demonstrating role of communications in supporting
the achievement
policy objective (outcome)
Brussels,
9-10ofDecember
Output: Final evaluation report
B3 Key performance indicator categories
Activity
Effectiveness
Result
Brussels, 9-10 December
Top tips
1.
Strategic alignment – Ensure activity objectives are SMART
and supports policy delivery.
2.
Business impact – Aim to measure true business impact
(outcome) rather than for example, the perceived quality of
specific channels.
3.
PROOF the big IDIA – Try to adopt the suggested guiding
principles and follow the big IDIA stages.
4.
Continuous improvement – Ensure results drive appropriate
actions and any learnings inform future activities.
5.
Best practice – Be objective, share results and make
evaluation an integral part of your communications planning
process.
Brussels, 9-10 December
Thank you!
Contact: Paul Njoku
Email : [email protected]
Web link to guide: https://gcn.civilservice.gov.uk/guidance/evaluation/
24
Appendix
Brussels, 9-10 December
How reputation is created
Direct Experience
Products
Investments
Customer
Service
Employment
What your department
Says/Does
Branding
Marketing
Public
Relations
Social
Responsibility
What Others Say
MEDIA
(Traditional,
Social)
Topic Experts,
Leaders,
Friends/Family
Perceptions &
expectations
Supportive
Behaviour
Results
Reputation drivers & dimensions
Reputation Strength
A measure of the emotional connection.
Reputation Dimensions
The seven dimensions specify at a more operational level,
which aspects are most important for stakeholders’
perceptions and expectations – i.e. what’s driving a
company’s reputation
Reputation Attributes
The attributes and dimensions have
different meanings and importance for
different stakeholders. Beneath the 7
dimensions, 32 attributes underpin the
individual dimension themes. Different
stakeholder groups typically have unique
attributes that are found more important
than others (reputation drivers).
Supportive Behaviour
Reputation has a positive/negative impact on
support. An increase in reputation = an increase
in support. Support (such as buying products
and services, saying something positive, giving
the benefit of doubt in times of crisis (etc.) leads
to increased business results
Brussels, 9-10 December