Monitoring public satisfaction through
user satisfaction surveys
Committee for the Coordination
of Statistical Activities
Helsinki
6-7 May 2010
Steve . Clarke @ ec . europa . eu
Structure of the presentation
 Introduction
 Background information on user satisfaction surveys in the ESS
 Summary of information on user satisfaction surveys
 Eurostat’s experience
 Issues for international statistical organisations
6 May 2010
2
Introduction
How are international organisations ensuring public satisfaction?
 Satisfaction with what?
 What public?
 What level of aggregation?
6 May 2010
3
Introduction
Satisfaction with what?
6 May 2010

Published statistics?

Development and use of statistics for policy purposes?

The way data are collected and compiled?

The way data are kept secure?

The integrity of the official statistical institutions?
4
Introduction
What public?
6 May 2010

All citizens?

All users?

Key users?

Stakeholders?
5
Introduction
What level of aggregation?
6 May 2010

Individual

Local communities

Specific interest groups (culture, lifestyle)

Regional

National / International
6
Introduction
Eurostat experience with user satisfaction surveys to measure public
satisfaction, in the following sense:
6 May 2010

Published statistics

Users

Regional / National

Follows the Code of Practice (Principle 11)
7
Introduction
Code of Practice Principle 11: Relevance
6 May 2010

European Statistics must meet the needs of users

Users are consulted about their needs

Users are asked for feedback
8
Background
Eurostat user satisfaction surveys

General surveys in 2007 and 2009

Domain specific surveys from 2007 onwards
Member State user satisfaction surveys

6 May 2010
Eurostat overview in 2008
9
Eurostat general user satisfaction survey 2009

Based on an agreed model questionnaire for the ESS

Users views on the quality of the statistics and on the way
they are published

Carried out via the Internet, open to Eurostat web site users
6 May 2010
10
Eurostat general user satisfaction survey 2009

Over 1400 replies: 37% from students, academic and private
users

Identified areas for improvement (metadata, comparability,
quality control, easier data access)

Full report on Eurostat web site (quality pages)
6 May 2010
11
Eurostat domain specific surveys

Component of the Eurostat Quality Assurance Framework

To assess broader statistical production processes

Prices, Public Health, Transport,..

User satisfaction survey customised to the particular process

Provide valuable input for evaluating a statistical process
6 May 2010
12
National user satisfaction surveys
Summary information (Quality in Statistics Working Group 2009)
6 May 2010

Timing, frequency and regularity

Target population

Type of satisfaction survey

Data collection modes

Themes and domains covered
13
National user satisfaction surveys
Timing, frequency and regularity
6 May 2010

Prompted by the Peer Reviews

Implemented regularly in a number of countries

Typically every 2 years
14
National user satisfaction surveys
Target population
6 May 2010

Expert users

Internet users

Registered users

Official statistics users

Varying response rates
15
National user satisfaction surveys
Type of satisfaction survey

General user satisfaction surveys

Web-user surveys

Image surveys

Domain-specific

More focused / specialised user surveys (business
community)
6 May 2010
16
National user satisfaction surveys
Data collection modes
6 May 2010

Computer-assisted web questionnaire

Emails / paper questionnaires

Telephone / face-to-face
17
National user satisfaction surveys
Themes and domains covered
6 May 2010

Quality of the data

Quality of the products and services

User characteristics

Variety of question styles
18
Conclusions (1)
How are international organisations ensuring public satisfaction?
 By promoting user satisfaction surveys in countries (Code of
Practice)
 By doing own their own international user satisfaction surveys
 By regularly monitoring the implementation of national user
satisfaction surveys
6 May 2010
19
Conclusions (2)
How are international organisations ensuring public satisfaction?
 Providing a forum for developing and exchanging good practices
 Contacting user representatives (e.g. ESAC)
 Identifying common themes relating to data quality and service
quality
6 May 2010
20
Conclusions (3)
By engaging in a dialogue for further development…
6 May 2010

questionnaire design

move from “users” to the “public”?

extend consideration of specialised / minority groups?
21
Conclusions (4)
 Going beyond satisfaction with “published statistics”?
Satisfaction with …
- Availability of statistics within and across the different policy
domains?
-
The way data are collected (burden) and used (confidentiality)?
-
The official statistical authorities (independent, honest,
accountable)?
6 May 2010
22
Conclusions (5)
By showing they add value to national data…
 Presenting national data together for comparison
 Respecting the different quality dimensions
 Highlighting data differences and their implications
The End
Steve . Clarke @ ec . Europa . eu
6 May 2010
23