Measuring e-Business
Processes
Experiences from the
e-Business W@tch surveys
2005 & 2006
Hannes Selhofer
empirica GmbH
19th Bled eConference – "eValues"
05 June 2006
An initiative of the
European
Commission
19th Bled eConference, 06 June 2006
Hannes Selhofer
1
Table of Contents
Background
Method used by e-Business W@tch
Objectives and focus of the surveys
Challenges in Overview
Challenges in Questionnaire Design
Technical / systems perspective
Functional perspective
Impact perspective
Recommendations
19th Bled eConference, 06 June 2006
Hannes Selhofer
2
e-Business Surveys 2005 & 2006
Survey approach
CATI method
No cut-off by firm size
Since 2005: only computer-using companies
Different industries, with a focus on manufacturing
Field-work outsourced to market research company
Scope
2005: 5,200 interviews, 7 countries
2006: 14,000 interviews, ~30 countries
• Field-work just completed – first results by end of June
Previous surveys: 2002, 2003
19th Bled eConference, 06 June 2006
Hannes Selhofer
3
Survey objectives
Enable informed policy decisions
Test assumptions on current e-business activity
(what is 'state-of-the-art' ?)
Enable comparisons
• by industry / company size / country
Provide data basis for further research activities
Specific questions
Studies on ICT impact
• on firm level / industry level
• using factor analysis & other statistical methods
Ultimately: productivity growth in firms
19th Bled eConference, 06 June 2006
Hannes Selhofer
4
e-Business as "… intra- and inter-firm
processes…" – Activity Based Firm-Theory
"Automated
business
processes (intraand inter-firm over
computer-mediated
networks"
Company
e-Transactions
e-Commerce
e-Business
e-Interactions
Government
19th Bled eConference, 06 June 2006
Consumers
Hannes Selhofer
c-Commerce
5
Challenges to cope with (1/3)
Questionnaire design
Manufacturing perspective not 100% applicable to
other sectors
• Construction, tourism, health, public sector, …
Use of different instruments too costly
Small enterprise or large enterprise perspective?
Population and sampling
Adequate definition of the population
• Cut-offs (e.g. computer / internet using companies, by size)
• Sector definitions
Reliability of business directories for sampling
19th Bled eConference, 06 June 2006
Hannes Selhofer
6
Challenges to cope with (2/3)
Telephone interviews: natural limitations
Time constraint (20 min. maximum)
No change of target person
Questionnaire: breadth vs. depth
Increasing refusal rate
• ICT professional over-researched
• "No-interview" policy in many large firms
ICT outsourcing
Limited information about technical issues
No access to background data
Turnover, profit, …
19th Bled eConference, 06 June 2006
Hannes Selhofer
7
Challenges to cope with (3/3)
Data presentation
The weighting issue
• By share of employment
• In % of enterprises
Use of compound indices
The benchmarking dilemma
• the more ICT the better … ?
19th Bled eConference, 06 June 2006
Hannes Selhofer
8
Questionnaire design:
different types of questions
Technologies & systems perspective
Hardware and software systems used
Communication networks used
Functional perspective
Which processes are ICT supported and how?
Amount of e-transactions?
Impact perspective
What is the effect of this activity?
What has changed in the company?
19th Bled eConference, 06 June 2006
Hannes Selhofer
9
1. Technologies and systems perspective
Examples
Advantages
Type of internet access
ERP use
CRM use
SCM use
EDI, XML & other estandards used
RFID use
W-LAN use
Security measures used (e.g.
secure server technology)
19th Bled eConference, 06 June 2006
Hannes Selhofer
+ Mostly easy to ask and to
understand
+ Time efficient in telephone
interviews
+ Includes important 'basics'
Challenges
- Does not tell us what the
system is actually used for
- Bias towards large firms –
small firms use work-around
solutions
- Gets complicated if different
systems are being used
10
2. Functional perspective (i)
Examples
B9: "Does your company enable direct customer booking
processes by means of ICT which were previously dealt with
by intermediaries such as travel agents?"
A5: "Can employees of your company access your computer
system remotely from outside the company?"
D5f: "Does your company send e-invoices to customers in
the public sector?"
• D6: "Please estimate the percentage of invoices your company sends
as e-invoices."
E8: "Which of the following sourcing or procurement related
processes does your company support by specific ICT
solutions?" … (c) order goods or services …
F13: "Is your company's ICT system linked to the ICT system
of (a) suppliers / (b) customers?"
19th Bled eConference, 06 June 2006
Hannes Selhofer
11
2. Functional perspective (ii)
Advantages
+ More informative about actual activity in firms
+ More flexible than questions on systems
+ Can better be adapted to smaller firms
Challenges
+ Some concepts difficult to paraphrase
+ Different understanding of what is included or not
(e.g. "collaborative design", "e-invoice")
+ Many decisions involved on how broad / narrow a
question should be
+ More time consuming to ask – 'opportunity cost' in
interviews
19th Bled eConference, 06 June 2006
Hannes Selhofer
12
3. Impact perspective (i)
'Objective' impact:
Correlation analysis of e-business activity with firm
performance (if data available)
Perceived impact
Direct questions on quantitative or qualitative effects
Examples
H4: "In what ways have ICT influenced the business of your
company? (…) for each of the following whether ICT has had
a positive, a negative or no influence at all: (a) revenue
growth, (b) the efficiency of business processes, (c) …
H8: "Do you expect that ICT will have a high, medium, low or
no impact on the following … in your company in the future:"
(a) …, (b) …
19th Bled eConference, 06 June 2006
Hannes Selhofer
13
3. Impact perspective (ii)
'Objective' impact
Perceived impact
Advantages
Advantages
+ Allows to quantify impact
+ Use of established,
accepted statistical
methods
+ Can build on substantial
research record
+ Data can be directly
collected
+ Quick barometer for the
overall experience
Challenges
+ Data availability
+ Bias against confirming
negative experience
+ Possibly a cultural bias
+ 'Pseudo-objectivity'?
+ Validity of results: cause
and effect?
+ Quantification of results
hardly possible – no exact
measurement
Challenges
19th Bled eConference, 06 June 2006
Hannes Selhofer
14
Recommendations
Consider ways of using specific instruments for
non-manufacturing industries
Combination of different perspectives in
questionnaire design
Beyond EU: international dimension
It's not just the figures: combination of methods
Case study approach
• Helps to understand concepts and explain figures
• Adds value for users in policy and industry
Context and assessment in reporting results!
19th Bled eConference, 06 June 2006
Hannes Selhofer
15
More information
Firms selling goods/services online
(1995 – 2004)
50
Publishing
Tourism
Pharma
Food
Construction
40
30
www.ebusiness-watch.org
[email protected]
20
10
0
<95 95
96
97 98
99
00 01
02
03
04
[email protected]
Data weighted by employment. Base: EU-7 (CZ, DE, ES, FR, IT, PL, UK).
Based on questions about the starting point of buying/selling online.
Source: e-Business W@tch (Survey 2005)
19th Bled eConference, 06 June 2006
Hannes Selhofer
16