Community
Innovation
Survey 2008
(CIS 2008)
Excerpts from the Quality Report for
BELGIUM
8
Reference period
Observation period
Person who filled the report
Date
2008
2006 - 2008
Jeoffrey MALEK
MANSOUR
1 OVERVIEW
The purpose of this report is to get an overview of the quality of the sixth Community Innovation Survey 2008
(CIS 2008) carried out in each member state. The quality report is to be established for the CIS 2008. The same is
also envisaged for subsequent Community Innovation Surveys.
This quality assessment will be based on different quality dimensions and indicators. The quality dimensions are
based on the standard ones as defined in the Eurostat standard statistical quality framework. Also the indicators
themselves are in line with these recommendations. Indeed, the criteria to judge statistical quality will correspond
to a specific chapter in the report. These criteria are: Relevance, Accuracy, Timeliness and Punctuality,
Accessibility and Clarity, Comparability, Coherence and Cost and Burden. In addition each report should contain a
short methodological description of the national methodology used for the CIS 2008.
2
2 SHORT DESCRIPTION OF THE NATIONAL CIS 2008
METHODOLOGY USED
2.1 Target population
NACE Rev.2
In accordance with annex IV of the Commission Regulation No. 973/2007 on innovation statistics, the following
industries are included in the core target population of the CIS 2008:
-
mining and quarrying (NACE 05-09)
manufacturing (NACE 10-33)
electricity, gas steam and air conditioning supply (NACE 35)
water supply; sewerage, waste management and remediation activities (NACE 36-39)
wholesale trade, except of motor vehicles and motorcycles (NACE 46)
transportation and storage (NACE 49-53)
publishing activities (NACE 58)
telecommunications (NACE 61)
computer programming, consultancy and related activities (NACE 62)
information services activities (NACE 63)
financial and insurance activities (NACE 64-66)
architectural and engineering activities; technical testing and analysis (NACE 71)
Please indicate if there were some national particularities hereon (e.g. more detailed breakdowns or also NACE
activities not covered):
Comments: We believe it is useful to include NACE 72 (Research and Development), as a number of enterprises
transfer their R&D activities to legally separated subsidiaries, that are – by definition – classified into the NACE
72sector. By not surveying those firms, we are likely to miss a part of the picture.
Size-classes
According to the Commission Regulation No. 1450/2004 on the production and development of Community
statistics on innovation the statistics by size class are in general to be broken down into the following size classes:
 10 - 49 employees
 50 - 249 employees
 250 + employees
Please indicate if there were some national particularities.
Comments: There were no deviations
3
Statistical units
The main statistical unit for CIS 2008 is the enterprise, as defined in the Council Regulation 696/1993 on statistical
units or as defined in the national statistical business register. EU Regulation 2186/1993 requires that Member
States set up and maintain a register of enterprises, as well as associated legal units and local units.
Please indicate if there were some deviations.
Comments: : There were no deviations (firms were sampled at the level of the VAT number).
The observation and reference periods
The observation period to be covered by the survey is 2006 - 2008 inclusive i.e. the three-year period from the
beginning of 2006 to the end of 2008. The reference period of the CIS 2008 is the year 2008.
Please indicate if there were some deviations from these rules (in particular a higher national frequency for the
surveys or the production of results).
Comments: : There were no deviations
2.2 Survey type
Data are collected through a census, sample survey or a combination of both.
Please indicate the survey type used.
Comments: We used a combination of survey and census, depending on the size of the population in the various
strata, so as to make sure to match Eurostat’s quality criteria (see Section 2.4 on sampling design for more
details).
If no sampling is used, proceed to section 2.9.
Panel samples, even if constant during several years are samples. Therefore, if you use panel samples please
answer the questions which are relevant to sampling.
2.3 Combination of sample survey and census data
Please indicate the population classes which are covered by sampling and those which are covered by complete
enumeration (census) if applicable.
4
For the Brussels Region (BE1):
In the Brussels Region, a census was performed in all non-emply strata. The following Table gives an overview of
the cells that were actually surveyed.
SIZECLASS
1
2
3
8 CENSUS CENSUS CENSUS
10 CENSUS CENSUS CENSUS
11
.
CENSUS CENSUS
12
.
CENSUS
.
13 CENSUS
.
.
14 CENSUS CENSUS CENSUS
15 CENSUS CENSUS
.
16 CENSUS CENSUS
.
17 CENSUS CENSUS CENSUS
18 CENSUS CENSUS
.
19
.
.
.
20 CENSUS CENSUS CENSUS
21 CENSUS CENSUS CENSUS
22 CENSUS CENSUS
.
23 CENSUS CENSUS CENSUS
24 CENSUS CENSUS CENSUS
25 CENSUS CENSUS CENSUS
26 CENSUS CENSUS
.
27 CENSUS CENSUS CENSUS
28 CENSUS CENSUS CENSUS
29 CENSUS CENSUS CENSUS
30
.
.
CENSUS
31 CENSUS
.
.
32 CENSUS CENSUS
.
33 CENSUS CENSUS CENSUS
35 CENSUS CENSUS CENSUS
36 CENSUS
.
.
37 CENSUS
.
.
38 CENSUS CENSUS CENSUS
39 CENSUS
.
.
46 CENSUS CENSUS CENSUS
49 CENSUS CENSUS CENSUS
50
.
.
.
51 CENSUS CENSUS
.
52 CENSUS CENSUS CENSUS
53 CENSUS CENSUS CENSUS
58 CENSUS CENSUS CENSUS
61 CENSUS CENSUS CENSUS
62 CENSUS CENSUS CENSUS
63 CENSUS CENSUS CENSUS
64 CENSUS CENSUS CENSUS
65 CENSUS CENSUS CENSUS
66 CENSUS CENSUS CENSUS
71 CENSUS CENSUS CENSUS
72 CENSUS CENSUS CENSUS
Note:
a " . " indicates there were no
firms in the frame population
NACE2
NOTE: Sector 46 was initially supposed to be covered by sampling. However, in order to match Eurostat’s quality
criteria, it was decided to run a census as well..
5
For the Walloon Region (BE3)
In the Walloon Region, a mix of census and sampling was used, according to the description in the Table below:
SIZECLASS
1
2
3
8 CENSUS CENSUS CENSUS
9 CENSUS
.
.
10 SAMPLE CENSUS CENSUS
11 CENSUS CENSUS CENSUS
12 CENSUS
.
.
13 CENSUS CENSUS
.
14 CENSUS
.
.
15 CENSUS CENSUS
.
16 SAMPLE CENSUS CENSUS
17 CENSUS CENSUS CENSUS
18 SAMPLE CENSUS CENSUS
19
.
.
.
20 SAMPLE CENSUS CENSUS
21 CENSUS CENSUS CENSUS
22 SAMPLE CENSUS CENSUS
23 SAMPLE CENSUS CENSUS
24 CENSUS CENSUS CENSUS
25 SAMPLE CENSUS CENSUS
26 CENSUS CENSUS CENSUS
27 CENSUS CENSUS CENSUS
28 SAMPLE CENSUS CENSUS
29 CENSUS CENSUS CENSUS
30 CENSUS CENSUS CENSUS
31 SAMPLE CENSUS
.
32 CENSUS CENSUS
.
33 CENSUS CENSUS CENSUS
35 CENSUS
.
.
36
.
.
CENSUS
37 CENSUS CENSUS
.
38 SAMPLE CENSUS CENSUS
39 CENSUS CENSUS
.
46 SAMPLE CENSUS CENSUS
49 SAMPLE CENSUS CENSUS
50
.
CENSUS
.
51
.
.
CENSUS
52 SAMPLE CENSUS CENSUS
53 CENSUS
.
.
58 CENSUS CENSUS
.
61 CENSUS CENSUS CENSUS
62 SAMPLE CENSUS
.
63 CENSUS CENSUS CENSUS
64 CENSUS CENSUS CENSUS
65 CENSUS CENSUS CENSUS
66 CENSUS CENSUS CENSUS
71 SAMPLE CENSUS
.
72 CENSUS CENSUS CENSUS
Note:
a " . " indicates there were no
firms in the frame population
NACE2
6
For the Flanders Region (BE2)
See Section 2.4 on Sampling Design
2.4 Sampling design
Please describe the sampling and allocation scheme (number of strata, number of samples…) of the CIS 2008 used.
Comments:
Belgium is composed of 3 Regions (at NUTS 1 level): Flanders, Brussels and Wallonia. Each Region is endowed
with its own statistical office. Therefore, three separate samples were drawn.
For the Brussels Region:
The problem of NUTS 2 allocation is irrelevant since Brussels is a City-Region and has no NUTS 2 subcomponents.
As to size classes, we did not include firms with less than 10 employees, so we had 3 size classes: [10-49
employees], [50-249 employees ], [250 employees and more].
We used all “core nace” sectors, plus NACE 72 (“Research and Development”). We stratified according to
Regulation 1450/2004. That is, we stratified our sample over sectors: NACE 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 35, 36, 37, 38, 39, 46,49, 50, 51, 52, 53, 58, 61, 62, 63, 64,
65, 66, 71, 72 That makes 46 sector classes.
Therefore, in total, we have 3 x 46 = 138 strata.
For the Walloon Region
A global sampling was done at NUTS 1 level, according to the 3 size classes and the core NACE sectors (at 2digits level), plus Nace 72, just as in the Brussels Region. The number of firms to be sampled was then
proportionally allocated to the 5 NUTS 2 regions within the Walloon Region (BE31 to BE 35).
For the Flanders Region
Besides firm size and sector a third stratification variable was taken into account for sampling in the Flanders
region, i.e. whether or not a firm was known to have continuous R&D spending. The inventory of firms with
continuous R&D spending as obtained from the 2008 R&D survey was used as a base for this variable.
Census sampling was done for all large size firms (250 or more employees), for all medium size firms (50-249
employees) and for small size firms (10-49 employees) of NACE 20-22, 26-30, 61-63, and 71-72. .Census
sampling was also done of the small size firms known to have continuous R&D spending in the other core NACE
sectors. .
For the remaining small size firms first sampling rates were set that would meet the Eurostat precision criteria for
NACE sectors grouped according to their technology level: low-tech industry (5-19, 23-25, and 31-39) versus lowtech services (NACE 46,49-53, 58, and 64-66). These were then applied proportionally to each of the NACE
sectors belonging to those technology level groupings, as well as to each NUTS 2 (province) level grouping within
each NACE sector. The NACE sectors considered were at the 2-digit level (divisions). Cells that consisted of 15 or
fewer firms were fully included in the sample (exhaustive sampling
7
2.5 Sampling frames
Please describe the sampling frame used (e.g. the national business register or other national frames).
Comments
Due to confidentiality constraints the official Belgian business register could not be used. Instead, we
used as frame population the register available from the Belgian National Social Security Office that
contains all active employers in Belgium. This official register is at the enterprise level. We used its
December 2008 version. This register was agreed upon by Statistics Belgium as being statistically
equivalent to the official business register
2.6 Sample size
Please describe the national sample sizes used for the regular CIS surveys:
Comments
The national realized Belgian sample counts n=3427 firms, with the following repartition:
Brussels: n = 708
Flanders: n = 2200
Walloon Region: n = 516
2.7 Overall sample rate
The overall sample rate is the ratio of realized sample size over population size.
Please indicate the overall sample rate for CIS 2008 if available.
For the Brussels Region: 42.6%
For the Flemish Region: 21.6%
For the Walloon Region: 15.9%
2.8 Weights calculation method (short description) – only for sample surveys
Please describe the weight calculation method.
Comments: The weights were computed separately in each region.
For the Brussels Region as well as for the Walloon Region, they are simply the inverses of the realized (ex-post)
sampling fractions.
For the Flanders Region: A non-response adjustment was done to the basic weights Nh/nh using calibration. The
program g-CALIB 2.0 available from Statistics Belgium was used.
Please indicate the data source used for deriving population totals (universe description).
Comments: Flanders region: pooling of non-response survey results and regular survey results.
………………………………………………………………………………………………………………………………………
Please indicate the variables used for weighting.
Comments: Flanders region: innoact (product innovation, process innovation and/or ongoing or abandoned
innovation)
Please describe the calibration method and the software used.
Comments:
8
Flanders Region , the program g-CALIB 2.0 available from Statistics Belgium was used. A linear calibration
function was applied
2.9 Data collection method
Please indicate the data collection method used (e.g. face-to-face interview, telephone interview, postal, other).
Please specify whether interview data collection was computer-assisted or not.
Data were collected both through a mail survey and an electronic questionnaire: firms were sent a paper booklet
by regular mail and could opt to answer either on paper or online. The data collection was not computer assisted.
2.10 Transmission
CIS 2008 data will be transmitted to Eurostat via EDAMIS using the following consignment: CIS_CIS_32. This
safe, secure procedure guarantees a method of tracking transmission. All necessary steps will be taken to ensure
that the EDAMIS system is working at national level.
Please indicate if there are some deviations.
Comments: Tabulated data were transmitted using eDAMIS.
2.11 Overall assessment of national methodology
Please give a short overall assessment of the quality of the CIS methodology. Indicate perceived strength and
weaknesses and describe any plans, and their schedule, you have for relevant improvements.
Comments: Firms in the services sector had a harder time to fill in the questionnaire, as the wording at first sight
seems more applicable to firms in manufacturing. This is especially true for firms in NACE 46, wholesale, where
many respondents commented to us that they thought that the survey was not relevant for them.
Small firms also complained about the length of the survey.
9
4 ACCURACY
4.1 Introduction
Accuracy in the statistical sense denotes the closeness of computations or estimates to the exact or true values.
Statistics are not equal with the true values because of variability (the statistics change from implementation to
implementation of the survey due to random effects) and bias (the average of the possible values of the statistics
from implementation to implementation is not equal to the true value due to systematic effects).
Several types of error occur during the survey process which comprises the error of the statistics (their bias and
variability). A typology of errors has been adopted:
1. Sampling errors. These only affect sample survey; they are simply due to the fact that only a subset of the
population, usually randomly selected, is enumerated.
2. Non-sampling errors. Non-sampling errors affect sample surveys and complete enumerations alike and
comprise:
a)
b)
c)
d)
e)
Coverage errors,
Measurement errors,
Processing errors,
Non response errors and
Model assumption errors.
4.2 Sampling errors
The aim of this sub-chapter is to measure the sampling errors for CIS 2008 data. The main indicator used is the
coefficient of variation (CV).
Definition of coefficient of variation:
Coefficient of Variation= (Square root of the estimate of the sampling variance) / (Estimated value)
Please provide the coefficient of variation as a percentage (%) in the following table.
Table 4.1: Coefficient of variation (%) for key variables by NACE and size (cf. Annex 10.1)
NACE
Total NACE
B_C_D_E
NACE sections H and K, and
NACE divisions 46, 58, 61, 62,
63 and 71
Breakdown
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
Total industry (excluding construction)
Total
Core Services
1
2
3
4
2.00
2.65
2.11
2.57
4.01
5.34
5.36
5.91
14.21
19.14
18.97
20.34
3.71
5.24
4.26
4.38
8.23
14.24
15.10
12.29
2.30
4.56
21.47
4.20
9.73
Total
3.18
6.59
16.91
6.02
[1] = Coefficient of variation for the percentage of innovating enterprises.
[2] = Coefficient of variation for the percentage of innovators that introduced new or improved products to the market.
[3] = Coefficient of variation for the turnover of new or improved products, as a percentage of total turnover.
[4] = Coefficient of variation for percentage of innovation active enterprises involved in innovation cooperation.
[5] = Coefficient of variation for total turnover per employee.
5
13.33
Comments: We have no particular comment.
Variance Estimation Method
10
Please indicate the method used for variance estimation including whether the sample design (e.g. clustering) and
weighting has been taken into account.
Comments: Variances were estimated using estimated linearized standards errors and taking into account the
sampling scheme, but not the fact that imputations were made for missing values (the weights used were those
provided by the weightnr variable ).
4.3 Non-sampling errors
Non-sampling errors occur in all phases of a survey. They add to the sampling errors (if present) and contribute to
decreasing overall accuracy. It is important to assess their relative weight in the total error and devote appropriate
resources for their control and assessment.
4.3.1 Coverage errors
Coverage errors (or frame errors) are due to divergences between the target population and the frame population.
Comments: There may be some undercoverage of recently founded firms due to the fact that the National Social
Security Office Employer database is based on information from the previous year. This is unavoidable, however,
given the delay in information available from these firms.
Please indicate whether the CVs reported in table 4.1 incorporate the effects of coverage errors.
Comments: No, coverage errors are not included in the CVs.
Miss-classification rate
The indicator measures the percentage of enterprises that changed stratum between the time the frame was last
updated and the time the survey was carried out. It is defined as the number of enterprises that changed stratum
divided by the number of enterprises which belong to the stratum, according to the frame. The rate can be
estimated based on the characteristics of the surveyed enterprises.
Table 4.2: Frame misclassification rate by size class
Number of employees
10-49
Number or surveyed enterprises in the stratum
(according to frame)
Number of surveyed enterprises that have
changed stratum (after inspection of their
characteristics)
Miss-classification rate
2122
50 - 249
965
249+
340
TOTAL
3427
216
67
17
300
10.2%
6.9%
5.0%
8.8%
Comments: We have no particular comments
4.3.2 Measurement errors
Measurement errors occur during data collection and generate bias by recording values different than the true ones.
The survey questionnaire used for data collection may have led to the recording of wrong values, or there may be
respondent or interviewer bias.
Please describe those errors if existing.
Comments: We have no particular comments
11
Please describe measures taken for reducing measurement errors (i.e. minimum standards for interviewer
experience, training, questionnaire testing etc.).
Comments: We have no particular comments
4.3.3 Processing errors
Between data collection and the beginning of statistical analysis on the base of the statistics produced, data must
undergo a certain processing: coding, data entry, data editing, imputation, etc. Errors introduced at these stages are
called processing errors. Data editing identifies inconsistencies in the data which usually represent errors.
Please indicate data entry method (data keying, scanning/OCR, CAPI, CATI) and respective error estimates, if
available.
Comments: In the three Belgian Regions, we provide respondents both with a paper version of the questionnaire
and a login/password to an electronic version of the questionnaire. In any case, a number of checks are performed
in real time when encoding the answer. In case of inconsistency or coding error, we re-contact the respondents as
soon as soon as possible and, if needed, provide them with some guidance. As these errors were corrected in real
time, at the time of encoding, we cannot estimate how important they were.
Besides, in the Flanders region, paper responses are entered manually in the database, using the interface for the
online survey. The responses are entered twice and then compared to each other, to detect transmission errors.
Correcting these errors usually takes quite some time. The online questionnaire is also extensively tested by at
least five persons before it is made operational, to detect any errors in programming
12
Please describe the editing process and method (give the editing rates 1if possible).
Comments: To edit the data, we used the SAS program provided by Eurostat in the context of CIS IV. We updated
it to fit the current version of CIS. The editing routine of this SAS application outputs coding errors and
inconsistencies between two or more variables. We also made use of a number of manual checks to verify the
consistency of quantitative data and corrected whenever possible by recontacting the firms, checking their
websites, or using available accounting data.
4.3.4 Non-response errors
Non response is when a survey failed to collect data on all survey variables from all the population units
designated for data collection in a sample or complete enumeration.


Unit non-response which occurs when no data (or so little as to be unusable) are collected about a
designated population unit.
Item non-response which occurs when only data on some, but not all survey variables are collected for a
designated population unit
The extent of response (and accordingly of non response) is measured with response rates.
1 Failure rates of edits are useful, as indicators of the quality of the original data (prior to correction). Editing rates for key variables should be
reported. They may be higher due to measurement error (for instance, because of poor question wording) or because of processing error (for
instance, data capture errors).
13
4.3.4.1 Unit response rate
In this part, the main interest is to judge if the response from the target population was satisfying by computing the
weighted and un-weighted response rate.
Definition:
Un-weighted Unit Response Rate= (Number of units with a response) / (Total number of eligible 2 and
unknown eligibility units in the sample)
Definition:
Weighted Unit Response Rate = (Total weighted responding units) / (Total weighted number of eligible /
unknown eligibility units in the sample)
Table 4.3: Un-weighted and weighted unit response rate
For the Brussels Region:
NACE
Breakdown
[1]
[2]
[3]
[4]
Total NACE
Total
710
1668
42.6%
42.6%
Small [10-49]
496
1164
42.6%
42.6%
Medium-sized [50-249]
153
365
41.9%
41.9%
61
139
43.9%
43.9%
146
348
42%
42%
342
825
41.5%
41.5%
Large [> 249]
B_C_D_E
NACE sections H and K, and
NACE divisions 46, 58, 61, 62,
63 and 71
Total industry (excluding
construction)
Total
Core Services
Total
[1] = Number of units with a response in the realised sample
[2] = Total number of units in the sample
[3] = Un-weighted unit response rate
[4] = Weighted unit response rate
2 Eligible are the sample units which indeed belong to the target population. Frame imperfections always leave the possibility
that some sampled units may not belong to the target population. Moreover, when there is no contact with sample units and no
other way to establish their eligibility they are characterised as ‘unknown eligibility units’
14
For theFlanders Region:
NACE
Total NACE
B_C_D_E (05-39)
NACE sections H and K, and
NACE divisions 46, 58, 61, 62, 63
and 71
Breakdown
[1]
[2]
[3]
[4]
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
Total industry (excluding
construction)
Total
Core Services
2184
1267
664
253
4940
2845
1745
350
44%
45%
38%
72%
41%
40%
40%
72%
1266
2800
45%
42%
Total
918
2140
43%
40%
[1] = Number of units with a response in the realised sample
[2] = Total number of units in the sample
[3] = Un-weighted unit response rate
[4] = Weighted unit response rate
For the Walloon Region
NACE
Breakdown
[1]
[2]
[3]
[4]
Total NACE
Total
516
1997
25.8%
26.2%
Small [10-49]
355
1368
26%
26.4%
Medium-sized [50-249]
138
516
26.7%
26.7%
23
113
20.4%
20.4%
311
1236
25.2%
24.7%
200
735
27.2%
27.9%
Large [> 249]
B_C_D_E
NACE sections H and K, and
NACE divisions 46, 58, 61, 62,
63 and 71
Total industry (excluding
construction)
Total
Core Services
Total
[1] = Number of units with a response in the realised sample
[2] = Total number of units in the sample
[3] = Un-weighted unit response rate
[4] = Weighted unit response rate
NB: The weight to be taken is the sampling weight
Please indicate the maximum number of recalls/reminders before coding an enterprise as non-responding.
Comments:
Only two paper reminders are sent when a firm does not respond, but there is also a targeted follow up of nonresponding firms by phone. In Flanders, large firms may receive up to 30 reminder phone calls before they are
coded as non-responding.
15
4.3.4.2 Item response rate
Definition:
Un-weighted Item Response Rate= (Number of units with a response for the item) / (Total number of eligible, for
the item, units in the sample)
Table 4.4: Item response rates
Turnover
Number of enterprises
Item response rate
(un-weighted)
91.6 %
3138
Imputation
(Y/N)
Y
If imputed, describe method used, mentioning which
auxiliary information or stratification is used
Turnover imputations were made on the basis of several
sources of information:
- First we checked within a commercial database
containing accounting data for all firms in Belgium
(Belfirst) whether this information could not be
retrieved;
- We also checked on the websites or through other
accounting databases (from journals or from th
Belgian post office) whether the information could
not be retrieved;
- finally, if no information was available, we let the
SAS program impute the missing value according to
the same rules that were already used in the CIS4 and
CIS 2006
289
Comments:
1. We do not quite understand what is meant by “number of enterprises”; the figures represent the number of
enterprises wihout missing turnover, and the number of enterprises for which turnover had to be imputed, in the
realized sample only.
2. Imputations: we only count here the number of enterprises in the realized sample for which a missing value had
to be imputed (Fturn08=1); we do not count the number of firms for which the reported value had to be changed
(Fturn08=2).
4.3.4.3 Reasons for item non-response
Please specify possible reasons (such as sensitive information…).
Comments: some firms, especially small ones, have a feeling that information about any monetary variable (be it
their turnover or their expenditures) might prove sensitive.
16
4.3.4.5 Extent of imputation
Imputation is the method of creating plausible (but artificial) substitute values for all those missing.
Definition:
Imputation ratio (for the variable x) = (Number of imputed records for the variable x) / (Total number of
possible records for x)*100
Table 4.5: Imputation ratio
For metric variables:
Nace
Breakdown
Turn
(1)
TurnIn
(2)
TurnMar
(3)
RrdInX
(4)
Rtot
(5)
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
Total industry
(excluding
construction)
Total
Core Services
8.43
8.11
8.08
11.47
24.32
24.13
18.60
35.18
22.74
22.46
17.31
33.33
16.76
18.18
14.97
15.41
8.58
10.20
7.87
4.51
7.49
25.55
23.79
15.39
7.21
Total
9.40
22.72
21.29
18.71
10.38
Total NACE
B_C_D_E
NACE sections H and K, and
NACE divisions 46, 58, 61, 62, 63
and 71
For ordinal variables: Proportions of imputed values separately per each category (“yes” and “no” answers)
Nace
Breakdown
NewFrm
(6)
INABA
(7)
INONG
(8)
RrdIn
(9)
Co
(10)
Total NACE
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
B_C_D_E
NACE sections H and K, and NACE
divisions 46, 58, 61, 62, 63 and 71
Total industry (excluding
construction)
Total
Y= 18.41
N= 12.74
Y= 16.63
N=13.37
Y=15.49
N=6.67
Y= 28.49
N=22.72
Y=4.87
N=2.10
Y=4.66
N=1.81
Y=3.20
N=2.38
Y=7.08
N=3.52
Y=1.72
N=1.52
Y=2.07
N=1.28
Y=1.63
N=1.67
Y=1.01
N=3.52
Y=5.83
N=19.69
Y=6.69
N=19.46
Y=5.32
N=21.38
Y=4.50
N=15.91
Y=2.20
N=
Y=2.62
N=
Y=2.06
N=
Y=1.51
N=
Y= 19.73
N=13.46
Y=4.29
N=1.73
Y=1.20
N=1.41
Y=6.57
N=17.90
Y=1.74
N=
Y= 17.14
N=12.23
Y=6.47
N=2.42
Y=2.37
N=1.63
Y=4.99
N=20.96
Y=2.99
N=
Core Services
Total
[1] = Total turnover in 2008.
[2] = Turnover due to new or improved product (Share).
[3] = Share of new or improved products to market.
[4] = Expenditure in intramural RD.
[5] = Total innovation expenditure.
[6] = New or improved product for your enterprise.
[7] = Abandoned or suspended before completion
[8] = Still ongoing at the end of the 2008
[9] = Engagement in intramural RD
[10] = Cooperation arrangements on innovation activities.
17
Definition:
Weighted imputation ratio= (Total weighted quantity for imputed values) / (Total weighted quantity
for all possible values)*100
Table 4.6: Weighted Imputation ratio
For metric variables:
Nace
Breakdown
Turn
(1)
TurnIn
(2)
TurnMar
(3)
RrdInX
(4)
Rtot
(5)
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
Total industry
(excluding
construction)
Total
Core Services
8.11
8.04
7.79
11.46
24.40
24.98
19.89
33.64
22.22
22.27
18.93
33.05
21.51
23.64
16.37
14.16
10.32
11.49
7.92
4.67
6.77
23.24
21.53
17.67
8.06
Total
9.13
25.39
22.75
25.28
12.50
Total NACE
B_C_D_E
NACE sections H and K, and
NACE divisions 46, 58, 61, 62, 63
and 71
[1] = Total turnover in 2008.
[2] = Turnover due to new or improved product (Share).
[3] = Share of new or improved products to market.
[4] = Expenditure in intramural RD.
[5] = Total innovation expenditure.
NB: The weight to be taken is the weight used in estimation (e.g. after calibration)
Please indicate whether the CVs reported in table 4.1 incorporate the effects of non-response.
The denominators for Turn, InAba and InOng is the whole sample; the denominators for TurnIn, TurnMar and
NewFrm are product innovators; the denominators for RRrdn, Rrdinx, Rtot and Co are the enterprises with
^product or process innovation activity.
18
7. COMPARABILITY
7.1 Introduction
Comparability aims at measuring the impact of differences in applied statistical concepts and definitions on the
comparison of statistics between geographical areas, non-geographical domains, or over time. This is the extent to
which differences between statistics are attributed to differences between the true values of the statistical
characteristics.
The factors that may cause two statistical figures to lose comparability are attributes of the surveys that produce
them. These attributes may be grouped into two major categories: (a) concepts of the survey and (b) measurement /
estimation methodology.
7.2 Methodological deviations
This part focuses on quantifying and measuring the deviations from the methodological recommendations defined
for the CIS 2008. So please fill in only if deviations from the methodological guidelines set for the CIS 2008 took
place when the CIS 2008 was implemented at national level.
Table 7.1: Methodological deviations
Methodological
recommendations for the
CIS 2008
TARGET
DEVIATIONS
(YES/NO)
QUESTIONNAIRE
yes (very slight)
Very slight deviation in the order of the questions: we grouped the
General information’ question (question 1) with the ‘economic
information’ question (question 11) in a ‘Module A: General
information’ question. This should really not impact the results.
no
Deviation from the
harmonised CIS 2008
questionnaire
National data collection
period
Deviation from the sampling
frame
no
- Nace sectors covered
all 2 digit
yes
- size classes
small,
medium,
large
enterprise
no
- statistical unit
COMMENTS ON THE IMPACT OF THE DEVIATIONS*
TARGET POPULATION (1)
With respect to Eurostat’s ‘Core Nace” sectors, we also added the R&D
sector (Nace 72). This should not impact the results.
no
SURVEY METHODOLOGY
SAMPLING FRAME
Sampling frame
Business
register
Date of business register
extraction
- Survey method
no
no
sample or
census
mail
2
no
- NACE
- Size
2 digit +
small,
medium,
large
no
- sampling method
Stratified
simple
random
sampling
no
- Mail survey
- Reminders
DATA COLLECTION
Mix of sampling and census: both methods were used, depending on
the stratum
no
no
STRATIFICATION OF THE SAMPLE
no
SAMPLING
SAMPLE ALLOCATION
19
Methodological
recommendations for the
CIS 2008
- allocation method used
TARGET
- Percentage of innovators
- share of new or improved
products in turnover
- total turnover per employee
(X ± 5%)
(X ± 5%)
proportion
al/optimu
m
DEVIATIONS
(YES/NO)
COMMENTS ON THE IMPACT OF THE DEVIATIONS*
no
PRECISION (2)
(X ± 10%)
UNIT RESPONSE
- Non-Response survey**
- Results? Reweighing after
the NR survey?
Use of Eurostat quality
control rules
Partial
Partial
no
DATA PROCESSING
Parameters (3)
We adapted the SAS application provided by Eurostat for CIS4 and
CIS2006 to CIS2008 questionnaire and processed the microdata with
the application.
* Please provide also a brief description of the deviation if possible
** No deviation in case that a non-response survey is not conducted due to a high response rate (>70%)
7.3 Comparability over time
This part compares key variables for aggregated CIS 2008 data with CIS 2006 data.
Definition of relative difference between CIS 2008 and CIS 2006 data: DIFF = (CIS2006/CIS2008)*100
Table 7.2: Comparison between CIS 2006 and CIS 2008 data (relative difference)
NACE
Breakdown
Inn_Ent
(1)
Co_N
(2)
RTOT
(3)
TURNNEW
ALL
(4)
TURNNEW
MKT
(5)
Total NACE
Total
Small [10-49]
Medium-sized [50-249]
Large [> 249]
(1)
(2)
(3)
(4)
(5)
109.05
91.41
96.46
79.49
72.36
127.94
126.74
95.56
101.86
102.07
84.83
106.60
119.97
84.62
115.93
116.22
103.57
77.86
121.03
92.67
Proportion of enterprises with innovation activity.
Enterprises with co-operation arrangements (CO_N = [(N_innocoop/N_innovation active) in cis2006] / [(N_innocoop/N_innovation active) in cis2008]).
Total innovation expenditure as a % of total turnover for enterprises with innovation activity.
Turnover from all new products as a % of total turnover, for enterprises with innovation activity.
Turnover from new products new to the market as a % of total turnover, for enterprises with innovation activity.
20