IS6000
Seminar 9
Research Methods – Quantitative
Surveys
1
Methods and Data


A number of methods are designed around
quantitative data
These include



Surveys, Experiments, Simulations
Today, I will focus only on surveys
But first, we need to look at some different
types of quantitative data

In each case, we assign a numerical value to an
empirical property of something that we are
interested in
2
Quantitative Data Types

Perceptual Data (Subjective)

Respondents to surveys are asked to tick or circle boxes
that correspond to their opinions on, e.g., 5-point scale


Respondents are asked to mark their perception on a line
that is anchored at agree and disagree


1 – strongly agree – 5 – strongly disagree
The position on the line can be calculated between (usually) 0
and 100
For Example



Management in your company is incompetent
Knowledge sharing is a waste of time
To be effective, a C2C website needs to be localised
3
Quantitative Data Types

Objective Data


Respondents are asked to indicate an exact number
or an answer within a range in response to a question
For Example

How many emails do you receive per day?


How many Facebook friends do you have?



1-5; 6-10; 11-20; 21-50; 51-100; 100+
Exact number requested.
How many people do you exchange knowledge with?
How well do you know each of these people?

For each person: Very well, Well, Average, Not well, Not at all
well
4
Quantitative Data Types


Demographic Data
These questions may be sensitive, so respondents
could tick a value in a range






How old are you?
What is your education level?
What is your monthly/annual salary?
How many children do you have?
What is your gender?
 Male or Female or Other, but coded as a number
How many years work experience in the current job?
5
Constructs and Variables

In Class 6, we looked briefly at constructs
and variables




A construct is a concept that has been formulated
so as to be used in science
A high level construct may contain multiple
variables
A variable is an empirical indicator of a construct
Typically, we measure 3-5 items for each
variable

Each variable needs to measure
6
For Example



Construct – Guanxi
Variables - Reciprocal Obligation, Trust, Face
Items (Reciprocal Obligation)
1.
2.
3.
4.
5.
My acts of knowledge sharing and seeking within my network strengthen the
ties of obligation between existing members in my network and myself.
My acts of knowledge sharing and seeking create obligations with other
members in my network.
My acts of knowledge sharing and seeking expand the scope of my
association with other members in my network.
My acts of knowledge sharing and seeking will encourage cooperation among
my network members in the future.
My acts of knowledge sharing and seeking create strong relationships with
members who have common interests in my network.
7
Chinese Version of These Items





1) 我在关系网中的知识分享和寻找行为巩固了网络成
员和我之间的在于责任义务方面的联系。
2) 我的知识分享和寻求行为建立了我与其他成员之间
的责任与义务。
3) 我的知识分享和寻求行为扩大了我与关系网中其他
成员的交往范围。
4) 我的知识分享和寻求行为促进我与关系网其他成员
的合作。
5) 我的知识分享和寻求行为为志同道合的关系网成员
创造了紧密的关系
8
Survey Design 1


In a survey, clearly we need to ask questions
relevant to the overall research question
A theory will make it easier, but we have to decide
which constructs we are interested in



These could be represented in a formal model, e.g.
TAM, TPB
For each construct, we need variables and items
We must also decide if items are formative or
reflective

Do they contribute to the variable or are they reflective of
the variable?
9
Survey Design 2

It is easier to borrow items from previous
research




Because they have been tested as reliable and
valid already
We may need to adapt them a bit to our context,
e.g. if we are interested in a specific technology
But we should not change them too much
If in previous research, three items were validated
together, then we must use all three – we can’t
drop any out
10
Survey Design 3





If we really can’t find a suitable construct, e.g.
because this is totally new research, then we may
need to develop our own
This is not simple
We need to collect lots of evidence (e.g. from
interviews) about what content should be covered in
the construct
Then we have to develop – and validate – variables
for the construct
This can take weeks or months!
11
Validity

Face validity


Content validity


Are all important issues included in the construct?
Convergent validity


Do the items make sense to potential
respondents?
Do all the items/variables for a construct show
statistical convergence (similarity)?
Divergent validity

Is a construct, and its variables/items, statistically
different from other constructs?
12
Validity Tests

Two techniques we can use to demonstrate
construct validity are

Cronbach’s Alpha



Do all the items in a construct/variable group together,
and how well?
A good score is 0.70 or higher; 0.90 is excellent
Factor Analysis

Do all the items in a construct/variable group together
but not together with items from other
constructs/variables


Expected loadings must be higher than loadings on other
constructs
A good score is 0.50 or higher; 0.80 is excellent
13
TAM Validity Scores (Gmail)





Factor 1: Perceived Ease of Use (Alpha = 0.93)
1. Learning to use Gmail would be easy for me
0.872
2. I would find it easy to get Gmail to do what I
want it to do.
0.862
3. It would be easy for me to become skillful at
using Gmail.
0.895
4. I would find Gmail easy to use.
0.894
14
Good Data: High Loadings, Low CrossLoadings
Factor 1
Factor 2
Factor 3
Factor 4
Item 1
0.85
0.15
0.15
0.33
Item 2
0.76
0.24
0.24
0.31
Item 3
0.89
0.45
0.45
0.15
Item 4
0.34
0.87
0.22
0.24
Item 5
0.44
0.79
0.16
0.45
Item 6
0.24
0.91
0.25
0.22
Item 7
0.15
0.15
0.77
0.16
Item 8
0.24
0.24
0.88
0.25
Item 9
0.45
0.45
0.85
0.33
Item 10
0.22
0.22
0.11
0.92
Item 11
0.16
0.16
0.08
0.88
15
Survey Design 4


When we know which constructs we want to
measure, which variables and which items,
we can create the survey
We may not put all similar items together –
instead we can mix them around


This helps to avoid the situation where a
respondent simply ticks “strongly disagree” to
everything
Different items may have different scales
16
Survey Design 5 - Scales


Agree to disagree is a commonly used scale
But we could create other scales like





Important to Unimportant
Effective to ineffective
Practical to Impractical
Each item may have a different scale
Some items may be reverse scaled

Disagree to Agree

But be careful when coding the data!
17
Survey Design 6

Length!

If a survey is too long, the response rate will be
low



Response rates below 15% are problematic
May not be a representative sample of people
If a survey is too short, your scope may be too
narrow or you may have too few items per
variable


A two-page survey (A4) is fine
A 10 question survey is probably too short for any
meaningful analysis
18
Survey Design 7

Sequence



You may put less sensitive questions at the start,
more sensitive later
Age, gender, etc. can be more sensitive
Optional or Compulsory

What if people refuse to answer a question – is
that ok?


On paper, they can refuse. Electronically, you can force
them to answer – or they cannot submit.
However, if you force them, they may drop out
19
Survey Design 8

Sampling

You can’t sample everyone – and you don’t need
to do so



But you need enough to be representative
So, who do you want to complete the survey?
If you want 50 responses, you might have to invite
10 times that many people!



Convenience samples
Random samples
Quota samples
20
Survey Design 9

Convenience Samples


Random Samples


A group of people (e.g. students) who are easily accessible
A randomly collected mixture of people from different
contexts (e.g. on the street, office, airport)
Quota samples

Similar numbers of people from pre-specified functions



Students, Academics, Cleaners, Gardeners, …
Housewives, businessmen, bus drivers, …
Etc.
21
Data Analysis



Survey research is often conducted in order
to collect data so as to test hypotheses
In a research model, hypotheses express
relationships between constructs/variables
As a rule of thumb, you need 10 responses to
a survey for each construct in your model that
you are testing

A 10-construct model needs 100 responses
22
Hypotheses


Hypotheses are key components of theories
and structural models
In the TAM model, there are hypotheses like:



H1: Perceived ease of use will have a significant
positive influence on perceived usefulness.
H2: Perceived usefulness will have a significant
positive influence on attitude toward using.
H3: Perceived ease of use will have a significant
positive influence on attitude toward using.
23
Hypothesis Testing


In order to test if a hypothesis is supported by
the data, we can use a variety of statistical
techniques
A commonly used technique is called PLS


Partial Least Squares : http://www.smartpls.de
PLS will calculate how strong is the ‘path’
between one construct and another


E.g. from an IV to a DV
PLS will also calculate the strength of all IVs to a
single DV
24
Model of TaoBao Buyers’ Development of
Trust and Swift Guanxi with Sellers (n=338)
25
Significant Values



On the previous slide, you can see numerical scores
and * or **.
These scores are indications of path strength
between constructs
The * and ** are indicators of significance



* = significant at the 95% level
** = significant at the 99% level
Significance refers to how confident we are that a
hypothesis is supported

Higher levels of significance are clearly better – they give
us more confidence that the relationship is demonstrated
26
Calculating Significance





Most statistical packages will calculate
significance for you, so you don’t have to
worry about this
The ‘t’ value will tell you if the score is
significant or not
A ‘t’ value below 1.96 is not significant
1.96 < t < 2.58 will be 95% (*) significant
t > 2.58 will be 99% (**) significant
27
Reporting Survey Research



In a survey research report, it is common to see lots
of tables showing scores and significance levels
This kind of evidence is used to demonstrate that
constructs are validated and that hypotheses are
supported (or not).
In addition, in a discussion section, the authors will
‘discuss’ the significance of their findings


Especially if some hypotheses are either very strongly
supported – or rejected
This is similar to qualitative research where there is strong
evidence in support of a proposition – or a lack of evidence
28
Conclusions


The survey is an excellent method for
collecting data from a large group of people
in order to get answers to broad, high level
questions
You must fix all the questions in advance


If you want to add more questions later, then you
must collect new data
If you want to ask deeper questions about
how and why, then a case study may be
better.
29