Asymmetric Theory and Testing:
Embracing Complexity Theory,
Performing Contrarian Case Analysis,
Modeling Multiple Realities
4 March 2105
Bond University
Arch G. Woodside, Boston College
School of Marketing
Curtin University
1
Ask for copy of my book!
[email protected]
Electronic copy available
from me for free.
2
Free software for fsQCA.com
http://www.u.arizona.edu/~cragin/fsQCA/
3
Free organization to join: compasss.org
4
The Grand Incompetency
• Not testing regression models for predictive validity is the grand
incompetency.
• Achieving high fit validity is easy.
• Real value of a model is predictive validity.
• Need to use new data (holdout) sample to test for predictive validity.
• “… weighting and adding can lead to over fitting––that is, to excel in
hindsight (fitting) but fail in foresight (prediction).
• The task of humans and other animals is to predict their world despite
its inherent uncertainty, and in order to do this, they have to simplify.
• While tallying simplifies by ignoring the information required to
compute weights, another way to simplify is to ignore variables
(cues).”
• “Grand” because reporting only fit validity of regression models is
the dominant logic in psychology, marketing, finance, and
management studies and journal articles.
• “Incompetency” because most of us have not read Gigerenzer. 5
The Grand Delusion
• Grand delusion: Asking what is the “net effect” of each
variable in a regression equation.
• Regression equations are symmetric tests that attempt to
predict both high and low scores for a dependent
variable.
• What we usually want to know/do in theory and
practice are how to craft algorithm (recipe, rubric) that
almost guarantees/predicts a high outcome score or the
negation of a high outcome score in an outcome
condition—a “right answer; a solution”—an accurate
asymmetric test.
• Asymmetric tests are case-based testing, not variable
based testing.
6
Iconic, brilliant, reading on the
Grand Incompetency and Grand Delusion
7
•
•
This point illustrates the fact that achieving a good fit to observations does not necessarily
mean we have found a good model, and choosing the model with the best fit is likely to result
in poor predictions.
Despite this, Roberts and Pashler (2000) estimated that, in psychology alone, the number of
8
articles relying on a good fit as the only indication of a good model runs into the thousands.
Panel A: Rectangular
High
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
abcd
High
efgh
Medium
Medium
Low
Low
Medium
a
High
b
e
f
c
g
ijkl
Low
mnop
Low
High X
Panel C: Asymmetric—
Sufficient but Not Necessary
Y
Medium
High X
Panel D: Asymmetric—
Insufficient but Necessary
Y
High
d
abcd
h
e
f
g
h
Medium
Medium
i
j
k
l
i
m
mnop
Low
Panel B: Symmetrical
Y
Y
Low
Low
Medium
High X
Low
n
j
k
l
o
Medium
p
High X
Figure 1
Rectangular, Symmetrical and Asymmetrical Relationships: Hypothetical Plots of
16 Cases (a through p) for Outcome Y and Complex or Simple Causal Statement X
9
Study/paper using the current
dominant logic that illustrates the
Grand Delusion
10
• The Prentice and Woodside study provides a first look at the perspectives
and profiles of casino problem gamblers.
• The study proposes that problem gamblers (1) have unique antecedent
conditions and (2) evaluate their casino service more favorably than nonproblem gamblers.
• While first proposition receives support, the findings counter the second;
surprisingly, problem gamblers view casino service with a harsh gaze.
• The coverage here includes overall and specific findings from face-to-face
interviews with gamblers (n = 348) inside seven casinos in the world’s
largest gaming destination (Macau).
• The interviews included asking participants to complete the “Problem
Gambling Severity Index” (identified to participants as “My gamblingrelated experiences”).
• The study includes both fit and predictive validities of overall service quality
models for each of the seven casinos—these findings support the
nomological validity that specific patterns of antecedents and outcomes
associate with problem gambling.
• Policy and managerial implications inform how to go about creating unique
marketing service designs to assist problem gamblers in managing their
gambling behavior.
11
12
13
14
15
Overcoming the Grand Delusion
16
•
For the distribution of cases, the symmetric main effect is negative; phi = .288, p < .081. ANOVA findings indicate
significant differences in overall service quality by problem-gambling segments that supports a significant symmetric
negative main effect, means (standard errors) for the five PG segments from low to high: 9.82 (.10); 9.31 (.21); 9.67
(.19); 9.66 (.24); 9.05 (.26); F = 2.68, DF= 4/406, p < .032. The findings include contrarian type 1 cases: cases with
high scores on the outcome condition that counters the negative symmetric main effect; the findings include contrarian
17
type 2 cases: cases with low scores on the outcome condition that counters the negative symmetric main effect.
18
• Table 3a reports the findings for high scores of problem gambling using
fsQCA and shows two complex recipes predicting high scores of problem
gambling.
• The two models for high problem gambling consist of males (both young
and old), one with low education and high income, and one with high
education and low income.
• Both algorithms have high occupational status and are in position of a
casino reward card, while their length of play of each visit to the casino is
low. One has high average bet size and low annual visits, while the other
one has low average bet size and high annual visits.
19
• The findings for immediate service evaluations only partly support the main claim
by Prentice and Woodside (2013) that problem gamblers gaze harshly on casino
services. In three of the four complex configurations, using the non-problem
gambling as an antecedent, non-problem gamblers give casinos positive
evaluations on their immediate services.
• However, in one of the configurations leading to high immediate service
evaluations, severe problem gamblers figure in, suggesting that there are indeed
problem gamblers that do not gaze harshly on casino services, depending on the
rest of the complex configurations.
20
Core Complexity Precept
• Cases contrary to supported hypotheses just about
always are found in large data files (n ≥ 100)
• Consequently, hypothesizing main effects and twoway interactions, moderating, and mediating
effects ignores important information in a data
set—information that is extractable and highly
informative.
21
Complexity Theory Tenets
• Equifinality
• Reversals in direction occur of the impact of
simple antecedent conditions on an outcome
• Insufficiency of simple antecedent conditions
• Un-necessity of complex antecedent conditions
• Causal asymmetry
• Necessary but insufficient simple antecedent
conditions
22
Complexity Theory Tenets
• (1) Equifinality tenet: a few (not one) of the many
possible paths lead to the same outcome, that is,
“equifinality” occurs—alternative asymmetric
combinations of indicators (i.e., algorithms) are
sufficient but …
• No one combination is necessary for example, for
accurately predicting customers' highly positive
evaluations of service performance and high
intentions to return to the same service provider.
23
Reversals in X-Y relationships occur
• (2) The contrarian cases tenet: simple antecedent
condition relates to an outcome positively as well
as negatively
• Observable when you create quintiles of all simple
conditions (variables)
• Example: divide cases (managers) self-evaluations
into 5 levels of decision competency and these
same cases into 5 levels of their supervisors’
evaluations of their decision competencies and
cases occur in all 25 cells.
24
Decision competency
Following
Persisting
H1
Planning/
guiding
H5
S1
Volunteering
Additional
Helping
Self: Demographics, Traits,
and Prior MOPA
Confirmed/Conflict
S1 & S2 Assessments
Edu
Pa/Na
MOPA
Years mgr
Overall Assessment of
Contribution by Manager
H3
S1
S1
H7
U
H8
S2
H9
S1 =Self
S1•S2 S2 = Supervisor
Marital
Gender
H4
Age
Helping
Additional
H2
Planning/
guiding
Volunteering
S2
H6
Persisting
Following
Decision competency
Figure 1
Complexity Theory of Antecedent Conditions Indicating High
versus Low Overall Manager’s Contribution—
The Hosie and Woodside Model
Key: Additional = includes training,
communicating, representing,
administrating; influencing
Pa/Na = positive and negative
personality traits; MOPA =
manager’s overall performance
assessment; S1 = self; S2 = supervisor
25
Study of managers’ self-assessment and their
supervisors’ assessments of their contributions
• Symmetric test issues: does a positive relationship occur? Effect
size?
• Asymmetric test issues: do contrarian cases occur? What recipes
explain outcomes for these contrarian cases?
26
• Symmetric tests: rraw = .495
• rquintiles = .646
• effect size is large, r ≥ .50 (Cohen, 1977)
27
• Cases contrarian to the main effect occur
• Contrarian 1 cases: “white hat” cases—manager reports
she/he is contributes little and supervisor reports she/he
contributes a lot, n = 13
• Contrarian 2 cases occur: “black hat” cases—manager
reports she/he contributes a lot; supervisors report that she/he
contributes little, n = 7
28
Table 3
Hospitality Employees’ Happiness and Managers’ Evaluations of Employees’ In-Role Performances
Very
low
In-Role Performance
Quality (IRP)
Very
high
Very low
Happiness
Quintiles for
Hospitality
Employees
Very high
Possibly surprising findings:
cases do occur of very unhappy employees
with very high IRP scores and vice versa.
•
•
•
•
1. Does happiness relate to job performance positively?
2. Are some employees very unhappy and perform very high?
3. Are some employees very happy and perform very how?
4. Note the question 1 is variable-based question and questions
2 and 3 are case based questions.
• 5. We model antecedent case based questions with recipes. 29
30
Fit-as-gestalts
• Fit-as-gestalts is at the core of configuration theory
(Ragin 2000) which focuses on complex,
multidimensional phenomena at different levels
(firms, groups, individuals) that tend to cluster into
archetypes described by common patterns of coherent
attributes.
• As its fundamental premise, configuration
[complexity] theory posits that the same set of causal
factors can lead to different outcomes, depending on
how each of the factors appear in different recipes.
• Ordanini, Parasuraman, & Rubera (2014)
31
qip
full-time
education
sup
1
lone
gender
Individual
demographic
configurations
irp
cderp
6
Manager’s
assessment of
employee in-role
and customer
extra-role
performances
pwe
2
marital
3
Employee’s Demographic
Configurations
7
5b
age
pconflict
children
5a
Work
environment
configurations
p
4
Happiness
dteam
9
On-the-Job Facet-Specifics-Configurations by the Employee
8
Manager’s Evaluation of Employee
Figure 2
Configural Modeling Associations with Very High/Low Hospitality-Service Employee Work Contexts,
Very High/Low Happiness, In-Role, and Customer-Directed Extra Role Performances
Key: dteam = job demands of teamwork; cderp = customer-directed extra role performance
irp = in-role performance; lone = do not join social activities with my colleagues
pconflict = peer conflict; pwe = physical work environment pleasing
qip = quality of interpersonal relationships; sup = supervisor support
32
33
consistency
(B)
2
3
1
6
8
(A)
n=9
11
33
n = 73 cases
23
9
8
4
2
1
CDERP
n = 7 cases
(C)
2
IRP
12 cases
2
coverage
Figure 4
Impact of Managers’ Evaluations of Hospitality Service Workers’
In-Role Performance (IRP) on Managers’ Evaluations of
Customer-Directed Extra-Role Performance (CDERP) (n = 243 cases)
Notes. Numbers indicate the number of cases for each dot. (A) While high IRP is informative in
explaining high CDERP; (B) nine cases exhibit high CDERP and low IRP. Thus, high IRP is not
necessary for high CDERP. Also, the few cases that are (C) high IRP and low in CDEREP indicates
that IRP is not completely sufficient for explaining high CDERP—seven cases occur with high IRP
and very low CDERP.
34
Table 4a
Configurations of On-Job Happiness and Additional On-Job Antecedents
Associating with Managers’ Judgments of Employees’ High In-Role Performances (IRP)
(Arrows 5 and 5b in Figure 3)
Model
1
2
3
4
5
6
7
Table 4b
Configurational Models for Demographics and Happiness for High In-Role Performance
Model
1
2
3
4
5
6
7
8
9
35
Table 5a
Configurations of On-Job (Un)Happiness and Additional On-Job Antecedents
Associating with Managers’ Judgments of Employees’ Very Low In-Role Performances (~IRP)
(Arrows 5 and 5b in Figure 4)
Model
1
2
3
4
Table 5b
Configurational Models for Demographics and (Un)Happiness for Very Low In-Role Performance
Model
1
2
3
4
5
6
36
1a no association
~qip
1b no association
Model D
~age (young)
~ education (not)
married
gender (male)
•
•
•
•
•
•
•
•
•
Model G
~age (young)
~education (not)
~married (not)
~gender (female)
children
~full-time
sup
5a (see notes below)
1
Work
environment
configurations
lone
7 C = .83
5b
1c Consistencies for ~pwe models D, G = .82, .91~pwe
pconflict
4
dteam
2
~irp
~ cderp
6
p
Consistencies for not happiness models D, G = .88, 1.00
~happy
manager’s
assessment of
employee in-role
and customer
extra-role
performances
3 Consistency for ~irp models D, G = .83, .91
Employee’s Demographics
On-the-Job Assessment by Employee
Manager’s Evaluation of Employee
Figure 5
Configural Modeling Associations with Hospitality-Service Employee Work Contexts,
Very Low Happiness, Very Low In-Role, and Very Low Customer-Directed Extra Role Performances
Key: dteam = job demands of teamwork; cderp = customer-directed extra role performance; irp = in-role performance; lone = do not join social
activities with my colleagues; pconflict = peer conflict; pwe = physical work environment pleasing; qip = quality of interpersonal relationships;
sup = supervisor support; yservice (not shown in figure)
Notes on findings. (1) For arrow 4: five work environment configurations have high consistencies in associating with not happiness.
For example, ~happy ≤ ~yservice●~qip●~pwe~dteam●pconflict●lone●~sup, consistency = .93. See Table 6b for detailed findings.
For arrows 5a and 5b, 6 of 7 models predicting high membership in low irp (i.e., ~irp) very well include ~yservice. Example model:
~irp = ~fulltime●~happy●~pwe●~dteam●~pconflict●lone●~supsup_c●~yservice●~qip_c (consistency = 0.982)
37
• Hosie study: managers’ self assessment of
abilities and contributions/performances and their
supervisors’ assessment these same managers’
abilities and contributions.
• Hosie study: managers’ self assessment and their
supervisors’ assessment on managers’ decisionmaking competency—do they agree?
• Hosie study: managers’ self assessment and their
supervisors’ assessment of that managers are
making low versus high contributions to the
organizations’ success—do they agree?
38
• Do managers’ and their supervisors’ agree or disagree on the
managers’ competencies making decisions?
• Correct answer: Yes. Both. Some agree and some do not. Need to
model both. But more disagree that agree.
• Statistically test findings: no relationship
39
• Overall contribution: some agreement, more so than for
decision-competency
• But some disagree as well.
40
Does confirmed high manager decision competency
alone predict high contribution?
consistency
n = 16
n=3
Case number 169
coverage
•
•
•
•
Yes.
Consistency = .853—a useful model
19 cases high in confirmed decision competency; 16 of 19 high in contribution!
Can we improve on the accuracy of this model?
41
•
•
•
•
•
•
•
•
•
•
•
In recipes that include confirmed decision-making competency or incompetency does…
Does gender indicate contribution?
Does education indicate contribution?
Does positive personality indicate contribution?
Does age indicate contribution?
Does negative personality indicate contribution?
5 models are very useful
4 models include highly decision competent male—2 models with positive personalities and 1 with
negative and positive personality, and 1 emotionally challenges;
1 female model with negation decision competency who have bi-polar personality; young with low
education.
Let’s read each model together
Model 1: positive personality AND low education AND old AND low level mgr AND married
AND MALE and confirmed highly competent
42
• Calibrating a scale.
• 200 cases (200 managers)
• Report on how many levels that they are from the
CEO
43
Median = 3.00 → become 0.50 point of
“maximum ambiguity” for calibrated scale
For “great distance from CEO.
• Set missing values = 3 = 0.50
• Set 1 = 0.05 “full non-membership = 10.6% of data
• Set 5 = 0.98 “full membership” = 6.9% have scores
• higher than 5
44
Calibrated
.01
.05
.18
.50
.82
.95
.99
1.00
1.00
1.00
1.00
45
46
• Age quintiles appear above.
• How would you calibrate by age quintiles?
47
•
•
•
•
•
1 = .05
2=
3 = 0.50
4=
5 = 0.95
48
Calibrating age
• fsQCA.com software will calibrate all values as
you calibrate values equal to 0.05; 0.50; 0.95
• Thus,
• Value
Calibrated value
• 1
=
.05
• 2
=
.18
• 3
=
.50
• 4
=
.82
• 5
=
.95
49
Calibrate Education
Mdn = 5.00
•
•
•
•
What would you do?
What is the “point of maximum ambiguity?
What is the value for a calibration = 0.05?
What is the value for a calibration = 0.95?
50
Calibrate Education
Mdn = 5.00
•
•
•
•
What would you do?
What is the “point of maximum ambiguity?
What is the value for a calibration = 0.05?
What is the value for a calibration = 0.95?
5.00
3.00
7.00
51
Managers’ self assessment models of high decision
competence
52
Supervisors’ assessment models of
managers’ decision competence
53
Confirmed decision competency
models
• Both models indicate married female managers highly educated in low
level organizational positions, young, with
• Model 1 highly negative personality
• Model 2 highly pleasant personality
• Are high in confirmed decision competency
• Interesting that females do not go on to be included in recipes for
confirmed high contribution very often.
54
Predictive Validity Test Confirmed Decision Competency
Models for First 100 cases and Second 100 Cases
Models from First 100 Cases
6 cases
Test of Model 2 from the first 100
cases on data for the second
100 cases
These findings do not provide high
predictive validity
3 cases
55
Confirmed DC including prior overall assessment in model
Model for second sample, n =100
The model for the second sample is not predictive for the first sample, n =100.
Implication: Only models with consistencies
above 0.85 are going to have high predictive
accuracies.
56
Nigel
•
•
•
•
Arch
People happy and unhappy at same time
People happy and not unhappy
People not happy and not unhappy [free of emotional involvement]
People not happy and highly unhappy
57
58
pa•na = people very happy and highly unhappy (bi-polar) (10%)
pa•~na = people very happy and not unhappy (16%)
~pa•na = people not happy and very unhappy (sad sacks) (18%)
~pa•~na = people not happy and not unhappy (emotionless) (11%)
59
60
Thank you!
61