WELCOME!
1
Critical Integrative Metatheory:
New Methods to Evaluate
Psychological Theories &
Models for Review
Steven E. Wallis, PhD
[email protected]
Director, Foundation for the Advancement of Social Theory
Adjunct Faculty, Capella University
Fellow, Institute for Social Innovation, Fielding Graduate University
Editorial Board, Integral Review
International Congress of Psychology 2012
July 22-27 – Cape Town, South Africa
2
We want psychology to be of
greater benefit to humanity
Learning
Psychoanalysis
Motivation
Communication
Behaviorism
Cognition
Stress
Emotions
3
Emergence of Psychology
Psychology is NOT much appreciated by
other sciences or the general public. For
GOOD reasons!
We are not advancing as a science –
there is no proof that psychology is
better now than 50 years ago.
Why have we failed to improve?
4
One Path to Improving
Psychology is the Process of
Submission & Review
Theory
Creation
Submission
Review
5
Many Rigorous Standards for
Evaluating Methods
Methods
Review
Participant Characteristics
Sampling Procedures
Sampling size, power, precision
Measures and Covariates
Research Design
Recruitment Statistics
Data Analysis
Etc., etc., etc., etc., etc.,
6
Standards for Evaluating
Theory?
Theory
Review
7
What is theory?
Lens
Map
Metaphor
Story
Diagram
Narrative
Set of axioms
Model
Ethics
Policy
Mental model
Schema
Mind map
Assumptions
These are all names for a conceptual construct
that may be useful for engaging the world.
8
Theory
A theory is a conceptual construct, a set of
abstract causal relationships.
Empowers (to take effective action)
Restricts (limits our sense of “truth”)
Is developed by education, experience, and
social construction
Is (apparently) INNESCAPABLE
(so it is apparently IMPORTANT)
9
Metaphorically…
Metatheory is a theoretical “lens” that
may be directed toward the
investigation of theory.
10
Or, more formally…
Metatheory is primarily the study of
theory, including the development of
overarching combinations of theory, as
well as the development and application
of theorems for analyses that reveal
underlying assumptions about theory
and theorizing.
11
Three Ways to Evaluate
Theory
Creation
Categorization
Critical Analysis
12
Common Methods for Evaluating
the Creation of Theory (and issues)
Voracious reading (what? How much?)
Be brave, Creative, etc… (how to measure?)
Research (empirical data – what is data?)
Synthesize existing theory (recursive problem)
Reflexivity (how to measure?)
Abstraction (How much is best?)
Grounded Theory (more structured approach)
Reflexive Dimensional Analysis (more structured
approach)
13
Common Methods for
Categorization of Theory
Historical
Geographical
Unit level / Middle Range / Grand
Style (Literary, Academic, Eristic, etc.)
Purpose (Analyzing, Explaining, Predicting)
(ISSUE: Within a category,
What Theories are Better?)
14
Common Methods for Critical
Analysis of Theory
Plausibility
Parsimony
Correspondence to observed facts
Coherence to existing theory
Application
Propositional Analysis
Falsification
(ISSUE: Rarely Done, Rarely Quantified)
15
CRITICAL Metatheory
Reflects The Need For RIGOR
Better to have a specific metatheoretical
lens
The results must be quantified
If our lens of metatheory is cloudy, we
won’t see the lens of theory with clarity
16
EXAMPLE: Propositional Analysis
1.
2.
3.
4.
5.
6.
Identify the logical propositions.
Diagram the causal relationships.
Integrate diagrams.
Identify and count the “Concatenated” aspects
(two or more causal influences).
Count the total number of aspects
(Complexity of the theory).
Calculate the Robustness
(divide Concatenated aspects by total aspects).
17
Abstract Example
Propositions
within a Theory
A
A
C
E
B
C
D
C
Carefully
Integrated
B
A
C
E
D
C=5
R = 0.20
18
Previous Research
Evolution of Theory & Changes in Complexity
Number of
Aspects
Changes in the Complexity of Theory as it
Evolves Through A Scientific Revolution Toward
Greater Usefulness
15
Complex
theories –
some
usefulness
10
Scientific
Revolution
5
0
0
500
1000
1500
2000
19
Previous Research
Evolution of Theory & Changes in Robustness
Robust
theories –
Very Useful
Robustness
Changes in Robustness of Theory as it Evolves
Through A Scientific Revolution Toward Greater
Usefulness
1
0.8
0.6
0.4
0.2
0
0
500
1000
1500
2000
Scientific
Revolution
Year
20
Theory of motivation and
competence (White, 1959)
1. More new
stimulation
2. More
cognition
3. More action
4. More effect
on environment
5. More learning to interact effectively with environment (more competence)
6. More feelings of efficacy (motivation)
C=6
R = 0.17
21
Theory of planned behavior
(Perkins et al., 2007)
1. More expected value
2. More subjective norms
4. More Behavioral
intentions
5. More
behavior
3. More perceived
behavioral control
C=5
R = 0.20
22
Combined Theories
7. More
expected
value
1. More new
stimulation
2. More
cognition
(includes
behavioral
intentions
8. More
subjective
norms
3. More action
(includes
behavior)
4. More effect
on
environment
5. More learning to interact effectively with environment (more competence)
(includes perceived behavioral control)
6. More feelings of efficacy (motivation)
C=8
R = 0.25
23
Comparing Psychological Theories
Theory
Robustness
Complexity
Perkins et al., 2007
0.20
5
White, 1959
0.17
6
Combined
.25
8
MEASURABLE IMPROVEMENT 24
Approach Provides Editors and
Reviewers With a Standard for
Evaluating Theory
Theory
Review
Propositional Analysis to
determine Complexity and
Robustness
Theories of higher Complexity
are privileged
Theories of higher Robustness
are privileged
25
To Improve, We Must Push
Theories of Psychology to
Higher Levels of Complexity
and Robustness
26
Without Rigorous Standards We
Become Biased and Our Theories
Become Simple
14
Theory of motivation
12
10
Change in Complexity
8
Complexity
Linear (Complexity)
6
90 years of (de) evolution
4
150 years until it vanishes?!?
2
0
1910
1920
1930
1940
1950
1960
1970
1980
1990
2000
2010
27
Without Rigorous Standards We
Become Biased and Our Theories
Become Weak
Theory of motivation
1
0.9
0.8
Change in Robustness
0.7
Year
0.6
90 years of (weak) evolution
0.5
Robustness
0.4
Linear
(Robustness)
0.3
0.2
900 years to success?!?
0.1
0
1900
1920
1940
1960
1980
2000
2020
28
To Conclude…
Empirical data is “theory laden” – we cannot
understand data without understanding theory.
If we are make psychology more beneficial to
humanity we must directly address theory.
If we are to have effective theory, that theory
must be addressed in a rigorous scientific way
(Critical Integrative Metatheory)
Those methods must be applied to review
submissions to journals
29
Many Thanks!
Steven E. Wallis, Ph.D.
Director, Foundation for the Advancement of Social
Theory http://projectfast.org
Fellow, Institute for Social Innovation, Fielding
Graduate University
Adjunct Faculty, Capella University
Editorial Board, Integral Review
[email protected]
Fielding Graduate University is a 501 c (3)
non-profit university of higher learning.
30