Not the What of Quantitative Training
But the Who
Leona S. Aiken
Arizona State University
Teaching Statistics and Quantitative Methods into
the 21st Century
Vanderbilt University, March 24, 2017
1
Goals of My Presentation
• Explore aspects of the magnitude of the task of improving training
– Number of Psychology Departments in U.S.
– Quantitative courses and requirements at the undergraduate level
• Explore labor force for teaching statistics and quantitative methods
– Who is hired to teach statistics and quantitative methods
– Production of quantitative psychologists
• Touch upon training resources
– to enhance quantitative knowledge
– for innovation in quantitative instruction
• Open a discussion of how our field can enliven and improve
teaching of statistics and quantitative methods
Leona Aiken, Arizona State University
2
Goals of My Presentation
• Explore aspects of the magnitude of the task of improving training
– Number of undergraduate and Ph.D. institutions
– Quantitative courses and requirements at the undergraduate level
• Explore labor force for teaching statistics and quantitative methods
– Who is hired to teach statistics and quantitative methods
– Production of quantitative psychologists
• Touch upon training resources
– to enhance quantitative knowledge
– for innovation in quantitative instruction
• Open a discussion of how our field can enliven and improve
teaching of statistics and quantitative methods
Leona Aiken, Arizona State University
3
Goals of My Presentation
• Explore aspects of the magnitude of the task of improving
training
– Number of undergraduate and Ph.D. institutions
– Quantitative courses and requirements at the undergraduate level
• Explore labor force for teaching statistics and quantitative
methods
– Who is hired to teach statistics and quantitative methods
– Production of quantitative psychologists
• Touch upon training resources for innovation in quantitative
instruction
• Open a discussion of how our field can enliven and improve
teaching of statistics and quantitative methods
Leona Aiken, Arizona State University
4
Goals of My Presentation
• Explore aspects of the magnitude of the task of improving
training
– Number of undergraduate and Ph.D. institutions
– Quantitative courses and requirements at the undergraduate level
• Explore labor force for teaching statistics and quantitative
methods
– Who is hired to teach statistics and quantitative methods
– Production of quantitative psychologists
• Touch upon training resources for innovation in quantitative
instruction
• Open a dialog addressing how our field can enliven and
improve teaching of statistics and quantitative methods
Leona Aiken, Arizona State University
5
A Day of Teaching Innovation
• Undergraduate versus graduate teaching (Joe
Rodgers)
• Innovations in the teaching of statistics and
quantitative methods
analysis of real data (Charles Reichardt)
analysis of simulated data (William Revelle)
data visualization (Robert Terry)
student response systems in teaching (Shane Hutton)
appropriate level of technology (David Rindskopf)
reverse classroom strategy (Pascal Deboeck)
Leona Aiken, Arizona State University
6
A Day of Teaching Innovation
• Teaching qualitative methods
(Scott Churchill, Linda McMullen, and Heidi Levitt)
• Interplay of philosophy with teaching methods
About assumptions (Rachel Fouladi)
About philosophy of science (Mike Edwards)
About psychological science and quantitative
methods (Jolynn Pek with Jessica Flake and Ian
Davidson); (Eric Youngstrom and Mian-Li Ong)
• Causal inference/research methods—a missing piece
(Stephen West)
Leona Aiken, Arizona State University
7
My Personal Observation about our Conference
Organizer Joe Rodgers and Today’s Participants
• Presenters are scholars in methodology
e.g., Journal Editors and Past Editors
Journal of Educational and Behavioral Statistics (Rindskopf);
Psychological Methods (Steve West)
Multivariate Behavioral Research (Joe Rodgers, Steve West)
The Humanist Psychologist (Scott Churchill)
• Presenters are not typical teachers of methods who
populate our Ph.D. programs
• Presenters are certainly not representative of
individuals who teach undergraduate statistics in
Departments of Psychology
Leona Aiken, Arizona State University
8
Sedona, Arizona
Aiken, ASU
9
Goal 1: Explore aspects of the magnitude
of the task of improving statistics and
quantitative training
Number of psychology departments
Quantitative course offerings
Leona Aiken, Arizona State University
10
How Many Departments of Psychology Are There?
Program Strata
Associate degree only
Associate dominant with bachelor’s degree
Bachelor’s degree only
Bachelor’s degree plus master’s degree
Bachelor’s degree plus doctoral degree
Total
Total Number of
Institutions
1,076
98
807
718
289
2,988
Norcross, J.C., Hailstorks, R. Aiken, L.S. , Pfund, R. A. , Stamm, K. E., & Christidis, P.
(2016). Undergraduate study in psychology: Curriculum and Assessment.
American Psychologist, 71(2), 89-101.
Data presented here are taken from Table 1 of Norcross et al., (2016)
Data from Carnegie Foundation for the Advancement of Teaching (2010) data base.
Program coding based on two Carnegie variables: 2010 Basic Classification and 2010
Undergraduate Instructional Program Classification of accredited colleges and universities
Leona Aiken, Arizona State University
11
What Methods Courses are Offered
in Baccalaureate Programs?
Course
Research
Methods
Psychological
testing/
assessment
Statistics
Statistics
(advanced
undergraduate)
2014 Course Lab Included
Alone or with
Lab
Required
versusexperimental
Elective,
Typically
psychology
and
When
Offered
100%
65%
APA among
style Elective
Required Chosen
Group
98%
1%
1%
67%
11%
13%
95%
48%
96%
31%
39%
20%
Data are 43%
taken from44
Tables 2
and 3 of Norcross et al (2016),
American Psychologist,
71(2), 89-101.
2%
2%
For example,
of all schools
15% 100% 54%
teach research methods;
65% of all schools teach research
methods with a research lab; the
65% is part of the 100%
Leona Aiken, Arizona State University
12
What Methods Courses are Required versus
Elective in Baccalaureate Programs?
Course
Research
Methods
Psychological
testing/
assessment
Statistics
2014 Course Lab Included
Alone or with
Lab
Required versus Elective,
When Offered
Required Chosen among Elective
Group
100%
65%
98%
1%
1%
67%
11%
13%
43%
44%
95%
48%
96%
2%
2%
Statistics
(advanced
39%
20%
31%
15%
54%
undergraduate)
Data are taken from Tables 2 and 3 of Norcross et al (2016), American Psychologist,
Leona Aiken, Arizona State University
71(2), 89-101.
13
What Methods Courses are Required versus
Elective in Baccalaureate Programs?
Course
Research
Methods
Psychological
testing/
assessment
Statistics
2014 Course Lab Included
Alone or with
Lab
Required versus Elective,
When Offered
Required Chosen among Elective
Group
100%
65%
98%
1%
1%
67%
11%
13%
43%
44%
95%
48%
96%
2%
2%
Statistics
(advanced
39%
20%
31%
15%
54%
undergraduate)
Data are taken from Tables 2 and 3 of Norcross et al (2016), American Psychologist,
Leona Aiken, Arizona State University
71(2), 89-101.
14
Desert
Blooms
near
ASU
Aiken, ASU
15
Goal 2: Explore labor force for teaching
statistics and quantitative methods
Part a: Who is hired
to teach statistics
and
quantitative methods?
Aiken, ASU
16
Who is hired to teach undergraduate statistics?
Excerpts from all job postings by
Association for Psychological Science
from September 1, 2016 through March 16, 2017
APS Employment Network
Acknowledgement to APS for sharing a complete data base
of all position advertisements in an easily searchable format
Acknowledgement to Christina Garneski
Director of Marketing and Communications, APS
https://www.psychologicalscience.org/employment
http://aps.psychologicalscience.org/jobs/aps_JobsSearchResult-new.cfm?all=yes&search=basic
Leona Aiken, Arizona State University
17
Caveat: I present portions of job advertisements in
what follows but I do not identify the individual
Institutions, with two exceptions.
.
Leona Aiken, Arizona State University
18
Prevailing Model in Undergraduate Statistics Instruction
Faculty member in a substantive discipline who is
also expected to teach statistics
FULL TIME TENURE TRACK: Bachelors Institutions
Cognitive Psychology / Cognitive Neuroscience: teach courses in Memory & Thinking,
Sensation & Perception, and Behavioral Statistics (#3).
Experimental Analysis of Behavior: teach Learning, specialty courses in Behavioral
Psychology, and courses in our Statistics and Research Methods (6 courses/year) (#4)
Health Psychology (mechanisms, biomarkers): introductory Health Psychology, an advanced
research methods in Health Psychology, Analysis of Psychological Data (statistics) (#8)
Industrial-Organizational: with demonstrated excellence in teaching undergraduate research
methods and statistics courses (6 courses/year) (#9).
Sensory-Perception: S&P plus teach introductory and advanced courses in Statistics, as well
…Research Methods…willingness to engage in statistical consulting for constituents across the
campus and in the community (#10)
19
Leona Aiken, Arizona State University
Prevailing Model in Undergraduate
Statistics Instruction
Faculty member in a substantive discipline who is
also expected to teach statistics
FULL TIME TENURE TRACK: Bachelors + Masters
Clinical, counseling, or school psychology: teach introductory and advanced clinical,
counseling, or school psychology and contribute to the department’s core courses of …
statistics, or research methods, a demonstrated interest or experience in the teaching of
statistics is preferred (4 courses per year) (#7).
Clinical or counseling psychology: teach undergraduate and graduate clinical psychology
courses and supervise undergraduate internships, teach statistics, research methodology.
Experience working with graduate students and licensure or license-eligibility (#27)
Leona Aiken, Arizona State University
20
Prevailing Model in Statistics Instruction, Ph.D. Institutions
Individual Hired to Teach Advanced Quantitative Courses
beyond the First Year Course
Department require person to have quantitative research program
2%
Department prefer person to have quantitative research program
4%
Department have no preference re. substantive versus quantitative
19%
Department prefer person to have substantive research program
32%
Department require person to have substantive research program
43%
Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral Training in Statistics,
Measurement, and Methodology in Psychology: Replication and Extension of the
Aiken, West, Sechrest, and Reno (1990) Survey of Ph.D. programs in North America.
American Psychologist. 63(1), 32-50.
Leona Aiken, Arizona State University
21
Common Model in Quantitative Methods Hiring in Ph.D. Institutions
Back to the APS Jobs Data Base
Quantitative Methods (assistant prof) : significant expertise in state-of-the-art
statistical methods and engage in research that develops and applies such methods….
contribute to our existing areas of strength (#1)
Quantitative Methods (open rank): Bayesian methods, categorical and latent class
models, methods for high-dimensional data, machine learning, and/or robust
statistics; affiliations with the doctoral programs in I/O; Social/Health;
Developmental, Cognitive, Behavioral Neuroscience; Clinical (#5)
Quantitative Psychology (assistant prof): We prefer applicants whose specific
research interests complement those of our current faculty members, many of whom
have expertise in Human Factors, Industrial-Organizational, and/or Health
Psychology. (#15)
and the familiar reverse
Industrial/Organizational Psychology (open rank): PhD in I/O or closely related field
required. Individuals with a strong quantitative background are particularly
encouraged to apply (#31).
Leona Aiken, Arizona State University
22
Sonoran Desert, Arizona
Aiken, ASU
23
The assistant professor
quantitative position
Aiken, ASU
Sonoran Desert, Arizona
24
Abstraction of Job Ad for a
Quantitative Ph.D. Program hire
Expectations for an assistant professor:
Publish quantitative methods research
Make significant contributions to other substantive areas
Teach required undergraduate and graduate methods
Teach elective quantitative methods courses
Consult with and advise both faculty and graduate students
(a) in psychology and (b) in other departments
Even for a well trained quantitative psychologist hired
into a research department, where is the time to develop
teaching innovations for required undergraduate classes?
Leona Aiken, Arizona State University
25
A real concern for newly hired assistant professors,
particularly “solo quant” position
1. Pent up demand for statistical support when the individual is hired
2. No mentor (and protector) in the quantitative area
3. Heavy demands for curriculum innovation—new courses
4. Heavy demands for consulting with faculty and students
5. Same expectation for level of scholarly productivity as for others
at the same rank
Leona Aiken, Arizona State University
26
A real concern for newly hired assistant professors,
particularly “solo quant” position
A war story: graduating Ph.D. student , “twofer” with quantitative
and substantive expertise –offer letter:
(a) build research lab in substantive area
(b) innovate substantive and quantitative curriculum
(c) produce scholarships equal to his peers in other areas, and
(d) contribute 10 additional hours per week to statistical consulting
We directed him to ask the chair whether all assistant professors gave
10 hours per week to the good of the commons—(e.g. programming
for other faculty, cleaning cages, whatever would benefit the
commons) - end of 10 hours consulting requirement
Leona Aiken, Arizona State University
27
A real concern for newly hired assistant professors,
particularly “solo quant” position
Other examples from the field:
New assistant professor asked to develop 5 new quantitative
courses pre-tenure
New assistant professor told that his/her extensive consulting time
could be taken out of “research”—no teaching reduction, no
reduction in expectation for research productivity
“He/she was a great teacher, but really did not get his/her research
program going—often trailing author on colleagues’ publications”
Leona Aiken, Arizona State University
28
Concern as we think about teaching innovation
Our young hires are in a position to innovate in teaching
But, we can’t put the burden of teaching innovation
on our assistant professors without easing up
somewhere else
Is there a way to address this matter?
Leona Aiken, Arizona State University
29
Two New Models in Quantitative
Hiring in PhD programs
1. Teaching faculty
2. Cluster hiring with joint appointment in a
Statistics Department or a Quantitative
Methods Cluster/Institute
Leona Aiken, Arizona State University
30
Teaching Faculty Model
A lecturer hired to
teach the undergraduate quantitative curriculum
coordinate undergraduate statistics instruction
have expertise in a range of quantitative areas,
statistics, psychometrics
In the main, this is not a tenure-track or tenured position
Leona Aiken, Arizona State University
31
Teaching Faculty Model
“Master Teacher” of Introduction to Applied Statistics….tenured or tenurable
faculty member, Associate Professor level; special focus on introductory statistics;
lead and coordinate introductory undergraduate statistics instruction (6 courses per
year). University of Oklahoma, Psychology, Robert Terry, search chair (#2)
Continuing Lecturer in Research Methods: teaching and research expertise in
behavioral statistics, psychometrics, and/or multiple research methods, a strong
record of methodological/statistical consultation. Simon Fraser University,
Psychology, Rachel Fouladi, search chair
Assistant Professor of Practice, non-tenure track: primary focus on undergraduate
instruction; teacher-scholar, Ph.D. in Psychology, who can contribute to a wide
range of undergraduate instructional needs, including research methods and
statistics (#24)
Lecturer…Teaching Professor, non-tenure track, renewable: a long-term, key role in
the department’s educational mission through the design and teaching of the
research methods and statistics curriculum, Ph.D. in Psychology Cognitive Science,
Neuroscience, or related discipline (#28)
Leona Aiken, Arizona State University
(not a quantitative focus…., posted March 16, 2017)
32
Cluster hiring with joint appointment in a
Statistics Department or a
Quantitative Methods Cluster
Quantitative Research Institute and Department: Assistant professor, joint hire
between Institute and a social science department that will be the candidate’s
tenure track home. Teaching load distributed between Institute and the
Department. (In this case, the Institute had the first say in who would be hired,
not the Department). (#32)
Social science faculty developing and applying techniques at the research
frontier of statistics and data science; associate or full professor; those with
applications to anthropology, economics, linguistics, political science,
psychology, or sociology sought; half of teaching cross-listed in statistics. (#25)
My observation: these joint appointments are very complex positions for new
assistant professors; I think they should be reserved for higher ranked faculty
Hires must serve two masters. Joint appointments are not for the young.
Leona Aiken, Arizona State University
33
A wrinkle in thinking about improving
statistics and quantitative training
Both undergraduate and graduate Ph.D. programs farm out the
teaching of statistics to other departments
Various models at graduate level
First year sequence in Psychology, all else out
All quantitative teaching farmed out
Two Consequences
We have no control here over nature of instruction when
Instruction is in other departments
Statistics is taught without a context in the discipline of
Psychology
Leona Aiken,
Arizona State University
Adjuncts are hired to teach
statistics
34
Grand Canyon, AZ, Cliff Dwelling
Aiken, ASU
35
Goal 2.
Continued,
Explore labor
force for teaching
statistics and
quantitative
methods
Part b: Production of quantitative
psychologists
Aiken, ASU
36
Challenges with counting the number of quantitative psychologists
graduated in any year
1. Source: Graduate Study in Psychology, American Psychological Association
Some departments that do have a quantitative Ph.D. program do not
check the index terms that identify their quantitative programs, and so
do not appear in the Index.
Joan Freund, Ph.D. Office of Graduate and Postgraduate Education and
Training, APA, August 1, 2006, personal communication
1. Source: IPEDS (Integrated Postsecondary Education Data System)
“The IPEDS database is flawed because many of these graduates are
placed into a catch-all category called "Psychology, General."
Peggy Christidis, Ph.D. Senior Research Officer, Center for Workforce
Studies, Education Directorate, APA, March 14, 2017, personal communication
Leona Aiken, Arizona State University
37
Number of new PhDs in Quantitative Psychology, 2014-2015
University
Degrees
Awarded
2014-2015
University
Degrees
Awarded
2014-2015
Arizona State U
Boston College
U British Columbia
UC Davis
UCLA
UC Merced
Fordham U
U Illinois, Champaign
U Kansas
U Manitoba
U Minnesota
1
0
0
2
1
0
2
2
1
0
1
U New Mexico
U North Carolina
Notre Dame U
Ohio State U
Ohio U, Applied Quant
U Oklahoma
U Southern Calif.
Vanderbilt U
U Virginia
U Washington, Seattle
York U
0
4
2
4
0
1
0
2
1
0
0
U Missouri
NR
24 total degrees awarded ; 27 newly enrolled students; 176 total enrolled students
a. Source: Graduate Study in Psychology, 2017, APA Publications. Schools listed were as
included in the Index under Quantitative Psychology. Eleven additional schools were
listed in the Index but reported no quantitative program in the program narrative.
b. Educational Psychology or Education Departments excluded;
c. Some obvious omissions: McGill U., Simon Fraser U;
Leona Aiken, Arizona State University38
Arizona Desert Poppies
Aiken, ASU
39
Goal 4: Explore training resources for
innovation in quantitative
instruction
Are there training
resources for learning
novel approaches to
teaching quantitative
methods, such as
those presented today?
Leona Aiken, Arizona State University
40
American Statistical Association: Focus on Education
• “The American Statistical Association has identified education as a
strategic goal. The ASA works to provide leadership to improve
statistical education at all levels, build a statistically literate society,
make the study of statistics and statistical careers more attractive,
and enhance the preparation of statisticians for success.”
• Statistics education resources are provided for K–12 teachers and
teacher educators, undergraduate faculty and department chairs,
graduate faculty, statistics students, and statisticians in developing
countries. Here, you will find classroom resources, publications in
statistics education, guidelines and reports, professional
development resources, communities, competitions, programs, and
resources for students.
• http://www.amstat.org/asa/education/home.aspx?hkey=81d9c14250f7-4706-b8ce-7a2edcef2da4
Leona Aiken, Arizona State University
41
American Statistical Association, Example of
Education Activity
Leona Aiken, Arizona State University
42
Sources for Teaching Materials
Andrew Gellman and Deborah Nolan (2002).
Teaching Statistics: A Bag of Tricks.
“Teaching Statistics is an excellent resource to
help instructors create more interactive
introductory statistics classrooms. The book is
organized in three parts, with the first devoted to
activities for an intro course…. The second part
focuses on the nuts and bolts of incorporating
demonstrations while keeping the class on track,
including a sample syllabus.”
http://bokcenter.harvard.edu/blog/teachingstatistics-bag-tricks-andrew-gelman-anddeborah-nolan, accessed March 22, 2017
2nd Ed. Expected, August, 2017
Leona Aiken, Arizona State University
43
Online Resources
Dale Berger, Claremont Graduate School, WISE
http://wise.cgu.edu., demonstration applets
Examples for use in class
“538”, Nate Silver, examples from sports, politics
science, economics, culture
Leona Aiken, Arizona State University
44
So there are resources for enhancing statistics instruction
However, they are many and varied
and not organized
and not evaluated
This puts a substantial burden on the new instructor
of statistics-to integrate new methods into statistics instruction
Leona Aiken, Arizona State University
45
I am asking the diffusion of innovation question —
how creative methods for teaching statistics,
developed by quantitative experts, get out into the
world?
I add a second question:
Do faculty members with expertise in various
substantive areas of Psychology
have (1) the interest, (2) the time, (3) the energy, and
(4) the financial resources
to acquire these new methods.
In closing, I add a third question:
Leona Aiken, Arizona State University
46
How do we educate our students of quantitative
methods in Psychology, our doctoral students,
about how to teach statistics and quantitative
methods in Psychology?
This is the “next step” discussion I hope we have as we
close today.
Leona Aiken, Arizona State University
47
Thank
you
Arizona State Flower
Sugaro Cactus Blossom
Arizona State Bird
Cactus Wren
48
Aiken, ASU