Department of Statistics
What Researchers Need to Know
about the
Statistical Consulting Process
or
Working with the Statistician
as a
Research Colleague
Walt Stroup
Professor & Chair, UNL Department of Statistics
1 December 2006
SSP Core Facility
Department of Statistics
Outline for Talk
I. Role of a Statistician
II. Statistical View of Research Process

From planning to finished product
III. Essential features of an effective partnership

What is expected of you; what you should expect
IV. What can go wrong & what to do about it
1 December 2006
SSP Core Facility
Department of Statistics
Cool Quote – Source Unknown
 Writers tell stories with words
 Painters tell stories with pictures
 Musicians tell stories with music
 Statisticians tell stories with numbers
However
There must be a story to tell
or the words, pictures, music,
numbers --- and sophisticated methodology --are “...full of sound and fury, signifying nothing...”
1 December 2006
SSP Core Facility
Department of Statistics
So...Statisticians:
The object of consulting is
The object of consulting collaboration
is
The Science
not
the math
nor
the statistical techniques
1 December 2006
SSP Core Facility
Department of Statistics
Cool Quote – Source Unknown
 There are three subjects everybody thinks
they can be an expert in with no formal
training:
− Law
− Medicine
− Statistics
NOT !!!
1 December 2006
SSP Core Facility
Department of Statistics
Role of Statistics in Research
 Another cool quote, source also unknown:
 “Think of ’Statistics’ as ‘the rules of evidence
for scientific inquiry’”
 Webster: “Statistics is a branch of
mathematics...”
 Better: Oxford English Dictionary – “Statistics
is the science of data collection, analysis,
and reporting for the purposes of
characterizing a population, testing a theory,
or making a decision
1 December 2006
SSP Core Facility
Department of Statistics
Some Influences
My Major Professor: R.L. Anderson
“This is the worst data set I ever saw!
Why didn’t they talk to me first? Unbelievable!”
Gerald 90% of the value of statistics is in what happens
before any data are collected
Hahn
When people see me after the fact I often feel that
the only thing I can do is write the death certificate
“Don’t argue – take data”
“You need to learn to think of yourself not as
a statistical consultant but as a
research consultant”
1 December 2006
SSP Core Facility
Department of Statistics
E. O. Wilson
not the exact quote, but true to spirit:
What sets good research apart is
usually not one’s cleverness in finding
the answer but one’s skill in
•asking the right question, and
•phrasing it carefully and well
to which I would add:
“...and collecting the right data...”
1 December 2006
SSP Core Facility
Theme?
Planning!
Department of Statistics
1 December 2006
SSP Core Facility
Department of Statistics
Steps in Research Planning
Protocol Description
What is the Research Question?
Answer
Data analysis
?
Conclusions
New questions
Relevant Evidence
to answer question
(response variables
Collect data
How much evidence?
(sample size?)
1 December 2006
How will evidence
be collected – design?
(clinical trial model?
sample survey? )
SSP Core Facility
How will evidence be
used to answer
(hypothesis test?)
Department of Statistics
When is this Planning Applicable?
 Grant writing
 Preparing a dissertation proposal
 Journal Publication
 Informing Policy Decisions
1 December 2006
SSP Core Facility
Department of Statistics
late Senator Daniel Patrick Moynihan
we in this nation face a choice
we can inform public policy
with
sound research
or
we can continue
to make policy on the basis of
data-free ideology
1 December 2006
SSP Core Facility
Department of Statistics
Another of “Walt’s mentors”: Dal Kratzer
 defined statistics as “experimentation science”
 Protocol description form
 Intended for pharmaceutical research &
development
 With word-smithing and tweaking, readily
applicable as a template for research planning
1 December 2006
SSP Core Facility
Department of Statistics
Protocol Description Form
What are the objectives – these must be translated into specific questions
that can be answered with data
Key: operating definitions
What is the relevant evidence? Distinguish between essential variables
and auxilliary variables
What are the measurement units? This implied defining the target
population & how you intends to sample / observe it
What is the treatment design?
Experiment design?
Power analysis – how many observations do I need
Murphy’s Law – what can go wrong? what will I do?
How will I analyze the data? How does this relate to objectives?
Plan for reporting? Target audience? Intended purpose
1 December 2006
SSP Core Facility
Department of Statistics
Working with Statistician
 Haste makes waste
 Build a relationship
 Plan to educate statistician about your research
− Two way street
− You’re not a statistical expert; the statistician is not an
expert in your discipline
 “De-jargonization”
1 December 2006
SSP Core Facility
Department of Statistics
What you don’t want
1 December 2006
SSP Core Facility
Department of Statistics
Types of statistical investigations
 “Data mining”
− characterization, “fishing expedition”
− looking for patterns that “might” be there
− latent class, factor analysis, other multivariate methods
 Associations
− “cross-tabs”, log-linear models, path analysis
− often based on survey or retrospective data
 Treatment or factor effect
− based on experiment design or quasi-design
− “clinical trial” model; closest thing to cause & effect
1 December 2006
SSP Core Facility
Department of Statistics
Essential Parts of the Process - I
 What is the research question?
− It is easy to be vague
− It is easy to be vague when you think you aren’t
− Operating definitions are essential
 Is “Math in the Middle” effective?
− what do you mean by “effective”?
− compared to what?
− once “effective” has an operating definition, can you
measure it? how?
− if you can’t, you don’t have an operating definition yet
1 December 2006
SSP Core Facility
Department of Statistics
Essential Parts of the Process - II
 Power
 First
− what is the population?
− what elements represent the population?
− how do you (ethically) observe the elements?
 Then
− what are the relevant sources of variation among the
elements?
− what is the magnitude of variation assoc with these
sources?
− how big an effect is considered important?
1 December 2006
SSP Core Facility
Department of Statistics
What power analysis is NOT
1 December 2006
SSP Core Facility
Department of Statistics
Essential Parts of the Process - III
 Power
 Should be a dialog
 At its best
− mutual learning
− dress rehearsal for data collection and analysis
 INCLUDE: what can go wrong & what is my
plan?
− missing data happen
− missing data are inevitable
1 December 2006
SSP Core Facility
Department of Statistics
Essential Parts of the Process - IV
 Stay away from jargon and “discipline-speak”
− applies to you
− applies to statistician
 Ask questions
− don’t let “stat-speak” go by
− it’s not your ignorance, it’s the statistician’s language skills
 Answer questions
− if a good statistician asks you a question, it’s because she is
curious & trying to get herself in the best position to be
helpful
− questions are not a hidden agenda to find fault or
embarrass you
1 December 2006
SSP Core Facility
Department of Statistics
Essential Parts of the Process - V
 If possible, face-to-face meetings are better
 Haste makes waste
− last minute, rush-job consulting invites trouble
 Another “mentor quote”
− “we don’t have time to do it right, but we have plenty of
time to do it over”
EGO
1 December 2006
SSP Core Facility
Department of Statistics
Analysis
 Vastly easier in well-designed study
 Vastly better if statistician has been involved all
along
 Parsimony
− if two analyses show essentially same results and plausibly
satisfy assumptions, the simpler analysis is better
− 90% rule
 However
− just because it is traditional does not make it right
− computer has had huge impact on statistical methods
1 December 2006
SSP Core Facility
Department of Statistics
Analysis & Reporting
 Part of statistician’s job is to help write report
− material & methods:
 statistical design
 analysis
− result and discussion
 help explain and interpret
 Statistical science itself is a work in progress
− your project may motivate statistical research
− methods may be unfamiliar to reviewers
− statistician may need to educate referees / editors
1 December 2006
SSP Core Facility
Department of Statistics
Etiquette & Ethics
 Authorship
− would research / grant have been possible without
statistician’s efforts?
− NO means a co-authorship should be offered
 Funding
− external funding is lifeblood of research university
− statistician’s dilemma in academia:
− “do I close my door to colleagues in allied disciplines to
write my own grants, or do I participate in
interdisciplinary teams?”
− for latter, co-PI grant arrangements with real $$$ essential
1 December 2006
SSP Core Facility
Department of Statistics
More Etiquette & Ethics
 Special Note on Graduate (Dissertation) Research
 Common Unfortunate Situation: Graduate
Student as Ping-Pong Ball
 When statistician advises graduate student on
design and/or analysis (ideally, “and” – ed.)
student, student’s major professor, and
statistician should be on same page
 Ideal – statistician on PhD committee
 But at least – meet face to face periodically
1 December 2006
SSP Core Facility
Department of Statistics
Final Thought
 Most Statistics departments are seriously
understaffed – UNL no exception
 Not a complaint – just a reality
 Conflicting demands
− teaching
− research
− consulting
 Applied statistics need interdisciplinary work –
just as quality research needs engaged statistics
 BUT – you may have to be a nag
1 December 2006
SSP Core Facility
that’s
all
thanks
Any
Questions?