Six Big Ideas: Implications for Data
Collection and Use
Foundations of Excellence Winter Meeting
February 2013
Joyce Romano – Vice President, Student Affairs, Valencia College
Kurt Ewen – AVP, Institutional Effectiveness and Planning, Valencia College
Lesson 1
A “culture of evidence” requires the
development of an institutional practice that
1. gives careful consideration to the question
being asked
2. gives careful consideration to the data
needed in order to answer the question.
A genuine “culture of evidence” is dependent
on a “culture of inquiry”
What is a Culture of Inquiry?
•Institutional capacity for
supporting open, honest and
collaborative dialog focused on
strengthening the institution
and the outcomes of its
students.
Lesson 2
• Structured reflection and dialogue allows
for data to be transformed into meaningful
(actionable) information
• The “meaning” of data in Higher Education is
not generally self-evident and requires the
benefit of the intersection of multiple
perspectives
Lesson 3
• Meaningful information promotes
consensus about lessons learned and a
shared vision / plan for the future
• No data should be shared as information until it has been
processed in a collaborative and thoughtful way.
Lesson 4
• A culture of evidence is one that seeks data
supported decisions.
• Data driven decision making runs the risk of over
looking / underestimating the human factor
which is very often concealed by the desire for
statistical significance
• Data driven decision making runs the risk of
underestimating the role / significance /
importance of evidence informed hunches to
inform our decisions
Lesson 4 (con’t)
• Data supported decisions concerning student
success oriented programs requires a consideration
of "meaningful improvement" and may require
balancing all or most of the following:
• Statistically significant improvement in target measures.
• Reflection on the “human impact”
• Economic efficiency in relationship to difficulty of the task
at hand.
• A consideration of perception as it relates to benefit versus
cost.
Lesson 5
• Meaningful information from data does not
generally emerge from a single data point but
from the intersection of multiple and varied
(quantitative and qualitative) sources of data
• There is rarely a silver bullet (and if there
is, then the question being asked is
probably not particularly interesting)
The vital role of conversation
●
In order to make data useful, ample time and space are
needed to discuss and analyze the information and
connect it back to the original research question.
●
Answers are not always immediately apparent,
so skilled facilitation may be needed
to dig out the deeper meaning.
●
Multiple perspectives and
types of information are often
needed to make sense of
individual data points.
!
Dialogue vs. Discussion
An Etymological distinction
Dialogue:
• Seeing the whole among the parts
• Seeing the connections between
parts
• Inquiring into assumptions
• Learning through inquiry and
disclosure
• Creating shared meaning
Discussion:
• Breaking issues / problems into parts
• Seeing distinctions between parts
• Justifying / defending assumptions
• Gaining agreement on one meaning /
result
Lesson 6
• The best insights often come as an unintended
result of simple questions asked about things
you were not planning to question.
Lesson 6 (An Example)
• How are our new students doing?
• Data was provide on FTIC Degree Seeking
Students
• Who are our new students?
• Development of our Philosophy statement on
the New Student
• All Students with less than 15 College-level credits at Valencia
• Who are our new Students?
Who are our New Students?
How are our New Students Doing?
National Institute
for Learning
Outcomes
Assessment
(NILOA)
“From Gathering to
Using Assessment
Results: Lessons from
the Wabash National
Study”
From Gathering to Using Assessment Results
• “Most institutions have routinized data collection,
but they have little experience in reviewing and
making sense of data. It is far easier to sign up for
a survey offered by an outside entity or to have an
associate dean interview exiting students than to
orchestrate a series of complex conversations with
different groups on campus about what the
findings from these data mean and what actions
might follow.”
Contact Information
Joyce Romano
Vice President, Student Affairs
[email protected]
Kurt Ewen
Assistant Vice President, Institutional Effectiveness
and Planning
[email protected]