Students and Faculty
Perspectives: Are We Engaged
Yet?
Presented by
Yuhfen Diana Wu
Dr. Martin Luther King, Jr. Library
San Jose State University
December 3, 2007
CARL-N Mini-conference
Teacher-Scholar Cohort of 2006-07
• Emily Allen, Engineering
• Joanne Rossi Becker, Science
• Elizabeth Cara, Applied Sciences and Arts
• Peg Hughes, Education
• Arvinder Loomba, Business
• Linda Mitchell, Humanities and the Arts
• Ronald Rogers, Social Sciences
• Diana Wu, University Library
CARL-N Mini-conference
Outline
• Genesis / Purpose of Research
• Student Engagement Defined
• Research Questions
• Survey Methods / CLASSE Pilot Instrument
• Analysis of Data
• Lessons Learned
• Future Research Directions
CARL-N Mini-conference
Genesis / Purpose of Research
• Purpose
− To understand university faculty members' and
students’ perspectives regarding multiple facets of
student engagement.
• Importance
− To monitor student engagement, a students’
“connectedness,” and sense of belonging is very
important because these factors individually and
collectively contribute to student success.
CARL-N Mini-conference
Definition of Student Engagement
• Student engagement depicts students’ willingness to participate
in routine school activities, such as attending classes, submitting
required work, and following teachers’ directions in class
(Nystrand & Gamoran, 1992).
• Motivated behavior is indexed by the kinds of cognitive strategies
students choose to use and by their willingness to persist with
difficult tasks by regulating their own learning behavior (Pintrich
& De Groot, 1990; Pintrich & Schrauben, 1992).
CARL-N Mini-conference
NSSE Instrument
• The National Survey of Student Engagement is a "best
practices" survey. The survey asks students to report
on how they choose to spend their time while in
college, recognizing that some uses of time produce
more benefits to students than others. The five NSSE
benchmarks are aggregations of clusters of survey
items and, as such, are defined by those clusters.
−
−
−
−
−
Level of Academic Challenge
Active and Collaborative Learning
Student Interactions with Faculty
Enriching Educational Experiences
Supportive Campus Environment
CARL-N Mini-conference
Background Literature
• Much of the research on student engagement has been
conducted at the high school and elementary school
levels; little exists on the university level (CLASSE still
in pilot stage).
• Much of the existing research is from the perspectives
of the students; faculty perspectives are relatively
unexplored.
• Little of the research investigates faculty perceptions
of the linkage between social characteristics (such as
race and ethnicity, class, and gender) and the degree
of student engagement.
CARL-N Mini-conference
Research Questions
1. What do faculty rate as important in a class?
2. Are there differences between faculty ratings in online and face-to-face classes?
3. Are there differences in student-reported
performance between on-line and face-to-face
classes?
4. Can such differences be accounted for by
demographic variables such as native language?
CARL-N Mini-conference
Survey Methods – the CLASSE
Survey
• A two-component tool that compares
faculty expectations with what students
report experiencing in a class.
− CLASSE faculty
− CLASSE student
• Based on National Survey of Student
Engagement (NSSE) supported by CIBER
at Indiana University.
CARL-N Mini-conference
Survey Question Format
• Survey has 38 questions distributed among
domains
−
−
−
−
Engagement Activities
Cognitive Skills
Other Educational Practices
Class Atmosphere
• 4-point Likert scale on all items
• Additional demographic items for both faculty
and students
CARL-N Mini-conference
Survey Question Format
• Faculty Survey
− Faculty rate the importance of each item
− Score is high if the item is important for
student to be successful
• Student Survey
− Students rate frequency or amount of
performance
− Score is high if they have higher
performance on this item
CARL-N Mini-conference
Engagement Study
Data Analysis
CARL-N Mini-conference
Interpreting our Results
• Comparison between faculty and student
scores
− Difference Scores
− Quadrant Analysis
• Differences between the item rankings
of Online and Face-to-face Faculty
CARL-N Mini-conference
Interpreting our Results:
Difference Scores
• Calculation:
− Teacher Rating minus Student Rating
• What do the differences mean?
− Positive Differences
• Teacher rates item higher than students
− Negative Differences
• Students rate item higher than teachers
CARL-N Mini-conference
Difference Scores: All Classes
Differences Between Faculty Perceptions and Student
Performance
(Across All 16 Courses)
Cognitive Skills
2.00
29
6
4
1.50
22
15
Delta Score
1.00
0.50
Class
Atmosphere
1
8
5
2
7
10
9
13
1112
3
18
21
17
16
19
28
24
23 252627
33
31
30 32
36
35
38
34
0.00
14
20
37
-0.50
-1.00
Other Ed Practices
Engagement
Activities
-1.50
-2.00
Question Number
CARL-N Mini-conference
Largest Positive Difference
(Engagement):
• How many times have you
Question #4:
“Worked on a paper or a project in your [Course XYZ] class
that required integrating ideas or information from various
sources?”
• Implies that students don’t recognize when they are “integrating
ideas or information from various sources,” or
• Faculty are not actually requiring this of their students despite
considering it important
Question #6:
“Came to your [Course XYZ] class without having
completed readings or assignments?”
• Reversed item: Actually implies that students try to come to
class prepared
CARL-N Mini-conference
Largest Positive Difference
(Engagement):
• How many times have you
Questions #15:
“Made a class presentation in your [Course XYZ]
class?”
Implies…
• Students perhaps don’t recognize when they are making a
“class presentation,” or
• Faculty don’t assign as many as they think are important,
or
• The presentations are concentrated at end of semester,
i.e., after our data collection
CARL-N Mini-conference
Largest Positive Difference
(Cognitive Skills):
• So far this semester, how much of your coursework in your
[Course XYZ] class emphasized the following mental activities?
Question #22:
“Synthesizing and organizing ideas, information, or
experiences into new, more complex interpretations and
relationships”
• Difference implies that..
• Students don’t recognize when they are “Synthesizing and
organizing ideas, information, or experiences,” or
• Students actually aren’t performing this in class
CARL-N Mini-conference
Interpreting
our Results:
Quadrant
Analysis
CARL-N Mini-conference
How faculty rank the importance
of different aspects of the course
• Next slides show questions divided by
domains
− Engagement Activities Q1-Q19
− Cognitive Skills Q20-Q24
− Other Educational Practices Q25-Q34
− Class Atmosphere Q35-Q38
CARL-N Mini-conference
Faculty Ranking of Importance
of Engagement Activities
(average ranking of all faculty in group)
4.5
4.0
3.5
3.0
2.5
FtF
OnLine
2.0
1.5
1.0
0.5
0.0
Q
No.
1
2
3
4
5
6
7
8
9
10
11
12
CARL-N Mini-conference
13
14
15
16
17
18
19
Questions with Significant Mode Differences
(F-to-F ranks higher than OL)
• How important is it that students in your
class…
Question #7: “Work with other students on projects
during your [Course XYZ] class”
Question #8: “Work with classmates outside of your
[Course XYZ] class to prepare class assignments”
Question #15: “Make a class presentation in your
[Course XYZ] class?”
CARL-N Mini-conference
Questions with Significant Mode Differences
(OL ranks higher than F-to-F)
• How important is it that students in your
class…
Question 11: “Use an electronic medium (list-serv,
chat group, Internet, instant messaging, etc.) to
discuss or complete an assignment in your [Course
XYZ] class”
Question #12: “Use email to communicate with you as
the instructor of your [Course XYZ] class”
CARL-N Mini-conference
Faculty Ranking of Importance of Cognitive Skills
(average ranking of all faculty in group)
4.5
4.0
3.5
3.0
2.5
FtF
2.0
OnLine
1.5
1.0
0.5
0.0
Q No.
20
21
22
CARL-N Mini-conference
23
24
Faculty Ranking - Importance of Other Educational Practices
(average ranking of all faculty in group)
4.0
3.5
3.0
2.5
FtF
2.0
OnLine
1.5
1.0
0.5
0.0
Q No.
25
26
27
28
29
30
CARL-N Mini-conference
31
32
33
34
Summary of Lessons Learned
1.
Very useful tool for faculty as formative assessment
2.
SJSU strength is approachability of faculty
3.
Surprising differences in terms of faculty
expectations in on-line vs. face-to-face
4.
Low response rate from on-line students
5.
More on-line faculty declined to participate (citing
#4 above)
CARL-N Mini-conference
Future Research Directions
• Engage proactively by academic librarian
• Administer survey in more classes next year
• Look for statistical significance
• Use for formative assessment
• Examine demographic data
• Revise survey for on-line classes
CARL-N Mini-conference
THANK YOU
…
Questions!!!
CARL-N Mini-conference