Working with Student
Success Data
WHATCOM COMMUNITY COLLEGE
PROFESSIONAL DEVELOPMENT WORKSHOP
WINTER 2013
ED HARRI – DEAN FOR INSTRUCTION
ANNE MARIE KARLBERG – DIRECTOR OF ASSESSMENT AND
INSTITUTIONAL RESEARCH
Session 3 Outcomes
Session 3 Outcomes: Upon successful completion of
the third session, faculty will be able to…
3.
Use data to better know their students and develop
informed teaching strategies.
5.
Analyze data resulting from college outcomes
assessment process.
6.
Provide leadership in his/her discipline/department/
program in interpreting and using data.
7.
Write well-developed, data-informed professional
plans, which include benchmarks for success at the
individual and discipline/program level.
Discuss Homework #2
1.
Institutional or instructor gathered data
a. Share one piece of existing data (institutional data or instructor
gathered) from your discipline that you “feel good” about.
b. Share one piece of existing data (institutional data or instructor
gathered) from your discipline that you “are concerned” about.
c. What happened when you shared these items with a colleague?
2. Data question/request
a.What was challenging about developing the request?
b.How do you plan to use your data to improve your program/
discipline , your own teaching, and the success of your students?
c. If you gathered data on your own, what did you discover?
Data Request Form
How did the form work?
Suggestions for improvements?
Course Outcomes Data
Develop course outcomes
2. Develop rubrics
3. Now what?
1.
Course Outcomes Data
Develop course outcomes
2. Develop rubrics
3. Now what?
1.
Course Outcomes Data
Develop course outcomes
2. Develop rubrics
3. Now what?
1.
Course Outcomes Data
Develop course outcomes
2. Develop rubrics
3. Collect and analyze data!
1.
Course Outcomes Data
Develop course outcomes
2. Develop rubrics
3. Collect and analyze
data!
1.
Course Outcomes
Business 132: Students will be able to…
1. write a major research paper.
2. apply the principles of ethical and legal conduct in
business.
3. evaluate business problems from multiple perspectives.
4. communicate qualitative and quantitative analyses.
Course Outcome and Indicators
Indicators are the criteria that are used to assess a student’s
proficiency of a specific outcome. Indicators break the
outcome down into component parts. The indicators for each
outcome are listed in the rubric.
Course outcome: Students will be able to write a major
research paper.
Indicator a. Identifies the problem and its historical context.
Indicator b. Uses a variety of academic research articles and
sources.
Indicator c. Uses writing conventions.
Indicator d. Provides a thoughtful, in-depth analysis of the topic.
Rubric
Expanded Gradebook Analyses
Expanded Gradebook* – Individual Instructor
Student
Jennifer
Indicators for the course outcome
Uses
academic
Provides inIdentifies
research
Uses writing
depth
problem (4)
articles (4) conventions (4)
analyses (4)
Emiko
Thane
Lijun
Michael
Dan
Scale ranges from 1 to 4
1=Beginning
2=Developing
3=Accomplished
4=Exemplary
*An expanded grade book disaggregates rubric scores by indicator.
Course outcome: Students will be able to write a major research paper.
Expanded Gradebook – Individual Instructor
Student
Jennifer
Indicators for the course outcome
Uses
academic
Provides inIdentifies
research
Uses writing
depth
problem (4)
articles (4) conventions (4)
analyses (4)
4
2
3
4
Emiko
4
1
4
3
Thane
4
3
4
2
Lijun
3
2
3
4
Michael
3
1
1
3
Dan
1
1
2
4
Course outcome: Students will be able to write a major research paper.
Expanded Gradebook – Individual Instructor
Student
Jennifer
Indicators for the course outcome
Uses
academic
Provides inIdentifies
research
Uses writing
depth
problem (4)
articles (4) conventions (4)
analyses (4)
4
2
3
4
Total
13
Emiko
4
1
4
3
12
Thane
4
3
4
2
13
Lijun
3
2
3
4
12
Michael
3
1
1
3
8
Dan
1
1
2
4
8
3.2
1.7
2.8
3.3
11
Average
Course outcome: Students will be able to write a major research paper.
Outcomes Data Activity
In pairs, respond to the questions at the top of
page 2.
Expanded Gradebook – Individual Instructor
Student
Jennifer
Indicators for the course outcome
Uses
academic
Provides inIdentifies
research
Uses writing
depth
problem (4)
articles (4) conventions (4)
analyses (4)
4
2
3
4
Total
13
Emiko
4
1
4
3
12
Thane
4
3
4
2
13
Lijun
3
2
3
4
12
Michael
3
1
1
3
8
Dan
1
1
2
4
8
3.2
1.7
2.8
3.3
11
Average
Course outcome: Students will be able to write a major research paper.
Expanded Gradebook – Individual Instructor
Student
Jennifer
Indicators for the course outcome
Uses
academic
Provides inIdentifies
research
Uses writing
depth
problem (4)
articles (4) conventions (4)
analyses (4)
4
2
3
4
Total
13
Emiko
4
1
4
3
12
Thane
4
3
4
2
13
Lijun
3
2
3
4
12
Michael
3
1
1
3
8
Dan
1
1
2
4
8
3.2
1.7
2.8
3.3
11
Average
Course outcome: Students will be able to write a major research paper.
Expanded Gradebook – Multiple
Sections/Instructors
Section
1
2
3
4
Overall
average
Average for each indicator
Uses
academic Uses writing Provides in# of Identifies
research conventions
depth
students problem (4) articles (4)
(4) analyses (4) Avg
30
3.2
1.7
3.8
3.3 12.0
29
2.2
1.5
2.9
3.1 9.7
28
3.1
1.6
2.7
30
117
3.4
3.0
1.8
1.7
2.9
3.1
=([30x3.2]+[29x2.2]+[28x3.1]+[30x3.4)]/117
3.4 10.8
3.3 11.4
3.3 11.0
Summary Table Analyses
Individual Instructor Summary Table
(Activity sheet, bottom page 2)
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
Exemplary Accomplished
(4)
(3)
3
Developing
(2)
Average
Beginning Indicator
(1)
Total
6
0
6
c. Uses writing
conventions
d. Provides indepth analyses
2
6
3
6
Total
8
7
4
5
24
Course outcome: Students will be able to write a major research paper.
Individual Instructor Summary Table
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
Exemplary Accomplished
(4)
(3)
3
2
Developing
(2)
0
Average
Beginning Indicator
(1)
Total
1
6
0
1
2
3
6
c. Uses writing
conventions
d. Provides indepth analyses
2
2
1
1
6
3
2
1
0
6
Total
8
7
4
5
24
Course outcome: Students will be able to write a major research paper.
Multiple Sections/Instructors Summary Table
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
c. Uses writing
conventions
d. Provides indepth analyses
Total
Exemplary Accomplished
(4)
(3)
30
35
Developing
(2)
25
Average
Beginning Indicator
(1)
Total
15
105
15
33
35
22
105
36
50
15
4
105
20
28
42
15
105
101
146
117
56
420
Course outcome: Students will be able to write a major research paper.
Multiple Sections/Instructors
(Activity sheet, p. 3)
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
c. Uses writing
conventions
d. Provides indepth analyses
Exemplary Accomplished
(4)
(3)
30
35
Developing
(2)
25
Average
Beginning Indicator
(1)
Score
15
2.76
15
33
35
22
2.39
36
50
15
4
3.12
20
28
42
15
2.50
Average Outcome 10.78/4
Score
=2.70
Course outcome: Students will be able to write a major research paper.
Multiple Sections/Instructors
Average Indicator Scores
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
c. Uses writing
conventions
d. Provides indepth analyses
Exemplary Accomplished
(4)
(3)
30
35
Developing
(2)
25
Average
Beginning Indicator
(1)
Score
15
2.76
15
33
35
22
2.39
36
50
15
4
3.12
20
28
42
15
2.50
Average Outcome 10.78/4
Score
=2.70
Course outcome: Students will be able to write a major research paper.
Multiple Sections/Instructors
Average Indicator Scores
Number of Students
Indicator
a. Identifies
problem
b. Uses academic
research
articles
c. Uses writing
conventions
d. Provides indepth analyses
Exemplary Accomplished
(4)
(3)
30
35
Developing
(2)
25
Average
Beginning Indicator
(1)
Score
15
2.76
15
33
35
22
2.39
36
50
15
4
3.12
20
28
42
15
2.50
Average Outcome 10.78/4
Score
=2.70
Course outcome: Students will be able to write a major research paper.
Multiple Sections/Instructors
Performance Level Percentages (page 4)
Indicator
a. Identifies
problem
b. Uses
academic
research
articles
c. Uses writing
conventions
d. Provides indepth
analyses
Average
%
# of Exemplary
Students
(4)
105
29%
% of Students
%
%
%
Accomplished Developing Beginning Total
(3)
(2)
(1)
%
33%
24%
14% 100%
105
14%
31%
33%
21% 100%
105
34%
48%
14%
4% 100%
105
19%
27%
40%
14% 100%
24%
35%
28%
13%
Course outcome: Students will be able to write a major research paper.
Looking at Multiple Course Outcomes
COURSE OUTCOMES: Students will be able to . . .
Average student
score (out of 4)
1. Solve linear equations and inequalities of one variable.
3.0
2. Solve application problems involving linear equations.
1.8
3. Graph linear equations in two variables in the Cartesian plane.
2.8
4. Write a linear equation of two variables in slope-intercept,
point-slope, or standard form.
2.7
5. Solve systems of linear equations of two variables graphically,
by substitution, and by elimination methods.
2.6
6. Simplify exponential expressions by applying rules of
exponents.
2.9
7. Add, subtract, multiply, and divide polynomial expressions.
3.1
Discussion Questions about Outcomes Data
What do you notice when observing the data?
How would you respond?
What would your next teaching steps be?
What would your next data steps be?
Outcomes versus Grades
(Activity sheet, page 5)
History 225: American Civil War (389 students)
Average student
score (out of 4)
Course outcomes: Students will be able to . . .
1. Summarize the major events leading to the American Civil War.
2. Identify the major battles of the American Civil War.
3. Describe major social and political issues existing today as a result of the
American Civil War.
2.1
2.4
1.7
Successful Course Completion Rate: 81% of students complete the
course with a C or better.
Grade Distribution
A/A-
B+/B/B-
C+/C
C-/D+/D/D-
F
W
46%
28%
7%
5%
4%
10%
Discussion Questions about Outcomes Data
What do you notice when observing the data?
How would you respond?
What would your next teaching steps be?
What would your next data steps be?
Professional Plans
What type of goals might be appropriate for your
professional plans?
Professional Plans
Professional Plans
I.
Goals in Teaching Effectiveness/Professional Effectiveness in
Instructional Support
Over the past year, 82% of my GERM& 101 students demonstrated that they
were performing at least at the “accomplished” level on outcomes 1-3 of the
course. Only 42% of my students were performing at this level on Outcome #4
in the same course. I will work to…




Research practices for presenting and assessing material related to this outcome
Develop new lesson plans for this material
Provide formative assessments related to this outcome to obtain early feedback from students
Discuss and work with other German faculty, since this problem is consistent across all sections of
the course
Over the past year, 85% of my GERM& 223 students demonstrated that they
were performing at least at “accomplished” level on all outcomes of the
course. The successful completion rate, however, was only 72% of students.
Upon further reflection, none of the course outcomes identified higher level
skills appropriate to a 200-level course, while I continue to evaluate these
skills in assignments. As a result, I will revise course outcomes and indicators
to identify the higher level skills expected of students in the course.
Professional Plans
I.
Goals in Teaching Effectiveness/Professional Effectiveness in
Instructional Support
Over the past year, 82% of my GERM& 101 students demonstrated that they
were performing at least at the “accomplished” level on outcomes 1-3 of the
course. Only 42% of my students were performing at this level on Outcome #4
in the same course. I will work to…




Research practices for presenting and assessing material related to this outcome
Develop new lesson plans for this material
Provide formative assessments related to this outcome to obtain early feedback from students
Discuss and work with other German faculty, since this problem is consistent across all sections of
the course
Over the past year, 85% of my GERM& 223 students demonstrated that they
were performing at least at “accomplished” level on all outcomes of the
course. The successful completion rate, however, was only 72% of students.
Upon further reflection, none of the course outcomes identified higher level
skills appropriate to a 200-level course, while I continue to evaluate these
skills in assignments. As a result, I will revise course outcomes and indicators
to identify the higher level skills expected of students in the course.
Discipline/Departmental/Program/Divisional
Work Plans
What type of goals might be appropriate for these?
Homework #3
 Examine your data.
 Set up a meeting with Anne Marie to help with pivot tables,
filtering, etc.
 Consider different presentation approaches for your data.
 Develop an interpretation of your data.
 Share with other colleagues and confirm or adjust
interpretation.
 Share the data with your discipline/program colleagues.
What is their reaction? What are their questions? How will
they use it? Any interpretations that surfaced when others
looked at the data that you didn’t arrive at on your own?
Questions?
Rubric – with weighting
Weighted Rubric Example
Number of Students
Weighted
Score=
Indicator
a.
a.
Identifies
problem
Uses
academic
research
articles
Accomplished Developing
(3)
(2)
Average
Beginning Indicator Average X
(1)
Score
Weight
Weight
Exemplary
(4)
10%
30
35
25
15
2.76
0.276
20%
15
33
35
22
2.39
0.478
a.
Uses
writing
conventio
ns
40%
36
50
15
4
3.12
1.248
a.
Provides
in-depth
analyses
30%
20
28
42
15
2.50
0.75
Average Weighted
Outcome Score
2.75
Course outcome: Students will be able to write a major research paper.