Assessing quantitative skills
Ken Heller
School of Physics and Astronomy
University of Minnesota
Details at http://groups.physics.umn.edu/physed/
Supported in part by Department of Education (FIPSE), NSF,
and the University of Minnesota
Assessment
First step - Setting Goals
Skills
Mathematical
Logical
Organizational
Integration of Quantitative reasoning and
concepts
a broader knowledge base
Second step - Determining the Initial State of the Learner
Pre-tests
Third step - Determining the Final State of the Learner
Post-tests
Types of Assessment
• Skill Characterization
• Quality Assurance (QA)
• Task Progress
• Pattern Recognition
• Rubrics
Characterization of Initial Sample
Easy to administer and analyze
Skills used in the class
Math Diagnostic
30 question multiple choice test on math skills
Math Diagnostic Test
4  10 3
Powers of ten
10
4
(a) 4 x
=?
(b) 4  10
(d) 40 [51-63%]
3
4
(c) 4 [20-28%]
(e) 4 x 107
For this right triangle, cos q = ?
Triangles
g
a
2b
f
q
(a) 2b/3c (b) a/3c
(c) 2b/a [7-16%]
(d) 3c/a [69-89%]
(e) a/2b
3c
The slope of the curve pictured is equal to:
(a) 0 m/s
(b) 1/3 m/s [85-96%] (c) 2 m/s(d)
(d) 3 m/s [4-12%]
(e) 6 m/s
4
y (in meters)
Graphs
10-7 [10-20%]
3
2
1
0
0
1
2
3
t (in seconds)
4
5
6
Quality Assurance
Easy to administer and analyze
Correlates with desired learning
Force Concept Inventory (FCI) for Mechanics
30 question multiple choice test on concepts
FCI Question 17
(A)
(B)
(C)
(D)
(E)
An elevator is being lifted up an elevator
shaft at a constant speed by a steel cable, as
shown in the figure. All frictional effects are
negligible. In this situation, forces on the
elevator are such that:
Pre Post
the upward force by the cable is greater than 64 36
the downward force of gravity.
the upward force by the cable is equal to
18 60
the downward force of gravity.
the upward force by the cable is smaller than
2
0
the down ward force of gravity.
the upward force by the cable is greater than
the sum of the downward force of gravity and 11
2
a downward force due to the air.
None of the above. (The elevator goes up
because the cable is shortened, not because an
1
upward force is exerted on the elevator by the 5
cable).
D, F94
E, F94
F, F94
G, F94
H, F94
I, F95
J, F95
K, F95
L, F95
D, F95
M, F96
G, F96
N, F96
G, F96
O, F96
P, F97
O, F97
K, F97
D, F97
N, F97
P, F98
G, F98
O, F98
G, F98
N, F98
M, F99
O, F99
K, F99
Q, F99
M, F00
R, F00
Q, F00
N, F00
S, F00
T, F00
Z, F01
W, F01
N, F01
K, F01
U, F01
aA, F02
U, F02
K, F02
I, F02
X, F02
V, F03
Y, F03
X, F04
U, F04
Q, F04
Y, F04
aT, F05
aU, F05
aV, F05
FCI average score (%) +/- Standard Error
of mean
Fall 1251 / 1301 average pre & post FCI scores, by section
pre-test
FCI changes
post-test
90.0
80.0
Change to semesters
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Instructor, term
Pattern
Recognition Student Problem
Solutions
Initial State
Final State
Rubric for evaluating problem solving
• Physics Approach – deciding on an appropriate way to use
physics knowledge in the problem.
• Symbolic Translation of Physics Approach - deciding how to
translate the physics approach into symbols that can be used in
mathematics.
• Appropriate Mathematics – use of mathematical rules to reach
an answer
• Logical Progression – an organized progress from the problem
statement to an answer
Logical Progression (entire problem solution)
Nothing written can be interpreted as logical progression.
0
1
Haphazard solution with obvious logical breaks.
2
Part of the solution contradicts stated principles, the constraints shown in the
explicit statement of the problem situation, or the assumptions made in
another part of solution.
3
Solution is logical but does not achieve the target quantity or haphazard but
converges to the target.
4
Solution is organized but contains some logical breaks.
5
Progress from problem statement to an answer includes extraneous steps.
6
Consistent progress from problem statement to answer.
The End
Please visit our website
for more information:
http://groups.physics.umn.edu/physed/