Overview
Entry Assessment Testing as a
Predictor of Student Success
†
†
presented by
†
Cindy James
†
Background Information
Introduction to the Study
Findings
Conclusions
Assessment Centre Coordinator
University College of the Cariboo
Kamloops, British Columbia, Canada
[email protected]
Background
The University College of the Cariboo (UCC) is a
public educational institute offering both college
and university programs.
†
†
The main campus is
located in Kamloops,
British Columbia,
Canada.
Seven regional
offices/campuses within
the central and southern
interior of B.C.
UCC Enrollment (2002/2003)
University Degrees
5575
Diplomas
1387
Vocational
1462
Developmental
1657
Totals
10081
1
UCC Assessment Centre
Purpose:
† To provide general educational assessments to
facilitate appropriate placement in courses/programs.
† To administer entry assessments for admission to
various UCC programs.
† To coordinate and/or invigilate examinations for
other educational institutions and outside agencies.
† Administered/Coordinate ~ 3500 assessments/tests
per year
Accuplacer Tests
†
†
†
†
†
†
†
Reading Comprehension
Sentence Skills
Arithmetic
Elementary Algebra
College Level Mathematics
WritePlacer Plus & WritePlacer ESL
Language of English Proficiency tests
Accuplacer
†
†
†
†
†
WEB-based computer testing system
Utilizes Adaptive Technology
Customized Testing
Computer scored
Built-in Database
UCC College Preparation
Purpose
Offers adult learners the opportunity to complete
prerequisites for admission into career, vocational and
academic programs and/or to complete an ABE diploma.
Courses (offered year round)
English, Math, General Science, Biology, Physics,
Chemistry, Student Success, Psychology, Computing and
Social Science.
Placement
†
Self – select for lower level courses (040 & 050)
†
Pre-requisites for the higher level courses (060)
2
The Study
Study Timeline
The Big Question
To improve the placement procedure and hence the
success rate for College Preparation courses and
programs, should entry assessment testing be
mandatory for College Preparation applicants?
The Consequential Question
Can Accuplacer Tests Scores Predict Student
Success/Nonsuccess in College Preparation
Courses?
Spring/Summer session 2003
Study Participants
Study Cohorts
†
Accuplacer tests administered to 6 different classes
of English and Math students on the first day of
classes.
Classes met on a daily basis for 7 weeks
(total of 75 hours of instruction).
Final Grades collected at the end of each course.
†
Data analysis conducted during the Fall 2003.
†
†
Math
Statistics
Total
Sample size (n)
93
Females
48
Males
45
Average Age
27
English
# of
participants
Cohort
# of
participants
Math 040
11
English 040
10
Math 050
15
English 050
7
Math 060
34
English 060
16
Total
60
Total
33
Cohort
3
Success Rates
Regression Analysis Utilized
Statistics
†
Total
Successful Students
To evaluate the linear relationship between the
predictor variables (Accuplacer test scores) and the
outcome (final grade).
84
(final grade ≥ 50%)
NonSuccessful Students
†
9
Logistic Regression Analysis
To evaluate the relationship between the predictor
variables (Accuplacer test scores) and the
dichotomous outcome (success vs nonsuccess).
(final grade < 50%, DNC, Withdrew)
Success Rate
Linear & Multiple Regression Analysis
90 %
Linear & Multiple Regression Analysis
Math Cohort
Linear Regression Analysis
Final Math Grade as a Function of Arithmetic Test Scores
Pearson’s Correlation Coefficient (r)
110
Accuplacer Test Scores
Final Grade
Math 040
(n = 10)
Math 050
(n = 13)
Math 060
(n = 32)
Arithmetic
Algebra
Both Tests
0.460
0.303
0.469
0.290
0.392
0.397
Final Grade in Math Course (%)
100
90
80
70
Math 060
60
Rsq = 0.1528
50
Math 050
Rsq = 0.0841
40
Math 040
0.391
0.617
0.619
30
20
Rsq = 0.2117
40
60
80
100
120
Accuplacer Arithmetic Test Score
4
Linear & Multiple Regression Analysis
Linear Regression Analysis
Final Math Grade as a Function of Algebra Test Scores
English Cohort
Pearson’s Correlation Coefficient (r)
110
Accuplacer Test Scores
90
80
Final Grade
70
60
50
Math 060
English 040
Rsq = 0.3806
(n = 10)
Math 050
Rsq = 0.1533
English 050
Math 040
(n = 6)
40
30
Rsq = 0.0916
20
40
60
80
100
120
Sentence
Skills
Both Tests
0.285
0.641
0.669
0.071
0.143
0.172
0.508
0.563
0.575
English 060
Linear Regression Analysis
Linear Regression Analysis
Final English Grade as a Function of Reading Comprehension Test Scores
Final Grade in English Course (%)
Reading
Comprehension
(n = 15)
Accuplacer Elementary Algebra Test Scores
Final English Grade as a Function of Sentence Skills Test Scores
100
100
90
90
80
70
English 060
Rsq = 0.2580
English 050
60
Rsq = 0.0050
Final Grade in English (%)
Final Grade in Math (%)
100
80
70
English 060
Rsq = 0.3169
English 050
60
Rsq = 0.0206
English 040
50
Rsq = 0.0814
20
40
60
80
100
120
Accuplacer Reading Comprehension Test Score
English 040
50
20
Rsq = 0.4111
40
60
80
100
120
Accuplacer Sentence Skills Test Score
5
Logistic Regression
Logistic Regression Analysis
Sample Classification Table - Math 050 cohort
Creates a model that predicts a dichotomous
outcome based on one or more predictor
variables.
Predictor – Elementary Algebra Test score
Predicted Outcome
Percentage
NonSuccess Success
Correct
2
2
50.0%
NonSuccess (4)
For this Study:
Outcome = Success or NonSuccess
Predictor variables = Accuplacer test scores
Success (11)
1
10
Overall
90.9%
80.0%
P-value
0.033
Cutoff = 0.5
Logistic Regression
Logistic Regression Models
Sample Classification Table - English 060 cohort
Math Cohort
Predictor – Sentence Skills Test score
Overall Percentage Correct
Predicted Outcome
Percentage
NonSuccess Success
Correct
Math 040
NonSuccess (1)
0
1
0%
Success (15)
0
15
100.0%
Math 050
Overall
93.8%
P-value
0.030
Math 060
Cutoff = 0.5
(n = 11)
(n = 15)
(n = 34)
Arithmetic
Elementary
Algebra
Both Tests
90.9%
90.9%
100 %*
73.3%
80.0%*
73.3%
88.2%*
88.2%*
94.1%*
*Level of significance (p-value) < 0.05
6
Logistic Regression Models
Answers to the Questions?
English Cohort
Overall Percentage Correct
English 040
(n = 10)
English 050
(n = 7)
English 060
(n = 16)
†
Reading
Comprehension
Sentence
Skills
Both
Tests
n/a
n/a
n/a
71.4%
100%*
100%
100%*
93.8%*
100%*
†
Can Accuplacer Tests Scores Predict Student
Success/Nonsuccess in College Preparation
Courses?
Yes!
Should entry Assessment Testing be
Mandatory for College Preparation
Applicants?
Yes, but used as an advising tool only!
*Level of significance (p-value) < 0.05
What Now?
Gather more data to support or refute the findings.
†
†
†
Test the prediction Models
Expand the study
Initiate new studies
Contact Information
Cindy James
UCC Assessment Centre Coordinator
900 McGill Road
Box 3010
Kamloops, BC Canada
Phone: 250 – 828 – 5471
Fax: 250 – 371 – 5690
Email: [email protected]
Website: www.cariboo.bc.ca/dsd/assmnt/
7