Ready, Set, FLY [First Learning Year]:
A CAUSAL MODEL
FOR RISK AT THE
UNIVERSITY OF PRETORIA
Juan-Claude Lemmens
[email protected]
Department for Education Innovation:
Unit for Higher Education Research & Innovation (HERI)
Outline
•
•
•
•
•
•
•
Aim of the research
Motivation for the research
Background
Methodology
Questionnaire design
Results (Quantitative & Qualitative)
Conclusion
Aim of the research
AIM OF STUDY
PROFILING RISK OF FIRST
YEAR STUDENTS
QUALITATIVE STUDY
EXIT INTERVIEWS WITH
STUDENTS
QUANTITATIVE STUDY
ACADEMIC READINESS
SURVEY
SYNTHESIS
RISK MODEL AND PROFILE
FOR EARLY WARNING
Motivation for the research
• Improve retention
• Lower drop-out
• Non-cognitive entry
characteristics
The Dean of Economic and Management Sciences
commissioned and investigation
Background
South African Higher Education Landscape
(1994 – 2005)
• Equitable system with access to all the racial
groups
• Increase and broaden participation
African Coloured
Indian
White
Overall
1993 9%
13%
40%
70%
17%
2000 13%
9%
39%
47%
16%
2005 12%
12%
51%
60%
16%
Gross participation rates for South Africa (Scott et al., 2007, p. 10; Bunting in Cloete, et al., (eds.), 2006b, p. 106)
Background
South African Higher Education Landscape
(2000 cohort)
• High withdrawal and low graduation rates
of student who are in the system
Grad within
5 years
Still registered
after 5 years
Left without
graduating
SA Universities
All degrees
50%
12%
38%
Academic first Bdegrees
Business/Management
50%
7%
43%
National graduation rates (Scott, Yeld & Hendry, 2007, p. 12)
Background
University of Pretoria Context (2001 cohort)
• Contact institution
• Tuition in both English and Afrikaans
Grad within 5
years
Still registered Left without
after 5 years
graduating
Total UP
54.8%
18.4%
26.8%
White
59.4%
17.3%
23.3%
Coloured
50%
19.6%
30.4%
Indian
31.5%
16.1%
22.4%
African
36.8%
23%
40.2%
Graduation rates of all Academic first B-degrees (BIRAP, 2009)
Background
Faculty of Economic and Management Sciences Context
•
•
Contributed 24.6% of all undergraduate enrolments for the 2008
cohort
Academic and professional first B-degrees over a three year
period
Grad within 5
Still registered
Left without
years
after 5 years
graduating
Total EMS
58.2%
17.1%
24.7%
White
63.5%
14.5%
22%
Coloured
56%
20%
24%
Indian
59.3%
22.1%
18.6%
African
40.7%
24%
35.3%
Graduation rates of Academic first B-degrees, 2001 cohort (BIRAP, 2009)
Data collection
First phase (2007)
Literature study: entry characteristics and biographical variables
that correlate with withdrawal and academic performance
QUANTITATIVE
Second phase (2008)
• Academic Readiness
Questionnaire administered
to students from the Faculty
of Economic and
Management Sciences in
the beginning of February
2008 during the orientation
week
• Biographical data (BIRAP)
QUALITATIVE
Third phase (2008)
• Exit interviews at the end of
the academic 2008 year
• Discovering the main reason
for withdrawal from studies,
as well as sub-reasons that
may have contributed to the
decision to discontinue
studies.
Questionnaire development
• Theoretical foundation
– Social and academic integration (Tinto, 1993)
– Psychological model of College Student
Retention (Bean & Eaton, 2005)
– Psychological perspectives: Constructs that
have been related to student success:
Internal locus of control, expectancy theory,
self-efficacy theory, and motivational theory
– Structured questionnaires
• Non-cognitive Questionnaire (Tracey and
Sedlacek, 2004)
• Survey of Academic Orientations
(Davidson, Beck & Silver, 2001)
Questionnaire development
• Academic
Readiness
Questionnaire
– 70 items and
is answered
on a five point
Likert-type scale
• Factor Analysis
– N=829
– 11 Items
discarded
Factors
Alpha
1. Achievement Motivation
0.76
2. Learning efficacy
0.75
3. Planning
0.74
4. Integration and support
0.63
5. Reading behaviour
0.74
Total variance explained = 57.9%
Entry characteristics
Definition
Achievement Motivation
The degree to which one values higher education and showing an
interest in academic work.
Learning efficacy
The degree of confidence in one’s own ability to achieve one’s
academic goals.
Planning
The degree to which one is able to plan your studies.
Integration/support
The degree to which the student has institutional, social and
financial support.
Reading behaviour
The degree to which one finds pleasure in reading.
M-score
A metric score based on the six best senior certificate subjects.
(Range between 0 – 30)
Parental education
One or both parents completed a degree.
Housing
Where a student is living while attending university.
School location
Distance of school from the university.
Risk for withdrawal
Students, who were discontinued, are on probation or have
withdrawn from their studies.
Risk for failure
Students who passed less that 100% of the credits registered for
and who are at risk for withdrawal.
Academic success
The degree of academic achievement at university.
Credits registered
Number of credits registered divided by the number of credits
prescribed.
Description of the sample
2008 Intake African
Coloured
Indian
White
University of 30.2%
Pretoria
2.2%
4.0%
63.5%
Faculty of
EMS
37.4%
2.2%
5.7%
54.7%
Sample
24.4%
2.5%
2.7%
70.4%
Description of the sample
Enrolment status
Frequency
Percent
Discontinuation
25
3.0
Withdrawal
53
6.4
Probation
18
2.2
Promotion 2nd
733
88.4
Total
829
100.0
Cross-tabulations
• Explore the relationship between the dependent
variables, Risk for withdrawal and Risk for failure and
each of the independent variables:
– Race, M-score, Parental education, Gender, Age,
Housing, School location, Home language,
Language of instruction, Achievement motivation,
Learning efficacy, Planning, Integration/support
and Reading behaviour
Results – Risk for withdrawal
Logistic Regression Analysis
• Full model (16, N=829) = 76.64, p<.000
• The model explained 11% - 21.3% of variance
• Race, M-score and number of credits registered
made a unique statistical significant contribution to
the model
• Partial effects causal model based on the maximum
likelihood analysis of variance (CATMOD)
– N=601 missing values for any variable are
omitted from the analysis
– Race language
Results - Risk for Withdrawal
Category
n
Odds Index
Mean
601
15.07
Race language*
 African
 Afrikaans
 English
134
385
82
3.844
0.491
0.529
M-score*
 Low
 Medium
 High
133
285
183
0.423
0.967
2.447
Credits registered*
 <1
 =1
 >1
193
217
191
0.436
3.145
0.729
Partial effects causal model for Risk for withdrawal
Results - Risk for Withdrawal
• Gender – Male students are at risk
• Parental education – Students whose parents have a
tertiary education are at risk
• Distance from school – The farther away a student
attended school, the more a student is at risk for
withdrawal
• Housing – Students who live in university residence are
at risk for withdrawal
Trends based on Partial effects causal model for Risk for withdrawal
Results - Risk for Withdrawal
• Achievement motivation – Student with medium
or high achievement motivation scores are at
risk
• Learning efficacy – Students with medium
learning efficacy scores are at risk
• Planning – Student who are less able to plan
their study time are at risk
• Integration and support – Students from all three
categories are virtually at baseline (1)
• Reading behaviour – Students who are average
readers are most at risk for withdrawal
Trends based on Partial effects causal model for Risk for withdrawal
Results – Risk for failure
Category
n
Odds Index
Mean
601
0.294
Race language*
 African
 Afrikaans
 English
134
385
82
2.245
0.639
0.697
M-score*
 Low
 Medium
 High
133
285
183
0.089
1.011
11.14
Reading behaviour f5
 Low*
 Medium
 High
190
184
227
1.433
0.944
0.739
Partial effects causal model for Risk for failure
Results - Risk for Failure
• Gender – Female students are at risk
• Parental education – Students whose parents have a
tertiary education are at risk for failure
• Distance from school – Student attending schools in
other provinces are at risk for failure.
• Housing – Students from all three categories are
virtually at baseline (1)
Trends based on Partial effects causal model for Risk for failure
Results – Risk for failure
• Achievement motivation – Student with low or high
achievement motivation scores are at risk for failure
• Learning efficacy – Students with medium or high
learning efficacy scores are at risk
• Planning – Students less able to plan their study time
are at risk for failure
• Integration and support – Students with low or high
integration and support scores are at risk for failure
• Reading behaviour – Students who enjoy reading
are at risk for failure
Trends based on Partial effects causal model for Risk for failure
Results – Academic success
Standardised
Coefficients
t
Sig.
Beta
B
p
Zero order r
Achievement motivation
-.013
-.312
.755
.054
Learning efficacy
-.048
-1.238
.216
.074
Planning
.122
3.396
.001
.158
Integration and support
-.054
-1.495
.135
-.050
Reading behaviour
-.091
-2.556
.011
.022
Credits registered
.203
6.520
.000
.210
Language of tuition
.030
.737
.461
.053
Housing
-.012
-.394
.694
.067
Age
-.016
-.507
.612
-.059
M-score
.571
17.563
.000
.529
Gender
.077
2.419
.016
-.018
Race
.223
4.966
.000
.072
Parental education
-.048
-1.510
.131
-.025
•
(Constant)
R squared = .38
Results – Causal model
R squared = .38
Qualitative Results (main reasons)
Reason from students
Academic
Frequency
Percent
3
7.1
Study choice
26
61.9
Family responsibilities
3
7.1
Work responsibilities
1
2.4
Health
3
7.1
Financial
2
4.8
Personal
1
2.4
Institutional
2
4.8
Faculty discontinuation
1
2.4
Total
42
100.0
Qualitative Results (Sub reasons)
Sub-reasons from students
Not performing as expected
Workload of programme
Wrong career choice
Uncertain career goals
Did not enjoy the programme
N
Percent
5
5.6%
7
7.8%
15
16.7%
7
7.8%
8
8.9%
Qualitative Results
Influence on studies
Problem influence
N
Percent
Caused stress/pressure
7
14.0%
Wanted to give up
2
4.0%
Disrupted studying
2
4.0%
Not motivated
10
20.0%
Not go to class
5
10.0%
Not enough time to study
6
12.0%
10
20.0%
Difficulty concentrating
1
2.0%
Did not study
2
4.0%
Lack of engagement
2
4.0%
No influence on studies
2
4.0%
Positive influence
1
2.0%
50
100.0%
Perform poor academically
Total
Qualitative Results
Start of problem and stop date
Start experiencing
problems
Orientation week
Stop date
Frequency
Percent
Frequency
Percent
January
1
2.4
February
5
11.9
5
11.9
17
40.5
Second semester
5
11.9
March
1
2.4
May/June
5
11.9
April
2
4.8
Third semester
4
9.5
May
36
85.7
3
7.1
6
14.3
June
1
2.4
42
100.0
July
25
59.5
August
2
4.8
September
1
2.4
October
1
2.4
42
100.0
First semester
Total
Missing
Total
Total
Qualitative Results
Use of student services
Make use of
SSD
Frequency
Percent
Yes
11
26.2
No
31
73.8
Total
42
100.0
Help in UP
None
N
Percent
15
30.0%
7
14.0%
10
20.0%
Peer
7
14.0%
Student support
division
7
14.0%
Faculty administration
3
6.0%
UP mentor
1
2.0%
50
100.0%
Department/faculty
head
Lecturer/tutor
Total
Conclusion
• Retaining students is of critical importance in
South Africa
• Skewed result from Logistic Regression model
necessitate a broader definition
• Exit interviews - study choice has an adverse
effect on academic achievement
• Students do not seek help proactively
• Window of opportunity
• Decentralise support to the faculty
Conclusion
• Early Warning and Referral System
– Student life cycle
• Predictive power of the Academic Readiness
Questionnaire
• Using the model for support
• Profiling for risk is contentious
• Recommendations
– Determine the interaction effect
– Profile students according to race
I THANK YOU FOR YOUR ATTENTION!
Questions? Questions? Questions?