Delia North ([email protected])
Temesgen Zewotir ([email protected])
Michael Murray ([email protected])
SCHOOL OF STATISTICS AND ACTUARIAL SCIENCE
UKZN
UTLO Seminar, November 2011
Aim of the study
To investigate the effect of academic /demographic
factors on successful progression of students in
first year at UKZN between 2005 and 2009
Unpack performance of first year students
UTLO Seminar, November 2011
OUTLINE

Background – setting the scene

The Data

Descriptive Statistics

Multi -Level Logistic Regression

Results: Factors affecting failure rates
Factors affecting drop out rates

Conclusion
UTLO Seminar, November 2011
Background
HEI’s play a pivotal role in
 development of human capital
 educational, social, economic and cultural
development of society
so that Governments around the world invest in
Higher Education
UTLO Seminar, November 2011
Funding of universities in South African
Department of Education (DOE) funds HEI’s –
government funds HEI’s, focusing on two key outputs
– student throughput and research productivity.
Accordingly, each of the South African Universities
has got their own strategies to improve deliverance
on these outputs in order to secure the maximum
share of the South African Higher Education budget
UTLO Seminar, November 2011
Identifying factors that are a threat to successful
progression of students is vital for improving
throughput rates - deans and other decision
makers can put strategies in place for potential “at
risk students” so that effective intervention
strategies can be designed.
UTLO Seminar, November 2011
To inform strategies that improve throughput rates
we thus need to identify factors that increase the
risk of
* failure
* drop out
UKZN study: Entry level focus (North)
Modeling attrition – UG and PG (Zewotir)
Competing risks framework (Murray)
UTLO Seminar, November 2011
Relevance of this study
UKZN 7 years post merger and about to restructure.
How do school results in English and Mathematics
affect failure rates and drop out rates of first year
students….and what other factors affect these rates?
Valuable information for new Deans and Heads of
schools, Teaching and Learning officers, etc.
UTLO Seminar, November 2011
THE DATA
DMI - 71 954 records of first year students
Year - 2005 to 2009
faculty – 8 faculties
race – African, White, Other (Indian)
gender – male, female
alien – Yes, No
access – Yes, No
repeats – Yes, No
Maths –symbol and English symbol
residence – Yes, No
fin aid – Scholarship, Bursary, Loan, No financial aid
cleared – Yes, No, Out
UTLO Seminar, November 2011
Descriptive Stats
UTLO Seminar, November 2011
Change in racial profile of students in first year at UKZN
between 2005 and 2009
W
A
O
2005
14%
46%
40%
2009
5%
60%
35%
UTLO Seminar, November 2011
Fin aid profile change – more students
on bursaries, fewer on scholarship
UTLO Seminar, November 2011
Scholar
ships
Bursary
Loan
No Fin
Aid
2005
12%
14%
15%
58%
2009
4%
23%
16%
58%
Pattern of student success
UTLO Seminar, November 2011
%
NO
OUT
YES
2005
10.86
36.24
52.89
2006
10.48
37.32
52.2
2007
21.11
28.57
50.32
2008
17.91
30.9
51.19
2009
22.09
30.66
47.25
MULTI-LEVEL LOGISTIC REGRESSION
Outcome of year 1: Fail, Left, Pass
This model produces odds ratio based interpretation
Links odds of
* No (Fail) vs Yes (Pass)
* Out (Left) vs Yes (Pass)
(Pass = baseline)
to a set of exploratory variables
(academic/demographic factors)
UTLO Seminar, November 2011
MULTI-LEVEL LOGISTIC REGRESSION
Outcome of year 1: Fail, Left, Passed
ODDS RATIO BY FACULTY
ENG
EDU
H D Sc
H Sc
LAW
MED
MntSc
Sc A
0.65*
0.96
0.99
1.13*
1.21*
1.01
0.84*
0.98
1.03
1.06
0.44*
1.17
1.27*
1.03
0.84*
0.87*
YEAR EFFECT (2005 - 2009)
YEAR 2005 TO 2009
YEAR 2005 TO 2009
FAILED
LEFT
UTLO Seminar, November 2011
ODDS RAT IO BY FACULT Y
ENG
EDU
H D Sc
H Sc
LAW
MED
MntSc
Sc A
FAILED
LEFT
FAILED
LEFT
1.42
0.97
0.85
1.29*
1.66*
1.88*
0.63*
0.59*
1.51*
1.05
0.93
1.08*
1.99*
2.33*
0.91
0.77*
1.78
0.91
1.32
1.05
6.60
1.23
6.75
0.45*
1.46*
1.14*
0.89
0.85*
2.57*
1.43*
0.90
1.14*
FAILED
LEFT
0.91
0.98
0.84*
0.85*
1.06
0.97
1.31*
1.18*
0.87
0.81*
0.72
1.01
1.04
1.01
1.01
1.00
0.61*
0.63*
0.52*
1.09
0.44*
1.91*
38.02
0.67*
10.50
0.66*
66.88
2.80*
0.75
0.97
0.92
0.90
1.12
1.47*
0.82
0.68*
0.82
0.59*
1.12
1.81*
0.15
0.86
0.09
0.87
0.16
1.06
0.02
1.00
0.09
1.34
0.01
0.88
0.50*
0.84*
0.88
0.90
0.86
1.55*
0.59*
1.02
0.41*
0.72*
1.09
1.69*
RACE (REFERENCE : W HIT E)
AFRICAN
AFRICAN
OTHER
OTHER
GENDER (REFERENCE: MALE)
FEMALE
FEMALE
FINANCIAL AID (REFERENCE: SCHOLARSHIP)
BURSARY
BURSARY
LOAN
LOAN
NO FINANCIAL AID
NO FINANCIAL AID
FAILED
LEFT
FAILED
LEFT
FAILED
LEFT
UTLO Seminar, November 2011
ENG
EDU
H D Sc
H Sc
LAW
MED
MntSc
Sc A
RESIDENCE (REFERENCE: IN
UKZN RESIDENCE)
NOT IN
UKZN RES
FAILED
1.49*
1.10
1.23*
1.59*
1.24
1.51
1.40*
1.29*
NOT IN
UKZN RES
LEFT
1.01
1.47*
1.14*
1.90*
1.49*
1.44
1.23*
1.21*
0.02
0.01
0.02
0.01
0.02
0.01
0.01
0.24*
0.90*
0.72*
0.34*
0.41*
0.75*
1.30*
1.62
1.36
1.86
0.10
0.00
1.24
1.75
1.69
1.34*
3.38*
0.88
267.89
1.18
1.76*
REPEATS OF YEAR 1 (REFERENCE:
REPEATING YEAR 1)
FIRST
ATTEMPT
of YEAR 1 FAILED
0.01
FIRST
ATTEMPT
of YEAR 1 LEFT
0.58*
CITIZENSHIP (REFERENCE: NOT
SOUTH AFRICAN)
SOUTH
AFRICAN
FAILED
1.57
SOUTH
AFRICAN
LEFT
1.67*
UTLO Seminar, November 2011
ENG
EDU
H D Sc
H Sc
ACADEMIC SUPPORT PROGRAM (REFERENCE: IN ACADEMIC SUPPORT
PROGRAM)
NOT IN
AC
SUPPORT
PROGRM
FAILED
0.74
1.21
0.77*
1.42
NOT IN
AC
SUPPORT
PROGRM
LEFT
0.60*
1.45*
0.64*
1.62*
EFFECT
SCHOOL
MATHS
MARK
MATHS
AT
SCHOOL
MATHS
AT
SCHOOL
LAW
MED
MntSc
Sc A
0.10
7.40*
0.63*
0.90
2.17
0.60
0.79*
0.76*
FAILED
0.90
1.00
0.89*
1.08
0.91
1.45
0.99
0.97
LEFT
1.17*
1.04
1.01
1.27*
1.09*
1.19
1.11*
1.02
0.93
1.29*
1.06
1.65*
1.02
1.40
0.98
0.99
1.05 UTLO Seminar,
1.06
1.16* 2011 1.64*
November
1.26*
1.01
1.19*
1.14*
EFFECT OF SCHOOL
ENGLISH MARK
ENGLISH
AT
SCHOOL
FAILED
ENGLISH
AT
SCHOOL
LEFT
CONCLUSION
Faculties differ…..Heads of
Schools/Deans and Teaching and
Learning officers in colleges and
schools are tasked to put strategies and
processes in place to monitor and
prevent student drop-out and failure, so
this study should be of value to review
current selection and teaching
practices as the university heads to a
restructure in 2012.
UTLO Seminar, November 2011
Table 1: Odds ratio by faculty
ENG
EDU
H D Sc
H Sc
LAW
MED
MntSc
Sc A
RACE (REFERENCE : WHITE)
AFRICAN
FAILED
1.42
1.66*
1.51*
1.99*
1.78
6.60
1.46*
2.57*
AFRICAN
LEFT
0.97
1.88*
1.05
2.33*
0.91
1.23
1.14*
1.43*
OTHER
FAILED
0.85
0.63*
0.93
0.91
1.32
6.75
0.89
0.90
OTHER
LEFT
1.29*
0.59*
1.08*
0.77*
1.05
0.45*
0.85*
1.14*
FEMALE
FAILED
0.91
0.84*
1.06
1.31*
0.87
0.72
1.04
1.01
FEMALE
LEFT
0.98
0.85*
0.97
1.18*
0.81*
1.01
1.01
1.00
BURSARY
FAILED
0.61*
38.02
0.75
0.82
0.15
0.02
0.50*
0.59*
BURSARY
LEFT
0.63*
0.67*
0.97
0.68*
0.86
1.00
0.84*
1.02
LOAN
FAILED
0.52*
10.50
0.92
0.82
0.09
0.09
0.88
0.41*
LOAN
LEFT
1.09
0.66*
0.90
0.59*
0.87
1.34
0.90
0.72*
NO FINANCIAL AID
FAILED
0.44*
66.88
1.12
1.12
0.16
0.01
0.86
1.09
NO FINANCIAL AID
LEFT
1.91*
2.80*
1.47*
1.81*
1.06
0.88
1.55*
1.69*
NOT IN UKZN RESIDENCE
FAILED
1.49*
1.10
1.23*
1.59*
1.24
1.51
1.40*
1.29*
NOT IN UKZN RESIDENCE
LEFT
1.01
1.47*
1.14*
1.90*
1.49*
1.44
1.23*
1.21*
GENDER (REFERENCE: MALE)
FINANCIAL AID (REFERENCE: SCHOLARSHIP)
RESIDENCE (REFERENCE: IN UKZN RESIDENCE)
REPEATS OF YEAR 1 (REFERENCE: REPEATING YEAR 1)
FIRST ATTEMPT OF YEAR 1
FAILED
0.01
0.02
0.01
0.02
0.01
0.02
0.01
0.01
FIRST ATTEMPT OF YEAR 1
LEFT
0.58*
0.24*
0.90*
0.72*
0.34*
0.41*
0.75*
1.30*
UTLO Seminar, November 2011
ENG
EDU
H D Sc
H Sc
LAW
MED
MntSc
Sc A
CITIZENSHIP (REFERENCE: NOT SOUTH AFRICAN)
SOUTH AFRICAN
FAILED
1.57
1.62
1.36
1.86
0.10
0.00
1.24
1.75
SOUTH AFRICAN
LEFT
1.67*
1.69
1.34*
3.38*
0.88
267.89
1.18
1.76*
YEAR 2005 TO 2009
FAILED
0.65*
0.99
1.21*
0.84*
1.03
0.44*
1.27*
0.84*
YEAR 2005 TO 2009
LEFT
0.96
1.13*
1.01
0.98
1.06
1.17
1.03
0.87*
YEAR EFFECT (2005 - 2009)
ACADEMIC SUPPORT PROGRAM (REFERENCE: IN ACADEMIC SUPPORT PROGRAM)
NOT IN ACADEMIC SUPPORT PROGRAM
FAILED
0.74
1.21
0.77*
1.42
0.10
7.40*
0.63*
0.90
NOT IN ACADEMIC SUPPORT PROGRAM
LEFT
0.60*
1.45*
0.64*
1.62*
2.17
0.60
0.79*
0.76*
MATHEMATICS AT SCHOOL
FAILED
0.90
1.00
0.89*
1.08
0.91
1.45
0.99
0.97
MATHEMATICS AT SCHOOL
LEFT
1.17*
1.04
1.01
1.27*
1.09*
1.19
1.11*
1.02
ENGLISH AT SCHOOL
FAILED
0.93
1.29*
1.06
1.65*
1.02
1.40
0.98
0.99
ENGLISH AT SCHOOL
LEFT
1.05
1.06
1.16*
1.64*
1.26*
1.01
1.19*
1.14*
EFFECT OF SCHOOL MATHS MARK
EFFECT OF SCHOOL ENGLISH MARK
UTLO Seminar, November 2011
Changes in first year throughput
over the period 2005 to 2009
Failure risk
Increased over 2005 to 2009: HDSSc, MntS
Decreasing over 2005 to 2009: ENG, HSc, MED, ScA
Drop-out risk
Increasing over 2005 to 2009: EDU
Decreasing over 2005 to 2009: ScA
UTLO Seminar, November 2011
Effect of Race
Failure risk
African > White: EDU, HDSSc, HSc, MntS, ScA
White students > Indian: EDU
Drop-out risk
African > White : EDU, HSc, MntS, ScA
Indian > White: ENG, HDSSc, ScA
White > Indian: EDU, HSc, MED, MntS
UTLO Seminar, November 2011
Effect of Gender
Failure risk
Male > Female : EDU
Female > Male : HSc
Drop-out risk
Male > Female : EDU, LAW
Female > Male: HSc
UTLO Seminar, November 2011
Effect of Financial Aid
Failure risk
Scholarship > Bursary : ENG, MntS, ScA
Scholarship > Loan : ENG, ScA
Scholarship > No Fin Aid: ENG
Drop-out risk
Scholarship > Bursary: ENG, EDU, HSc, MntS,
Scholarship > Loan: EDU, HSc, ScA
No Financial Aid > Scholarship:
ENG, EDU, HDSSc,HSc, MntS, ScA
UTLO Seminar, November 2011
Effect of Residence (On campus or not)
Failure risk
Not UKZN campus > UKZN campus :
ENG, HDSSc, HSc, MntS, ScA
Drop-out risk
Not UKZN campus > UKZN campus:
EDU, HDSSc, HSc, LAW, MntS, ScA
UTLO Seminar, November 2011
Effect of repeating year one
Drop-out risk
Repeating Yr 1 > First attempt at Yr 1:
ENG, EDU, HDSSc, HSc, LAW, MED, MntS
First attempt at Yr 1 > Repeating Yr 1: Sc A
UTLO Seminar, November 2011
Effect of citizenship
Drop-out risk
South African citizen > non- South African citizen:
ENG, HDSSc, HSc, ScA
UTLO Seminar, November 2011
Effect of academic support
programs
Failure risk
academic support > not academic support:
HDSSc, MntS
academic support > not academic support: MED
Drop-out risk
academic support > not academic support:
ENG, HDSSc, MntS, ScA
not academic support > academic support: EDU, HSc
UTLO Seminar, November 2011
Effect of school Mathematics and
English results
Failure risk
Higher for better school Math mark: HDSSc
Lower for better school English mark: EDU, HSc
Drop-out risk
Lower for better school Math mark:
ENG, HSc, LAW, MntS
Lower for better school English mark:
HDSSc, HSc, Law, MntS, ScA
UTLO Seminar, November 2011
Humanities, Development and
Social Sciences: Odds Fail vs Pass
 African students are 50% more at risk of failing than
White students
 Students not in on campus res have increased odds
(23%) to fail
 Further contributing factors to failing in this faculty
are those on academic support programs…..and those
with better marks for Mathematics at school!
UTLO Seminar, November 2011
Engineering : Odds Out vs Pass
 Indian students are the biggest racial group dropping




out of this faculty in first year (29% more of a risk than
White students)
Students with no financial aid are almost twice more
likely to drop out than those with scholarships
Students repeating year 1 are almost twice as likely to
drop out than first time registrations in year 1
South African students have 1.67 times the odds of
dropping out than students from other countries
Better marks for school maths results in lower risk of
dropping out
UTLO Seminar, November 2011
Threat: throughput rate
Evidence that a smaller proportion of students are
passing level 1 (53% ->47%), more are failing and staying
in the system (11% - 22%).
Profile Change -> student race, fin aid, gender, etc
-> students success profile
???? Which factors contributed to the risk of first year
students failing or dropping out as opposed to
successfully passing to level two modules.
UTLO Seminar, November 2011