ADecisionTreeAnalysisoftheTransferStudent
EmmaGunu,MS
ResearchAnalyst
RobertMRoe,PhD
ExecutiveDirectorofInstitutionalResearchandPlanning
Overview
• MotivationforAnalyses
• AnalysesandResults
• Descriptive
• LogisticRegression
• DecisionTreeAnalysis
• 1st YearRetention
• 6yearGraduationRate
• Impactof2yearDegreeonPerformanceat
CMU
• Conclusion
2
OfficeofInstitutionalResearch
Motivation
• CMUtypicallyenrollsbetween1400and1500
transferstudentseachyear
• Constitutesnearly25%ofallnewstudents
• Inthepastusedallnewtransfersascohort–
ignoringnumberoftransfercredits
• GiventhelimitedresourcesoftheOfficeof
StudentSuccess,canweidentifyspecificgroups
foroutreach(intervention)?
3
OfficeofInstitutionalResearch
EntryCredentialsbyClass
3-year(2011-2013)mean%countbyClassof
Entry
60%
Mean Transferhours byClassofEntry
(2006-2014)
50%
80
40%
60
30%
40
50.3%
20%
25.9%
21.2%
10%
0%
Freshman(<26hrs) Sophomore(2756hrs)
Junior(57-86hrs)
20
0
66.0
17.1
Freshman
Mean TransferGPA
4.00
3.50
3.00
2.50
2.00
2.90
3.04
Fresh
Soph
3.24
Junior
4
OfficeofInstitutionalResearch
41.1
Sophomore
Junior
FirstYearPerformance
%Persistinginto 2ndYear byClassofEntry
FirstTermGPAbyEntryLevel
4.00
3.50
3.00
2.50
2.00
2.46
Fresh
5
2.76
Soph
3.02
Junior
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
70.5%
77.5%
84.3%
Fresh
Soph
Junior
OfficeofInstitutionalResearch
LOESSofPersistencebyTransferHours
0255075100 125
6
OfficeofInstitutionalResearch
GraduationbyLevelofEntry
Last3-yearmean%GraduatinginYearbyClassofEntry
90%
80%
Freshman
70%
60%
Sophomore
50%
Junior
40%
30%
20%
10%
0%
1stYear
7
2nd year
3rdYear
4thYear
5thYear
OfficeofInstitutionalResearch
6thYear
Impactof2YearDegreeAtCC
• Nextwelookedattheimpactofobtaininga2year
degreeonpersistence,graduation,andGPA
8
OfficeofInstitutionalResearch
Impactof2yearDegree
90%
80%
70%
10%
75.8%
58.8%
20%
43.8%
30%
67.2%
40%
74.6%
50%
85.7%
60%
0%
Pers2
Grad4
Degree
9
NoDegree
OfficeofInstitutionalResearch
Grad6
Graduationin4yrs
Graduated
within4Years
NoAssociate
Degree
No
Yes
1006
(56.2%)
784
(43.8%)
190
389
(67.2%)
AssociateDegree (32.8%)
χ2 =95.714,p<.001
However, persistenceandgraduation areconfounded with
number oftransferhours.
10
OfficeofInstitutionalResearch
Logistic Regression- Graduation
in 4yrs
TRANHRS
Prior_2yr_Degree
Constant
B
S.E.
df
Sig.
Exp(B)
0.028
0.003
1
0.000
1.029
0.28
0.122
1
0.021
1.323
-1.197
0.106
1
0.000
0.302
a. Variable(s) entered on step 1: TRANHRS, Prior_2yr_Degree.
OfficeofInstitutionalResearch
11
Results
• Theimpactofa2yeardegreeonpersistence
tosecondyearandgraduationin4or6years
wassignificantbeyondtheimpactofnumber
oftransferhours
12
OfficeofInstitutionalResearch
BestWaytoClassifyStudentsWhoare
atRisk?
• Performancevariesbytransferhoursbutwhat
aboutotherfactors?
• Coulduselogisticregressiontodeterminewhich
factorsarepredictive
• However,thisisnotusefulindeterminingwhich
groupsofstudentsareatrisk
13
OfficeofInstitutionalResearch
FactorsThatLikelyImpactPersistence
• Variables
– Fullorparttimestudent
– TransferGPA
– Totaltransferhours
– TransferCollegeYear
– LowIncomeStatus
– FirstGenerationStatus
14
OfficeofInstitutionalResearch
Logistic- Stepwise
Variables in the Equation
Step 1
a
Step 2
b
TRANGPA
B
.796
Constant
-1.123
Step 3
Step 4
d
.161
df
Sig.
Exp(B)
.000
2.216
48.490
1
.000
.325
.710
.055 166.387
1
.000
2.034
TRANHRS
.011
.001
57.435
1
.000
1.011
-1.294
.163
62.740
1
.000
.274
FULLPART
.739
.128
33.566
1
.000
2.095
TRANGPA
.702
.055 161.845
1
.000
2.017
TRANHRS
.012
.001
68.157
1
.000
1.012
Constant
-2.020
.207
94.866
1
.000
.133
LOWINC
-.324
.067
23.659
1
.000
.723
FULLPART
.733
.128
32.890
1
.000
2.081
TRANGPA
.700
.055 160.898
1
.000
2.014
TRANHRS
.013
.001
75.412
1
.000
1.013
-1.968
.208
89.593
1
.000
.140
Constant
a. Variable(s) entered on step 1: TRANGPA.
b. Variable(s) entered on step 2: TRANHRS.
c. Variable(s) entered on step 3: FULLPART.
d. Variable(s) entered on step 4: LOWINC.
15
1
TRANGPA
Constant
c
S.E.
Wald
.054 217.894
OfficeofInstitutionalResearch
DecisionTreesforOutreach
16
OfficeofInstitutionalResearch
DecisionTreeModels
• Severaltypesofdecisiontreemodels
• HerewechosetheChi-squareAutomaticInteraction
Detection(CHAID)ModeloverClassificationand
RegressionTrees(CRT)
• WithCRT,GPAmightsplitseveraltimestoberefined
enoughtobepredictive
17
OfficeofInstitutionalResearch
DecisionTreeModels
• Thesemodelscanbeusedsimplytoclassifyaset
ofdata(e.g.whatisthebestwaytoclassifyour
transferstudentsintermsofretentionfactors)
• Orcanbeusedforprediction(e.g.canweflag
newtransferstudentswhoareatrisk(ornotat
risk)forpersistence?)
18
OfficeofInstitutionalResearch
CHAID
• Theprocedurecreatestree-basedmodelsthat
determinehowvariablesbestcombinetoexplain
theoutcomeinagivendependentvariable
• Dependentvariableisbinaryresponse–Retained
vsNot
• Predictorvariablesareanycombinationof
variabletypes(continuousorcategorical)
19
OfficeofInstitutionalResearch
Method
• Startbyselectingasubsetofdatafortraining
• Usemodeltopredictanewsetofdata
• Herewechoseasubsetof70%ofthedataandfit
totheremaining30%
• Checkmisclassificationrateandstandarderrorfor
predictability
20
OfficeofInstitutionalResearch
CHAIDAnalysisForFirstYearRetention
• Startbyclassifyingtodetermineifevenpossible
• Ifyes,buildpredictionmodel
• Inputvariables
–
–
–
–
–
–
Fullorparttimestudent
TransferGPA
Totaltransferhours
TransferCollegeYear
LowIncomeStatus
FirstGenerationStatus
• AllwereSelectedbyModel
21
OfficeofInstitutionalResearch
Persistto2 nd Year
<=2.410
22
(2.41-3.06]
(3.06-3.35]
OfficeofInstitutionalResearch
(3.35-3.73]
>3.73
23
OfficeofInstitutionalResearch
24
OfficeofInstitutionalResearch
WhichNodesareImportant?
Gains for
Nodes
Node
5
25
30
28
31
23
26
22
16
27
8
29
32
17
12
24
20
21
9
18
6
25
Training
Index
115.1%
114.5%
114.2%
110.8%
104.6%
103.9%
103.2%
102.5%
101.0%
98.6%
98.0%
95.5%
93.3%
89.1%
88.4%
86.7%
85.2%
85.2%
80.4%
72.5%
70.0%
Node5:TransferGPA>3.73
90.1%Persistto2nd year
Node6:TransferGPA<=2.41
TransferHrs <=23,
54.8%Persistto2nd year
OfficeofInstitutionalResearch
ModelEvaluation
Classification
Predicted
Sample
Training
Test
Not Persist2
Persist2
Overall Percentage
Not Persist2
Persist2
Overall Percentage
Percent
Not Persist2 Persist2 Correct
0
1242
0.0%
0
4469
100.0%
0.0% 100.0%
78.3%
0
563
0.0%
0
1917
100.0%
0.0% 100.0%
77.3%
Risk
Sample Estimate Std. Error
Training
.217
.005
Test
.227
.008
Growing Method: CHAID
Dependent Variable: Persisted to
2nd Year
26
OfficeofInstitutionalResearch
SummaryofPersistenceDecisionTree
• TransferGPAismostpredictive
• OtherpredictivefactorsvarybytransferGPA
• Forclarityandeaseofunderstandingforthose
whousetheinformation,wecreatedthe
followingtables:
27
OfficeofInstitutionalResearch
RetentionatRiskTable
TransferCredits
TransferGPA
4.0 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0
28
>60
58
56
54
52
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
6.2%
17.4%
1.5%
10.5%
5.0%
Lowincomein
this rangeat
higherrisk.
18.3%
Lowincomeinthis
rangeathigherrisk.
FirstGenerationin
this rangeathigher
risk.
3.5%
3.3%
4.5%
DecisionTreeAnalysisFor6Year
GraduationRate
• Inputvariables
– Fullorparttimestudent
– TransferGPA
– Totaltransferhours
– TransferCollegeYear
– LowIncomeStatus,FirstGenerationStatus
• Allvariablesexceptfullvsparttimewere
selectedbymodel
29
OfficeofInstitutionalResearch
6YearGraduation
<=2.31
2.30-2.69
30
2.69-2.99
2.99-3.29
OfficeofInstitutionalResearch
<=3.29
Summaryof6YearGraduation
DecisionTree
• ForlowtransferGPA(<=2.3),1st generationstatus
atgreaterrisk
• ForsecondlowestGPA(2.3-2.69],transfercredit
hoursand2vs4yearimportant
• FormiddleGPA(2.69-3.29],2vs4yearand
transfercredithoursimportant
• ForhighestGPA(>3.29)transferhours,2vs4year,
andlowincomeimportant
31
OfficeofInstitutionalResearch
GradatRiskTable
TransferCredits
TransferGPA
4.0 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0
>60
58
56
54
52
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
10.0%
7.9%
15.0%
First
Generationin
this rangeat
higherrisk.
CCtransferin
this rangeat
higherrisk.
CCinthis
rangeat
higherrisk.
OfficeofInstitutionalResearch
32
Summary
• Themorerefinedassessmentoftransferstudents
revealedsomeinterestingfindings
• Usingdecisiontreeanalysesontransferdata,we
areabletoidentifyveryspecificgroupsatrisk
(andgroupslikelytosucceed)
• Wehavepresentedtheseresultstotheretention
subcommittee,strategicenrollmentgroup,andto
thevicepresidentofenrollmentservices
• Questions?
33
OfficeofInstitutionalResearch