ExternalEvaluationReportYear4
“FastForward”Grant
SanJuanCollege2014/15
LauraGodfrey,ExternalEvaluator
TableofContents
Introduction........................................................................................................................................................3
OverviewofExternalEvaluationReport....................................................................................................................3
AdequacyofData.............................................................................................................................................3
AuditReadiness.................................................................................................................................................3
TimelyReporting..............................................................................................................................................4
RelationshipwithDepartmentofEducationProgramOfficer.......................................................4
CriticalQuestion1:TowhatextenthastheCollegeachievedtheobjectivessetforth
intheproposalwithinthetimelinesspecified?
Goal1:ImproveDegreeCompletionforNativeAmericanAA/ASStudent..................................................5
Goal2:ImproveTransferRatesforNativeAmericanStudents......................................................................8
Goal3:ImproveCourseSuccessRatesinkeyAA/ASdisciplines....................................................................10
Goal4:ExpandcapacitytobetterserveNativeAmericanAA/ASStudents.............................................11
Goal5:ImproveabilityofSJCpersonneltocollect,analyzeandusedatatobetterserveNative
Americanstudents..............................................................................................................................................................14
CriticalQuestion2:TowhatextenthastheCollegedeliveredhighquality
programming?.......................................................................................................................................................16
CriticalQuestion3:TowhatextenthastheCollegeuseddata
toimproveprogram?........................................................................................................................................17
CriticalQuestion4:TowhatextenthastheCollegefulfilledthelegislativeintentof
theTitleIIIAct?....................................................................................................................................................18
CriticalQuestion5:TowhatextenthastheCollegeusedgrant-fundedinitiativesto
fulfillitsmission?................................................................................................................................................18
ExternalEvaluator’sConclusions..............................................................................................................19
2
ExternalEvaluatorReport,2015
FastForwardGrant,Year4
SanJuanCollege
Overview
DuringthefourthyearofitsTitleIIINANTSIgrant(AcceleratingAA/ASDegreeand
CertificateCompletionforNativeAmericanStudents-locallynamed“FastForward”),San
JuanCollegehascontinuedtoimplementedtheprogramasoutlinedinthegrantandhas
continuedtomakeprogresstowardsmeetingtheobjectivesstatedinthegrant.Therehas
beenhigh-leveladministrativesupportforcollaborationbetweentheCollege’stwoTitleIII
programsaswellaswithotherrelevantprogramsacrosscampus.Thegrantisin
compliancewiththetimelinesestablishedintheproposalandwhereindicatedthecollege
hasworkedwiththeprogramofficertomakechangesinthewordingofobjectives.The
grantis“audit-ready”withfulldocumentationavailableandeasilyaccessible.Theteam
hasestablishedcomprehensiveevaluationandtrackingsystemsforallaspectsofthe
project.Responsibilitiesareclearlydelineatedforcollectingandreportingboth
quantitativeandqualitativedata.
AdequacyofData
TheExternalEvaluatorhasreviewedalldatasourcesanddatacollected.Dataforthis
reporthasbeenprovidedby:theInstitutionalResearchDepartment,theGrants
ComplianceOffice,theProjectDirector/Staff,andtheExternalEvaluator.Thedata
reportedisreliableandaccurate.DatasourcesfortheAnnualPerformanceReportinclude:
• Attendance/participationinformationcollectedbyprojectstaff
• AuditReviewRecordscollectedbyprojectdirectorandreviewedbyGrants
ComplianceOffice
• PresentationsatOversightandAdvisoryCouncilmeetings
AuditReadiness
Inadditiontocollectingdatathatdocumentsprogresstowardsachievingprogram
objectives,theCollegefundsafulltimeGrantsComplianceOfficerwhoisresponsiblefor
“regulatorycomplianceofallcollegegrants”.Eachyearthereisanannualreviewofaudit
documentsbytheExternalEvaluatorinconjunctionwiththeGrantsComplianceOffice.
TheCollegerequireseachgrantprogramtocompilearecordkeepingsystemfor16items
onthe“AuditReviewChecklist”.Itemsinclude:approvaldocuments,budgetfiles,
proceduresmanuals,inventoriesofequipment,travelreports,budgetledgers,project
participantsandallfinancial,performanceandannualreports.TheGrantsCompliance
OfficerhasreviewedtheauditreadinesschecklistfortheFastForwardprogram.The
3
EvaluatorhasconfirmedthattheFastForwardgrantisincompliancewithSJCpolicies,
proceduresandfinancialdocumentationrequirementsandis“audit-ready”.
TimelyReporting
TheCollegehassubmittedthorough,accuratereportstothegrantingagencyTheAnnual
PerformanceReport(APR),wassubmittedbythegranteebytheFebruary1,2016
deadline.Itcontainscompleteresultsforthegoalsandobjectivesdelineatedinthegrant.
TheExternalEvaluatorhasreviewedtheAPRandissatisfiedthatthegranteehas
systematicallycollectedquantitativeandqualitativedatademonstratingprogresstowards
meetingalloftheperformancemeasures.Intheinstanceswherebenchmarkshavenot
beenmetananalysishasbeencompletedandadjustmentstotargetsorprogramsmade
accordingly.
OneconcerntheEvaluatorhasisthelatereportingdeadline.ThegrantyearendsOctober
1yettheAPRisnotdueuntilFebruary.TheEvaluatorbelievesthathavingadraftAPR
beforeChristmasbreakwouldallowformuchbetteranalysisandfeedback.Anychangesor
improvementscouldbeimplementedforthespringsemester.Onthecurrentschedulewe
arelookingbacknearlyayear.Whilenotrequired,earlierreportingwouldallowthe
Director,theEvaluatorandtheAdministrationtoseeresultsinamoretimelymannerthat
betterreflectstheacademicyear.
RelationshipwithEDProgramGrantsOfficer
TheprojectstaffhasmaintainedaverypositiveworkingrelationshipwiththeDepartment
ofEducationProgramOfficer.SJCsoughtandreceivedapprovalfromtheProgramOfficer
toreplacetheFastForwardResearchSpecialistwithaProgramAdvisor.Thisisashift
fromgatheringinformationtoactingoninformation.Thispositionhasbeenfilledandthe
ProgramAdvisorwillfurtherimplementthenewCohortConnectionsProgram.Thetime
involvedinfillingthisandtheWebDesignervacancyresultedinasignificantcarry-forward
inthebudgetwhichtheDirectorhasdiscussedwiththeProgramOfficerandreceived
approvalforchangesinexpendituresforYear5.
CriticalQuestionsforEvaluation
TheExternalEvaluatorhaspreparedthisreportusingthefollowingcriticalquestionsas
theframeworkforreviewingallofthedata.
4
1. TowhatextenthastheCollegeachievedtheobjectivessetforthinthe
proposalwithinthetimelinesspecified?
2. TowhatextenthastheCollegedeliveredhighqualityprogramming?
3. TowhatextenthastheCollegeuseddatatoimproveprograms?
4. TowhatextenthastheCollegefulfilledthelegislativeintentoftheTitleIII
Act?
5. TowhatextenthastheCollegeusedgrant-fundedinitiativestofulfillits
mission?
CRITICALQUESTION1:TowhatextenthastheCollegeachievedthe
objectivessetforthintheproposalwithinthetimelinesspecified?
Goal1:ImprovedegreecompletionforNativeAmericanAA/AsStudents
ActivityObjective(s)
1a.BySeptember2015,increasethenumber
ofNativeAmericanstudentswhograduate
withAA/ASdegreeswithinthreeyearsto28
ResultsforYear4
Actual=15
Target=28
1b.BySeptember2015,increasethe
percentageoffirst-time,full-timeAA/AS
degree-seekingNativeAmericanstudentswho
completeanAAorASwithinthreeyearsto8%
Actual=4.2%
Target=8%
1c.BySeptember2015,increasethe
percentageoffirst-time,full-timeAA/AS
degree-seekingNativeAmericanstudentswho
completeanySJCdegreeorcertificatewithin
threeyearsto10%
Actual=11.6%
Target=10%
1d.BySeptember2015,improvetheratioof
first-time,full-timeNativeAmericanAA/AS
enrolledstudentspereachfirst-time,full-time
NativeAmericanAA/ASgraduateto22
Actual=23.5
Target=22
ExternalEvaluator’sAnalysisofGoal1:
Thegranteemetthebenchmarkfor1c-increasingthepercentageoffirsttime,fulltime
AA/ASdegreeseekingNativeAmericanstudentswhocompleteanySJCdegreeor
certificate;11.6%oftheFall2012cohortcompletedaSJCdegreeorcertificatewithinthree
5
years.Althoughitdoesnotshowinthecurrentyear’sdata,itisnoteworthythatthe
completionrateimprovedfrom2014by3.4%,whichissignificant.
Althoughitdoesnotfitneatlyintooneofthegrantreportingcriteriaitisimportanttonote
thatin2015SanJuanCollegesawa50%increaseinthenumberofgraduateswhowere
identifiedasfirst-time,full-time,degree-seekingNativeAmericanstudents.Community
CollegeWeekrankedSanJuanCollegesecondinthenationinthemostrecent“Top100
DegreeProducers2015”.SanJuanCollegewasonlytwograduatesbehindthenumberone
communitycollege.Thesearenotableachievementsrelatedtothegoalofimproving
degreecompletionforNativestudents.
WhileotherbenchmarksforGoalonewerenotmet,significantprogresswasmadefrom
lastyear.Inparticular:
1a–Whileonly15Nativestudentsfromthe2012cohortgraduatedin2015,this
representsa50%increasefrom2014.Alsonoteworthyisthatthecollege-widecomparison
grouponlygraduated34.
1b–Basedonthe2012cohort,only4.2%offirst-time,full-time,AA/ASdegreeseeking
Nativestudentscompletedadegreewithin3years.However,lookingatthelargerSJC
populationshowsonly4.5%ofallstudentscompletedadegreewithin3years.Bringing
thecompletionrateofNativestudentstowithinonlya.03%differencetotheoverall
populationisasignificantaccomplishmentnotreflectedintheannualdataalone.Looking
atseveralyearsdatarevealsthatthereisincrementalyetsignificantimprovementin
Nativestudentcompletionrates.
1d–Hereagainabroaderperspectiveisimportantforunderstandingresults.Basedon
2012cohortstheratioofenrolledNativestudentstoNativegraduateswas23.5compared
toaratioof22.2forallSJCstudents.Withadifferenceofonly1.3,ratesforNativestudents
arestatisticallythesameasforallstudents.Alsosignificantistheimprovementfor2015
withadecreaseof8.8from2014.
TheevaluatorfeelsstronglythateventhoughnotallbenchmarksforGoal1weremet,
significantimprovementsfrom2014to2015aretheresultofprogramchangesbasedon
carefulanalysisofthedata.Inlastyear’sreportseveral“activitiestoimprove
performance”forGoal1wereoutlined.Primarily,thegranteeshiftedfromcollecting
furtherredundantdatatoprovidingservicestoimprovestudentsuccess.Theinformation
gainedfromtheextensive“CohortStudentInitiative”grewintostrategiestoassistthese
studentswithcompletingtheirdegrees.TothisendtheCollegesoughtandwasgranted
reprogrammingofakeypositionfromResearchAssistanttoProgramAdvisor.TheCohort
StudentInitiativedevelopedintothe“CohortConnectionsProgram”withtheProgram
Advisoratitsheart.NativeAmericanstudentsfromeachyear’scohortreceiveadviceon
6
planningmajorsandcareertracks,planningandregisteringforcourseseachsemester,
understandingandobtainingfinancialaidand/orscholarshipsandotherinformationthat
facilitatestheirprogresstowardsadegree.
TheProgramAdvisorprovidedthistypeofextensive,intrusiveadvisingtostudentsfrom
the2012cohortandasthedatashowsthisenabledthemtocompletedegreesinamoretimelymannerandatratesvirtuallythesameasallSJCstudents.Thegrantalreadyhada
TransferSpecialistinvolvedwithextensiveadvisingsothesestaffworkedcollaboratively
toassistNativestudentsinattainingdegrees.AccordingtotheDirector’sAnnualReport,
theCollegesoughtandwasapprovedforfundingasecondAdvisor(usingcarryforward
fundsfromunfilledvacancies)bytheDepartmentofEducationProgramOfficeronOctober
29,2015.TheEvaluatorhasnotbeenmadeawarewhetherthispositionhasbeenfilled.
TheEvaluatorfindstheshiftfromresearchtoactionanexcellentexampleofhowgrant
fundedinitiativesshouldwork.Throughitsresearchthegranteewasabletoidentify
stumblingblocksfordegreecompletionthentookstepstoremedytheproblems.This
year’sdatareflectssignificantimprovementsinNativestudents’degreecompletionbased
onthegrantee’sactionsandtheworkoftheProgramAdvisorandTransferSpecialist.
AsthegranteeentersthefinalyearoftheprojecttheimpactsoftheProgramAdvisorand
TransferSpecialistshouldbestudiedfurtherandconsiderationgiventocontinuingthese
activitybeyondthegrantsinceitappearsthatthislevelofpersonal,intrusiveadvisingis
effective.
Afinalnoteonthegranteenotachievingthestatedbenchmarks:furtheranalysisshows
that,unfortunately,thedownwardtrendfordegreecompletionhasbeenacollege-wide
problem.DegreecompletionhasbeendownforallSJCstudents,notjustNativeAmerican
students;thusthegrant’stargetpopulationmirrorslargercollegewidetrends.Therefore,
thegranteewillmostlikelynotbeabletomeettheambitiousbenchmarksstatedinthe
proposal.
LookingattheInstitutionasawhole,informationfromtheCohortStudentInitiativeraises
questionsabouttheoveralladvisingthatstudentsreceiveatSanJuanCollege.The
advisingmodelforAA/ASdegreeseekingstudentshasbeenthatstudentsareadvisedin
theAdvisingandCounselingCenteruntiltheyhave24credits.Theyarethentransferredto
afaculty“coreadvisor”inanacademicfield.Howeverthereareno“holds”or
requirementsthatastudentseetheir“coreadvisor”sotheprocessbecomesquiteinformal
after24creditswithmanystudentstakingclasseswithoutawarenessoftheirprogress
towardsadegree.
7
SeveralimprovementshavebeenputforwardtoimproveadvisingatSJC.Theimpactsof
thesechangeswillneedtobeanalyzedbytheInstitution.
•
•
•
•
•
•
•
Advisingismovingtoa“CaseManagement”approachwherebyadvisorswillhaveadeeper
understandingandpersonalrelationshipwithasmallercoreorstudentsratherthan
advisingrandomlywhoeverwalksinthedoor.
TheAdvisingCenterpreparesachecklistforfacultyCoreAdvisorswithinformationoneach
student’sGPA,numberofcreditsandotherfactorsthatimpactgraduationandfinancialaid.
Bettercommunicationwithstudentsisbeingpromotedthroughnumerousoutlets.
ProgramEvaluationsareonthePortalincludinga“ProgresstoDegree”optionthatiseasy
forstudentstoaccessanduse.TherehavebeenincreasedpostingsonFacebookwhichalso
hasanalyticdataondegreecompletionstudentcanuse.“Canvass”isbeingexploredasa
methodofcommunicatingwithstudents.
Gatekeepercourses,thosewiththehighestD’s,F’sandWithdrawals,arebeingidentifiedso
thatmoreresourcescanbeallocatedtoensuringstudentsuccessinthem.
TheCollegerecentlyinstitutedarequirementthatanystudentwhoisenrolledina
developmentalcoursemustalsoenrollinaLearningSkills(LRNS)class.PartoftheLRNS
curriculumwillbemappingoutdegreerequirementsforeachstudentandmakingthem
awareofsequencingandprerequisites.
NewStudentOrientationnowincludesinformationabouthowtotrackprogresstowardsa
degreeandtheimpactonfinancialaidofaccumulating“random”credits.
Therehasbeensomeinitialconversationaboutplacing“holds”onstudentswhoreacha
criticalnumberofcreditswithoutattainingadegree.Ideasaboutwaystorequirestudents
toseeanadvisorhavealsobeendiscussed.
Goal2:ImprovetransferratesforNativeAmericanstudents
ActivityObjective(s)
ResultsforYear4
2a.BySeptember2015,increasethepercentage
offirst-time,full-timedegree-seekingNative
Americanstudentswhosuccessfullytransferto
fouryearcollegesby16%
Actual=6.5%
Target=16%
ExternalEvaluator’sAnalysisofGoal2:
ThegranteewassignificantlybelowthetargetforGoal2,improvingtransferratesfor
Nativestudents.Resultsfor2014showedaslightincreasefrom5.1%to7.1%butwere
stillsignificantlybelowtheever-increasingtarget.Resultsfor2015showaslightdecrease
to6.5%offirsttime,fulltime,degree-seekingNativestudentswhosuccessfullytransferto
afour-yearcollege.Thiscompareswith10.2%ofallstudentstransferring.
8
HowevertheFastForwardDirectorhasnotedinhisExecutiveSummary“thegrantis
satisfiedwiththisresultbecausetransferrate,asdefinedbyIPEDS,isactuallydetermined
bythosestudentswhotransferoutwithin150%ofcompletiontimeandthereforedoes
NOTincludethosestudentswhocompleteadegreeandthentransfer”.(emphasisadded)
Theevaluatoragreesthatmeasuringsuccessfortransferbasedonadefinitionthat
excludesstudentswhoearnadegree(SeeGoal1above)isfaulty.
Theevaluatoralsonotesthatthereportnumbersinisolationdonottellthestoryofthe
highlysuccessfulTransferCenter.TheTransferCenterwhichisagrant-fundedinitiative,is
ahighlyvisiblyofficewithafulltimestaffperson.Thenumberofstudentcontactswiththe
Center,thetypesofquestionsaskedandtheactionstakenhavebeencarefullytracked.
AdvisingattheTransferCenterhassteadilyincreasedwith478studentsreceivedone-ononetransferadvisement,upfrom317lastyear.Inaddition,studentsalsovisitedthe
Centerforbriefwalk-ins,emailsorphonecallswithquestionsabouttransferringbutnot
one-on-ontransferadvising.
Inthespringof2014theExternalEvaluatorconductedanextensiveonlinesurveyofall
studentswhohadvisitedtheTransferCenter.Thissurveywillbeconductedagaininthe
finalyearofthegrant.Insummaryfromthelastsurveynearly40%ofstudentsvisitedthe
Centerthreeormoretimes,36%visitedtwiceand25%visitedonce.Whatdidstudents
askabout?Studentscouldcheckmorethanoneresponse.Resultsindicate78%askedfor
informationabouttransferringtoaspecificcollegeoruniversity;71%soughtgeneral
informationabouttransferring;56%askedabouthowSJCcreditswilltransfer;and33%
askedaboutfinancialaidandtransferring.
Didstudentsfindtheinformationhelpful?Resultsindicate41%foundtheinformation
theyreceivedfromtheTransferCenterveryhelpful;45%foundithelpful;and11%
thoughtitwasnotveryhelpful.98%ofstudentswhovisitedtheTransferCenterfoundthe
staffinthecentertobeveryapproachableand90%saidtheirquestionswereansweredin
atimelymanner.86%wouldrecommendtheCentertofriendswhoarethinkingabout
transferring.
TheTransferCenterdefinitelyhasanimpactontransferratesaccordingtothestudents
whovisitedit.73%saidtheyaretransferringtoa4-yearcollegeoruniversity,another
18%arenotsureyetandonly9%arenottransferring.
Inadditiontoin-personvisits,theTransferCenterwebsiteishighlyused.Inyearthreethe
sitewasaccessed12,424times;a16%increasefromthepreviousyearandinyearfour
13,450timesandadditional8%increase.TheevaluatorfindstheTransferCentertobean
effectiveprogramwithsignificantpositiveimpactsandincreasingstudentusage.
9
Infurthereffortstoimprovetransferratesthegranteehasundertakenseveralactivities
overthepastseveralyears.
Otheractivitiestoincreasestudentstransferringduringyear4include:
•
•
•
Sevenworkshopswereconductedontransferringwith90studentsattending.
ThesecondannualTransferWeekwasheld;over200studentsattendedtheTransferFair
withover40college/scholarshiprepresentatives.
CollegeExplorationTripsforstudentswhoareinterestedintransferring.Eighttripswere
sponsoredandwerewellattendedwith36Nativestudentsparticipating.
TheFastForwardGranthasalsomadesignificantprogressinestablishingArticulation
AgreementstofacilitatetransferfromSJCtofour-yearuniversities.InYear4thegrantee
addedsevennewAgreementswithNMHighlandsUniversityinspecificmajors.Intotal29
ArticulationAgreementshavebeensignedwithNMHighlandsandonewithUniversityof
NewMexicosincethegrantbegan.TheEvaluatorwillpursuereasonswhythereareso
manyagreementswithoneinstitutionandnonewithotherfour-yearinstitutionsinthe
area.
AlthoughtheyarenotformalizedArticulationAgreements,thegranteehascontinuedto
expandthenumberoffour-yearinstitutionswithwhichtheyhavecollaboratedand
establishedpathwaystofacilitatetransfer.Duringthisgrantyearnewinstitutionsinclude:
ColoradoSchoolofMines,ColoradoStateUniversity,theInstituteofAmericanIndianArts
andNavajoTechnicalUniversity.Thisbringsthetotalto18institutionswithwhichthe
granteehasmetanddiscussedSJCstudentstransferring.Duringthefinalyearofthegrant
itisreasonabletoascertainhowmanyofthesemightbecomeformalAgreements.
InsummarytheEvaluatorfindsthatalthoughtheFastForwardprogramfailedtomeetthe
benchmarkforGoal2,instrivingtoimprovestudenttransferratestheyhaveinitiated
numeroushighlyeffectiveprogramstoenhancestudents’abilitytotransfertofour-year
collegesoruniversities.
Goal3:ImprovecoursesuccessratesinkeyAA/ASdisciplines
ActivityObjective(s)
3a.BySeptember2015Improvecourse
successratesforallstudentsinEnglish,
Business,ScienceandMathematicsto68%
ResultsforYear4
Actual=64.9%
Target=68%
10
ExternalEvaluator’sAnalysisofGoal3:
Asstatedinpreviousreportstheevaluatorfindstheseoutcomesinterestingbecausethe
granthaslimitedprogramactivitiesthatimpactcoursesuccess.TheCollege’sotherTitle
IIIgrant,the“Launch”grant,hasnumerousprogramsaimedatimprovingsuccessratesfor
Nativestudentsinmathandscienceandcanshowdirectcorrelationsbetweenprograms
andstudentsuccess.Howeversuccessfor“allstudents”innumerousdisciplinesseems
moredependentonenrollmenttrends,institutionalsupportservices(suchastutoring,
mentoring,facultymentoring,etc.,)andhighschoolpreparationthanonasinglegrantwith
nodirectprogrammingthatalignswiththegoal.
Nonetheless,inyearfour,65%ofstudentsweresuccessfulinkeydisciplines,upfrom60%
inyearthree.Asnotedwithseveralobjectives,theoverallpictureismorerevealingthan
theannualbenchmarknumbersinisolation.
Programactivitieswhichshouldcontributetothisincreasingstudentsuccessincludethe
extensiveandhighlyratedfacultydevelopmentofferedbythegrant(anddiscussedin
detailinGoal4)Whileitisdifficulttoshowacausalrelationshipbetweenfaculty
developmentandstudentperformancewenonethelessknowthatfacultywhohave
participatedingrant-sponsoredtrainingssaytheyaremoreawareoftheneedsofNative
students,haveagreaterstrategiestoeffectivelyteachtheircoursesandfindthemethods
usefulintheirclassrooms.Afteryearsoffacultydevelopmentactivitiesaimedatimproving
studentsuccesstheevaluatorbelievestheincreasesinstudentsuccessinkeycoursesisto
someextentattributabletothistraining.
Goal4:ExpandcapacityforSanJuanCollegetobetterserveNativeAmerican
AA/ASStudents
ActivityObjective(s)
ResultsforYear4
4a.BySeptember2015,increasethepercentage
ofAA/ASGeneralEducationsectionsoffered
outsideofM-Th8:00am–4:00pmtraditional
formatto47%
Actual=39%
Target=47%
11
4b.BySeptember2015,increasethenumberof
full-timeandadjunctfacultywhoparticipatein
grant-sponsoredfacultydevelopmentto24
Actual=27
Target=24
4c.BySeptember2015,increasethenumberof
studentswhoparticipateingrant-sponsored
assessmentoftechnologyskillsto1300
Actual=1,132
Target=1300
4d.BySeptember2015,increasethenumberof
studentswhoparticipateingrant-sponsored
remediationoftechnologyskillsto300
Actual=127
Target=300
4e.BySeptember2015,increasethenumberof
distinctwebpagesenhancedeachyearbythis
grantto60
Actual=26
Target=60
ExternalEvaluator’sAnalysisofGoal4:
AcriticalcomponentoftheFastForwardgrantistoexpandSanJuanCollege’scapacityto
betterserveNativeAmericanAA/ASstudents.Goal4dealswitharangeofactivities
includingdatacollection.Objective4a“toincreasethepercentageofAA/ASGenEd
sectionsofferedoutsidethetraditionalformat”dependsofupper-leveladministrative
decisionsaboutwhichcoursestoschedulewhen.Tohelpinformthesedecisionduring
Year2theFastForwardgrantconductedanextensivesurvey,theCommunityInputSurvey
with622respondentsfromawiderangeoflocalorganizations,businesses,andindividuals.
TheCommunityInputSurveyincludedquestionsaboutcourseschedulingandenrollment.
Duringyearfourtherehasbeenaslightrisewith39%ofcoursesofferedoutsidethe
traditionalM-F,8–4formats.Thisrepresentsa4%increasefromyears2and3.In
particularlatestartingclassesandSaturdayCollegehaveexpanded,inpartbecauseof
valuabledataprovidedbytheFastForwardgrantwhichindicatedadesireforAA/AS
programsaswellaseveningandweekendclasses.
Anothertoolforclassscheduling,ClassTracks,waspurchasedwithFastForwardfundslast
yearandimplementedduringyear4.ClassTracksfacilitatesschedulingclassesand
analyzespatternsofenrollmentformaximumqualitymanagement.Asystematicapproach
toclassschedulingforAY2015/16wasinstitutedbasedoninformationfromClassTracks
andincludingincreasingthenumberoflatestartingclassesandcontinuingSaturday
CollegeontheEastCampus.
Objective4b,increasingfacultydevelopmentopportunities,hasbeenanareaofgreat
success.Thetargetforthisyearwasfor24facultyparticipants;27unduplicatedfaculty
12
participatedingrant-sponsoredfacultydevelopmentactivities.Inyearthreethegrantee
farexceededthegoalwith42unduplicatedfacultyparticipating.TheProgramDirector
attributesthisyearsdeclineinnumberstothecancellationofthesummerworkshop“The
BistiWritingProject”whichwasofferedthepasttwoyears.
Successfulfacultydevelopmentactivitiesduringyear4include:
•
•
•
•
AttendingnationalconferenceswithafocusonNativeAmericanEducationandstrategies.
(14participants)
Aonedayinstitute,basedontheBistiWritingProject,focusedonWritingAcrossthe
CurriculumandDiverseLearners(9faculty)
FastForwardsponsoredthe4thAnnualNativeLeadersSeries.Thelecturewasattendedby
morethan160students,facultyandstaff
NativeLeaderSpeakerSeriesworkshop(9participants)
Objectives4c,”increasingthenumberofstudentswhoparticipateingrant-sponsored
assessmentoftechnologyskills”and4d,“thenumberofstudentswhoparticipateingrantsponsoredremediationoftechnologyskills”arecloselyrelated.Theassessmentof
technologyskillstakesplaceduringnewstudentorientationandinafewonlineclasses.
TheCollegeexperiencedadeclineinenrollmentforthefall2014sofewerstudentstook
theassessmentthananticipatedinthegrant.
Ofthe1,132studentswhowereassessedfortheirtechnologyskills,only110wererated
“lowreadiness”.Obviouslythereismuchlessneedfortechnologyremediationthan
originallythoughtwhentheproposalwaswrittenand300studentswereanticipated.
Thenumberofstudentswhoparticipatedingrant-sponsoredremediationcloselyfollows
thetestresultswith127studentsenrollingintechnologyremediation.Itisnoteworthy
thatthisisanincreaseof103studentsfromthepreviousyear.Theevaluatorfoundit
interestingthatthereisnorequirementtoparticipateintechnologyremediationbut
studentsseemtobewelcomingtheopportunitytoimprovetheseskills.
OverthefirstthreeyearsofthegrantoneofthegreatestimpactsofFastForwardrelatedto
4e,“Increasingthenumberofdistinctwebpagesenhanced”.Forthisobjectivethegrantee
hasexceededbenchmarksinyears2and3with96distinctwebpagesenhancedduring
thattime.Inyear4only26webpageswereenhancedthroughgrantactivities.Thisisin
partbecausethecollegeinitiatedacampus-widewebredesigntherebyhinderinggrantbasedeffortsuntilthecollege-widesystemwasfinalized.Becauseoftheoutstandingjob
thegrant’stechnologystaffhasdone,theywereaskedtotakeanactiveroleinthecollegewideeffortservingontheWebAdvisoryCommitteeandtheCollegeWebsiteRedesignTask
Force.
13
Goal5:ImprovetheabilityofSJCpersonneltocollect,analyzeandutilizestudent
datatobetterserveNativeAmericanstudents.
ActivityObjective
5a.BySeptember2015,administerretention
assessmentsto1000students
ResultsforYear3
Actual=672
Target=1,000
5b.BySeptember2015,surveyarepresentative
sampleof25%ofNativeAmericanstudents
regardingsatisfactionandneeds
Actual=ResultsonSENSEwillnotbe
availableuntilMarch,2106
Target=25%
5c.BySeptember2015,basedonsurveysofSJC
personnel,80%ofsurveyrespondentswill
indicatetheyareusinggrant-sponsoreddatain
strategicplanningfortheyear
Actual=64%
Target=80%
5d.BySeptember2015,buildatleast10
customizablereports
Actual=77
Target=10
ExternalEvaluator’sAnalysisofGoal5:
ImprovingtheCollege’sabilitytocollect,analyzeandusestudentdatatobetterserve
NativeAmericanstudentshasbeenstrengthoftheFastForwardGrant.Retentionand
satisfactionassessments(5a)werepreviouslyrequiredduringNewStudentOrientationfor
allincomingstudents.However,therewereconcernsthatstudentswerebeingover
assessedduringOrientationsothenewinstrument“SENSE”(SurveyofEnteringStudent
Engagement)wasonlyadministeredto672studentswhoexhibited“non-cognitive
concerns”i.e.nonacademicconcernssuchastransportation,childcare,housing,etc.This
nationallynormedinstrumentwasadministeredduringthefirstthreeweeksto
systematicallyunderstandtheearliestexperiencesofentering/newstudents.Those
studentswhoweredeemed“highrisk”werereferredtotheStudentAchievementCenter
andpairedwithanAchievementCoach.
UnfortunatelyresultsfromSENSEwillnotbeavailableuntilMarch2016sotheevaluator
hasnocommentsonthefindings.Thislatereportingalsomeansthatwhilethegrantee
expectstomeetthebenchmarkfor5b(arepresentativepercentageofNativeAmerican
students{25%}surveyedregardingsatisfactionandneedseachyear)thereisnodata
availableatthistime.
14
Collectingdatahasbeenpartofthegrantsinceitsinceptionbutitistheuseofdatathatis
criticalforinstitutionalimprovement.BasedontheFastForwardgrant’ssurveyofSJC
personnelinkeydecision-makingpositions,64%ofrespondentsindicatedtheyare“using
grant-sponsoreddatainstrategicplanning.“Thisisaslightdecreasefromlastyearwhen
68%wereusinggrant-sponsoredwhichwasveryclosetothebenchmarkof80%.The
Directorstatesinhisreportthatthedecrease“canbeattributedtohighernumbersof
personnelturnoverandeliminationofcertainemploymentpositionsduringthegrantyear
whowerehistoricusersofgrantdata.”Theevaluatorbelievesthisrelatestotheendofthe
TitleIII“Launch”whichwascloselyalliedwithFastForward.Unfortunatelynoneofthe
grant-fundedpositionswerepickedupbytheinstitutionresultinginasignificantturnover
ofpeoplewhowerecloselytiedinwithdatageneratedbyFastForward.
Nevertheless,overthecourseofthegrantthegranteehasdoneanexcellentjobofbuilding
asolidinfrastructureforcollectingdataandreporteddatausageincreasedfrombelow
60%to68%inYear3and64%inYear4.ThisindicatesthatamajorityofCollegedecision
makersrelyongrant-sponsoreddataforstrategicdecisions.Duringthisfinalgrantyear
theevaluatorencouragesthegranteetosystematicallyinformandeducateallnew
personnelaboutthedataanditspotentialusages
AnoutstandingareaofachievementfortheFastForwardgranthasbeentheabilityto
providecustomizablereports(5d).Thesereportsallowdecisionmakerstogetanswersto
specificquestionsinatimelymanner.Likeallinstitutions,theInstitutionalResearchOffice
isoverwhelmedandcannotprovideallofthedataneededinatimelymanner.TheFast
Forwardgranthasprovidedvaluableservicesthroughcustomizablereports.In2014/15
thegranteeprovided77suchreports,farexceedingthebenchmarkof10.Thebenchmark
forthisgoalhasbeenexceededinthismannereveryyearofthegrant.Specifically
DashboardswereupdatedontheNativeAmericanGraduateSurvey,NativeAmerican
StudentSuccessRatesandtheNativeAmericanStudentProfile.
In2015theNativeAmericanStudentProfilewasupdatedandenhancedwithdatatotrack
retention,coursesuccessanddegreecompletionrates.The2015AmericanIndian
GraduateSurveywasalsorevisedanddistributed.
15
CriticalQuestion2:TowhatextenthastheCollegedelivered
highqualityprogramming?
Grantstaffhasbeendiligentincollectingdataonthequalityofitsprograms.
The“CollegeExplorationTrips”programissystematicallyevaluated.Theinstrumentwas
developedbytheExternalEvaluatorandtheProjectDirectorinYear1andhasbeenused
consistentlysincethen.Themethodologyisapre/postquestionnairewherestudentsare
askedtheirlevelofknowledgeabouttransferrequirementsandprocesseswiththespecific
schooltheyarevisiting.Theyarealsoaskedabouttheircomfortlevelwithaccessing
resourcesatthetargetinstitution.Inreviewingthedata,theevaluatornotesasignificant
increaseinstudent’sknowledgeofthetransferprocess,whatdocumentationisrequired,
whatSJCcoursestransferandwhendeadlinesare.Studentsarealsosignificantlymore
knowledgeableaboutresourcesavailabletothematthetargetinstitutionandmore
comfortableaccessingthoseresourcesbecauseoftheexposuretheygainedthroughthe
ExplorationTrip.TheEvaluatorhasreviewedallofthedataandissatisfiedthatthisisa
highqualityprogramthatprovidesanoutstandingservicetoNativestudentswhoare
consideringtransferring.
TheeffectivenessandimpactoftheTransferCenterwascomprehensivelyevaluatedin
Year3.ThissameevaluationwillberepeatedinSpring2016togainfinalsummativedata
onstudents’satisfactionwiththeCenter.Sincethenumbershaveincreasedsteadilyover
theyearsthefinalsurveywillhavemorestatisticalrelevance.
TheSummer(Bisti)WritingProgramwillalsobere-evaluatedinYear5forsummative
datatoascertaintheextenttowhichfacultyhaveincorporatedtechniquesintotheir
classes.
Theevaluatorhasreviewedalloftheactivitiesundertakenbythegrantandisimpressed
bytherangeofactivitiesconductedbytheFastForwardstaff.Eachactivityiscorrelated
withone(ormore)ofthegrantobjectives.AlistofactivitiestakenfromtheAPRisshown
below:
COMPONENT 1: Increasing Post-Secondary Success
•
•
•
•
Transfer Center and Transfer Initiatives (Objectives 1a, 1b, 1c, 1d, 2a, and 3a)
Building Partnerships with Native American Servicing Four-Year Institutions (Objectives 1a, 1b,
1c, 1d, 2a)
Developing Alternative Delivery Formats (Objectives 1a, 1b, 1c, 1d, 2a, 3a, 4a)
Strengthening Professional Development for Faculty (Objectives 3a, 4b, 5c)
16
•
•
Creating a Technology Readiness Assessment and Remediation Program (Objectives 4c, 4d)
Enhancing Interactive Electronic Student Resources (Objectives 2a, 4e)
COMPONENT 2: Enabling More Databased Decision Making
•
•
•
Conducting a Pilot Research Project: Retention Assessment (Objective 5a)
Collecting Extensive Quantitative and Qualitative Data to Improve Native American Student
Success (Objectives 1a, 1b, 1cc, 1d, 2a, 3a, 5b, 5d)
Empowering Data-Based Decision Making (Objectives 3a, 5a, 5b, 5c, 5d)
CRITICALQUESTION3:Towhatextenthasthegranteeuseddatato
improveprograms?
TheFastForwardgrantisadata-drivenprogramonalllevels.Atthemicrolevelall
activities,suchasthoselistedabove,areevaluatedforeffectivenessandresults
systematicallyusedtoimproveprograms.Inparticularprogramsthatprovidedirect
servicestostudentssuchastheTransferOffice,theCollegeVisitsprogramandtheWriting
Project,collectfeedbackfromparticipants,analyzethefeedbackanduseittoimprove
programs.
AtthemacroleveltheFastForwardgrantprovidesdatatodecisionmakersattheCollege.
Forexamplegrantstaffhaveextensivelycollecteddataonalternativeformatsforcourses.
Thisdatahasbeenpresentedtothehighest-leveladministratorswhowillultimatelymake
suchdecisions.TheFastForwardstaffhasalsoconductedallofthesurveysoutlinedinthe
grantandprovidedresultstoCollegeofficials.
Oneofthecomponentsofthegrantistoenablemoredatabaseddecision-making.Tothis
endthegranteedevelopedandmaintainedtheNativeAmericanStudentProfilethatis
updatedannuallytoincorporatecurrentdataonNativeAmericancohortstudents.The
granteealsodevelopedtheCohortStudentInitiativegatheringsignificantnewdataon
stumblingblocksexperiencedbyNativestudentsthatresultedinlowdegreecompletion
rates.ThegranteethentookthatdataandwithProgramOfficerpermissionbegan
deliveringservicestostudentsviatheCohortConnectionsProgram,mostnotablyaddinga
Advisortoprovideongoing,intrusiveadvisingasdiscussedextensivelyabove.
Inconclusionthegranteeisusingdatatoimprovegrant-fundedactivitiesandtheSanJuan
Collegeisusingdatageneratedbythegranteetoimproveprogramsandservicesatthe
Institution.
17
CRITICALQUESTION4:TowhatextenthastheCollegefulfilledthe
legislativeintentoftheTitleIIIAct?
ThepurposeofthelegislationistoenableNativeAmerican-servinginstitutions,likeSan
JuanCollege,toimproveandexpandtheircapacitytoserveNativeAmericanandlowincomeindividuals.SpecificallytheSJCFastForwardProgramfulfillsthelegislativeintent
byincreasingpost-secondarysuccessofNativeAmericanstudentsandbyenablingmore
databaseddecisions.AllofSanJuanCollege’sFastForwardgrantactivities,programsand
personnelaredirectlyengagedinfulfillingthelegislativeintentoftheTitleIIIAct.Thereis
novariancefromthelegislativeintentandtheevaluatorbelievestheCollegeisdoingan
exemplaryjobofputtingTitleIIItheoryintogoodpractice.
CRITICALQUESTION5:TowhatextenthastheCollegeusedgrantfundedinitiativestofulfillitsmission?
AlloftheprogramsandactivitiesthatarepartoftheFastForwardgrantdirectlyrelateto
SanJuanCollege’smissionasaninstitutionservingNativeAmericanStudents.Themission
oftheCollegereads,“Inspireandsupportlifelonglearningtoachievepersonaland
communitygoalsbyprovidingqualityeducation,servicesandculturalenrichment”.Grantfundedactivities,extensivelydiscussedearlier,willimproveSanJuanCollege’sserviceto
Nativestudents.FastForwardgrantinitiativesspecificallyrelatedtheCollege’smission
include:
QualityEducation:Professionaldevelopmentopportunitieshelpfacultyimprovetheir
instructionalmethods,qualityofeducationandclassroompedagogy.TheupdatedNative
AmericanProfileandDashboardsonstudentsuccessratesenabledatabaseddecisions
regardingstudents’educationalneedsandchallenges.TheCohortStudentInitiative
trackedAmericanIndiancohortgroupsontheiracademicprogress,retention,course
completionanddegreecompletion.ThenewlyimplementedCohortConnectionsprovides
extensive,intrusiveone-on-oneadvisingtostudentstoaidthemindegreecompletion.
QualityServices:Thegranthasformalizedadvisingtosupportstudentsinthetransfer
processbycreatingacentralTransferCenteroncampusandaninteractiveTransferCenter
website.Thegranteehascommunicatedtransferinformationandopportunitiesto
studentsthroughclassroomvisits,workshops,bulletinboards,flyers,posters,transfer
explorationtripsandsponsorshipofTransferWeek.ThroughitsFacebookpagethe
TransferCenterhasexpandedstudentscommunicationtoincludesocialmedia.TheFast
ForwardgranthassupportedqualityservicesbyparticipatinginNativeAmericanCenter
eventsandActivities,andparticipatinginCollege-wideeventssuchasStudentRush,
StudentResourceDays,Convocation,etc.
18
CulturalEnrichment:
TheFastForwardgrantstaffhasbeenactivelyinvolvedintheCollege’sculturalenrichment
opportunities.TheyhaveparticipatedinandsupportedtheNativeAmericanCenterwith
eventsandactivities.TheyassistedwithSJCPow-Wow,andassistedwithcollege
informationboothsattheSJCountyFairandNorthernNavajoNationFair.TheGrant
sponsoredthefourthannualNativeLeadersSpeakerSeriesattendedbymorethan160
students,faculty,staffandcommunitymembers.
Evaluator’sConclusions
DuringYear4thegranteehasmadeprogressinmeetingtheobjectivesbutinsomecases
hasnotmettheannualperformancebenchmarks.Thisreportcontainsananalysisof
objectivesthatwerenotmetandprovidesamorecontextualunderstandingof
accomplishmentsfortheyear.
TheEvaluatorconcludesthatfortheObjectiveswherethegranteehasdirectprogramming
thereissuccessandsignificantimpact.HowevermanyoftheObjectivesinthegrantare
largerinstitutionalobjectives(suchascompletionratesandtransferrates)thatthegrantee
hasminimumabilitytoimpactinanystatisticallyrelevantway.Inmanycases,downward
trendsinthegrantmirrordownwardtrendsattheinstitution;grantandinstitutional
enrollmentandcompletiontrendsshouldbeanalyzedtogether.
OneareawheretheEvaluatorhasconcernsisthelengthoftimetheCollegerequiresfor
fillingvacanciesingrant-fundedpositions.FillingtheAdvisorpositiontookatotalofsix
months,fromOctobertoFebruary,twothirdsoftheAcademicYear.FirsttheCollege
LeadershipTeamhadtoapprovethereposition,whichitdidinSeptember,thenthe
ProgramOfficerapprovedthechangeinOctober.Bythetimethepositionwasapproved,
advertisedandinterviewed,mostoftheyearwasgone.Obviouslytherearepoliciesand
proceduresthatmustbefollowedbutitseemsifmoreofthecollege-levelplanningand
approvalcouldtakeplaceoverthesummerthenstaffwouldbeinpositionsearlierand
havemoreoftheAcademicYearinwhichtoimplementtheirjobresponsibilities.Filling
theweb-designerpositiontookthreemonthswhichseemsmorereasonablegiventhatthe
CollegeLeadershipTeammustapprovethevacancy.Butasitwasonlyareplacement
positionitseemsitcouldhavebeenfilledmorequicklyespeciallysincethepositionwas
vacantfromSeptemberthroughNovember,essentiallyhalfoftheAcademicYear.
Inconclusion,theEvaluatorfindsthatthegranteeisofferinghighqualityservicesaligned
withgrantobjectives.Asystematicevaluationprocesshasbeenfollowedandresults
19
reportedinatimelymanner.Participantsingrant-sponsoredactivitiesratethemhighly
andindicatetheyareeffective.Thegranteeisextremelyconscientiousinusingdatato
improveprograms.Overall,thegrantiswell-managed,responsivetoinputandagreat
benefittoSanJuanCollegeinfulfillingitsMission.
Forfurtherinformationorclarificationonthisreport,pleasecontactLauraM.Godfrey,
[email protected](970)749-0148
20