AcademicAssessment
101:Workbook
February2017
2
AcademicAssessment101:Workbook
Thegoalsofthisworkbookare:
• toprovideanoverviewofacademicprogramassessmentatNazarethCollege,
inaccordancewithouraccreditor,theMiddleStatesCommissiononHigher
Education.
• toguideyouthroughtheprocessofcreatinganassessmentplanand
completingyouracademicprogramassessmentreport.
TheAcademicAssessmentCommitteeencouragesyoutotakethefollowingsteps:
1.Readandworkthroughthisguideand/orviewAssessment101video
2.Considerwhereyoucoulduseadditionalsupportorguidance
3.SetupanappointmentwithSLOAssessmentCoordinator
TableofContents
PartI:CreatinganAssessmentPlan
3
StudentLearningOutcomes
3
CurriculumMapping
6
Evidence
8
Criteria
9
Summary 10
PartII:ReportingonAssessmentResults
11
Findings
11
ActionsandClosingtheLoop/FollowUp 12
RoleoftheAcademicAssessmentCommittee
13
RecommendedAssessmentResources
14
Thepurposeofassessmentistousedatainordertobetterunderstandaprogram’s
strengthsandweaknessesandtohaveasystemformakingimprovementstothe
program,curriculumandpedagogy.Assessmentshouldnotbeasolitaryactivity
thatoccursatyourdesksimplyforthepurposeofannualreporting.
Pleaseengageprogramfacultyinongoingconversationsthathelpyouto
collaborativelycreateameaningfulassessmentplanandtoanalyzestudentworkso
thatyoucanmakeinformed,well-consideredimprovementstoyouracademic
program.
NazarethCollege,AcademicAssessment101:Workbook
3
PartI:CreatinganAssessmentPlan
An“assessmentplan”isjustthat:aplanforhowyouwillassessandanalyzethe
effectivenessofyouracademicprogram.ComponentsA-Dbelowdescribe
importantstepsincreatinganassessmentplan.
A.StudentLearningOutcomes(SLOs)
Astudentlearningoutcome(SLO)isaclearstatementofyourcourseorprogram
objectives,including,thoughnotlimitedto,thefollowing:expectedknowledge,
comprehension,application,analysis,synthesis,evaluationskills,attitudes,or
competenciesthatstudentsareexpectedtoacquireinacourseorprogram.
SLOsatNazarethareassessedatthreedifferentlevels:
College-levelSLOsareassessedbytheSLOAssessmentCoordinator.
SLOs:StudentswhograduatefromNazarethCollegewill:
Institution
1. Havedemonstratedabroadanddeepunderstandingofthemselvesandthe
worldthroughthemultipleperspectivesoftheliberalartsandsciences(UG)
2. Havedemonstratedcriticalthinkingandknowledgepertinenttotheir
respectivedegreeprogramandthedemandsoftheirdiscipline
3. Bepreparedtoengageasinformedmembersofthelocalandglobalcommunity
TheseSLOsareassessedbasedonoutcomesofstudentlearningassessmentatthe
programlevel.
Program-levelSLOsareassessedbyprogramfacultyeachyearwiththepurposeof
usingassessmentdatatoinformcurriculardecisionsandimproveprogramquality.
Program
SLOsatthislevelarebothmorespecificthan,andalignedto,institutionalSLOs.
EachprogramdevelopsitsownSLOs.
ProgramSLOsaremadeavailablepubliclyontheAcademicAssessmentwebsiteand
shouldbeupdatedandrevisedasnecessaryforcontinuousimprovementofprogram.
Course
Course-levelSLOsareassessedbyindividualcourseinstructors.
SLOsatthislevelarethemostspecificandarealignedtoprogramSLOs.
Outcomesofstudentlearningincoursesareoftenthedatausedinprogram
assessment.
NazarethCollege,AcademicAssessment101:Workbook
4
NOTE:AnnualreportingthatisdueonOctober1eachyearisconcernedprimarily
withprogram-levelassessment.Whilealllevelsareimportantforcourse,
programandinstitutionalimprovement,theremainderofthisworkbookdeals
primarilywithprogram-levelassessment.
SLOsshouldalwaysbewrittenintermsofwhatstudentswillbeabletodo.
Thisformatwillbestenableyoutoidentifyevidencetolateranalyzeinorderto
determinewhetheryourgoalhasbeenmet.FramingyourSLOstatementsinthis
wayhelpstoensurethatyourSLOsaremeasurable,attainable,andfocusedon
results.
AcceptableSLOs:
Studentswillbeabletoidentifythefourdimensionsofliteracyandexplainhow
theyrelatetoclassroominstruction.
Studentswilldemonstrateunderstandingoftonalandpost-tonalmusic.
UnacceptableSLOs:
ThecurriculumemphasizesX,YorZ
TheprogrampreparesitsstudentsforX,YorZ
StudentsparticipateinX,YorZ
Practice:Inthespacebelow,usethestudentswillorstudentswillbeableto
formattodraftanSLO:
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Reflect:TakealookatthepracticeSLOyouwroteabove.Isitmeasurable?Whyor
whynot?
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
NazarethCollege,AcademicAssessment101:Workbook
5
Whileactionslike“appreciate,”“value”or“understand”aredifficulttomeasure,
therearecertainverbsyoucanchoosetoincludeinyourSLOstatementsthat
ensureyourSLOsaremeasurable.
Examples*ofMeasurableVerbsforSLOStatements
Knowledge:Define,Describe,Recall,Reproduce,Select
Comprehension:Explain,Translate,Identify,Compare,Summarize
Application:Apply,Use,Demonstrate,Construct,Solve
Analysis:Analyze,Calculate,Diagnose,Theorize,Debate
Synthesis:Develop,Create,Design,Integrate,Compose
Evaluation:Justify,Defend,Assess,Evaluate
*YouarenotlimitedtousingonlytheseverbsinyourSLOs.Theseareexamples.Youshould,
however,attendtovariouslevelsofBloom’sTaxonomy,listedabove.
Thisdoesnotmeanthatyoucannothavewordslike“appreciate”inyourSLO
statements.Itjustmeansthatyouneedtothinkcarefullyabouthowstudentswill
demonstratetheselessactionablewordsinordertoprovideyouwithevidenceyou
cananalyze.Forexample:
Insteadof:Studentswillbeinspiredbyapassionforintellectualengagementwith
others
Considerthis:Studentswillanalyzetheirunderstandingsandbeliefsabout
intellectualengagementwithothers.
Ifyouwouldliketounderstandsomethingaboutstudents’inspiration,makesure
thatyourassignmentguidelinesandrubric(i.e.,evaluationcriteria)includea
componentrelatedtotheirinspirationfortheirunderstandingsandbeliefs.
ProgramsatNazarethtypicallyhavebetween3to6SLOs.
ThenumberofSLOsyourprogramhaswilldependuponmanythings,suchas
professionalstandardsinyourdiscipline,requirementsofaccreditors,oryourown
goalsforstudents’competencies.Thereisno“right”numberofSLOsthatyour
programshouldhave.Thenumberwilllookdifferentfromprogramtoprogram.
NazarethCollege,AcademicAssessment101:Workbook
6
Reflect:Takeamomenttothinkaboutthestudentsinyourprogram.Whatkinds
ofdiscipline-specificthingsdoyoumosthopethattheywillknowand/orbeableto
doupongraduation?Whatarethethingsyourprofessionalstandardsand/or
accreditingbodysayyourstudentsshouldknow/beabletodo?
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Consider:Circlecompetenciesyouwrotedownthatseemtobethemost
important.ConsiderwhichoftheseshouldberepresentedbyoverarchingSLOsfor
yourprogram.
WriteyourSLOs:Inthespacebelow,draftsomeSLOsforyourprogrambasedon
themostimportantcompetenciesyoudescribedabove.Ifappropriate,youmayalso
wishtolookattheSLOscurrentlyinplaceforyourprogramandmakeanyneeded
revisionstostyle,languageorcontent.
SLOsfor_______________________________________(program)
1.
2.
3.
4.
5.
B.CurriculumMapping
Acurriculummapisausefultoolinassessmentplanning.Itusuallylookslikea
matrixthatdisplaystherelationshipbetweenaprogram’sSLOsandthecoursesin
thatprogram.Thisdocumentallowsaprogramtoarticulatehow/whethereach
courseisaddressingorassessingeachSLO.
NazarethCollege,AcademicAssessment101:Workbook
7
Itisimportantthateachprogrammaintainsacurriculummapsothatlearning
outcomesareclearlylinkedfromthecoursetotheprogramlevel.Thiswillhelpyou
toensurethatprogram-levelSLOsare,infact,assessedatthecourselevel.A
curriculummapshouldbesubmitted(orupdated)eachOctoberalongwith
assessmentresults.
CompletetheMap:Usingthetemplatebelow,draftacurriculummapforyour
program.
Reflect:TowhatextentiscreatingaCurriculumMapusefulforyou?Whatdoesit
allowyoutodoorthinkaboutthatyoumaynototherwise?Ifyouhavenotyet
createdacurriculummap,whatbarriersexistordoyouanticipate?
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
ACurriculumMaptemplatethatyoucaneditandsavecanbefoundonour
AcademicAssessmentwebsitehere:https://www2.naz.edu/academicaffairs/academic-assessment/workshop-materials-and-resources/
NazarethCollege,AcademicAssessment101:Workbook
8
C.Evidence
EachSLOshouldhaveparticularpiecesofevidenceassociatedwithit.Evidenceis
datathatyouwillcollectthatwillhelpyoutoanalyzestudents’progresstowardan
SLO.Forexample:
IfoneofmySLOsis:
Studentswillbeabletoidentifythefourdimensionsofliteracyandexplainhowthey
relatetoclassroominstruction
IneedtoidentifysourcesofevidencethatIcanrelyupontotellmewhetherornot
studentsare,infact,abletodothesethings.
Iwilllookattwosourcesofevidence:
1)afinalpaperinacoursemid-programwherestudentsdescribeandanalyzethe
fourdimensionsofliteracy(directevidence),and
2)asurveyattheendoftheprogramthatincludesaquestionabouthowwellthey
thinktheyunderstandthefourdimensions(indirectevidence)
Byanalyzingoutcomesonbothoftheseassessments,Icangetagoodsenseof
whetherandhowwellstudentshavemetthegoalIset.
Therearetwodifferenttypesofevidence:DirectevidenceandIndirect
evidence.
● Directevidenceisanassessmentthatisadirectdemonstrationofstudents’
abilityorknowledge.Theirabilityisobservedthroughdirectsourcesof
evidence.
Examplesofdirectevidenceinclude(butarenotlimitedto)exams,
essays,courseprojects,performances,portfolios,orquizzes.
● Indirectevidenceisanassessmentwherethestudentdetermineswhator
howwelltheyhavelearned.Theirskills/abilitiesareinferredthrough
indirectevidence.
Themostcommonlyusedindirectevidenceissurveydatawhere
studentsrespondtoquestionsaboutwhattheybelievetheyhave
learned.Theymayalsodothisthroughreflectivewriting.
NazarethCollege,AcademicAssessment101:Workbook
9
Ifoneofyourevidencesourcesisindirect(e.g.,asurvey),youshouldalsohavea
directsourceofevidenceinplace.
Practice:UsingoneoftheSLOsyoudraftedintheprevioussection,comeupwith
twosourcesofevidenceyoumightanalyzetohelpyouunderstandwhetherornot
studentshaveachievedtheoutcomeyoudesired:
SLO:
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
EvidenceSource#1–Direct
EvidenceSource#2–Indirect
____________________________________________
____________________________________________
____________________________________________
____________________________________________
Considerhowyoumightuseevidencefromvariouspointsinaprogramtogether
inordertobestunderstandwhatstudentsknow/areabletodo.
Onecouldeasilylookatoutcomesatcompletiontofigureoutwhetherstudentshave
achievedaprogram’sSLOs(e.g.,capstoneproject,certificationexam,thesisproject,
etc.).However,oncestudentsaredonewithaprogram,it’stoolatetogobackand
addressweaknessesforthosestudents.
AcrossallofyourSLOs,werecommendlookingatevidencethatrepresents
students’knowledge/abilitiesatvariousdifferentpointsinyourprogram.Afterall,
youarenotsimplyassessingstudents,youareassessingthestrengthsand
weaknessesofyourentireprogram.Lookingatevidenceproducedinvarious
coursesandprogramexperiencesgivesyoutheabilitytomakedata-driven
improvementsatvariouspointsintheprogramandseealmostimmediatelyifthey
areworking.Afterall,improvementisthewholepurposeofassessment!
D.Criteria
Eachevidencesourceyouincludeshouldhavesomecriteriaassociatedwithit.The
criteriaarewhatyouwilllookforinordertoknowwhetherornottheSLOhas
beenmet.Forexample:
NazarethCollege,AcademicAssessment101:Workbook
10
IfmySLOis:Studentswillbeabletoidentifythefourdimensionsofliteracyand
explainhowtheyrelatetoclassroominstruction.
AndmyEvidenceSource#1:afinalpaperinacoursemid-programwhere
studentsdescribeandanalyzethefourdimensionsofliteracy
Criteriamaybe:Allstudentswillearnascoreof3(proficient)orhigheronthe
assignmentrubricrow“KnowledgeoftheDimensions”
Orcriteriamaybe:Allstudentswillearna25/30orhigherasatotalgradeonthe
assignmentrubric
Havingcriterialinkedtoyourevidencetellsyouwhattolookforandfacilitatesthe
analysisofyourdata.
Practice:Basedonevidencesourcesyoulistedaboveconnectedtooneofyour
SLOs,whatcriteriawouldyoulookforinordertofeelsatisfiedwithyourresult?
EvidenceSource#1–Direct
EvidenceSource#2–Indirect
____________________________________________
____________________________________________
____________________________________________
____________________________________________
Criteria:_________________________________
Criteria:__________________________________
____________________________________________
____________________________________________
E.Summary
Beforeyoubeginassessinganacademicprogram,youfirstneedtocomeupwithan
assessmentplan.Componentsoftheassessmentplanincludethefollowing:
StudentLearning
Outcomes
Clearstatementsofexpectedknowledge,skills,attitudes,competencies
thatstudentsareexpectedtoacquire.
SLOStatementsbeginwith“Studentswill…”or“Studentswillbeable
to…”
Evidence
Specificdatasourceintendedtoindicatestudents’progresstowardan
SLO.
It’susefultohavemorethanoneevidencesourceperSLO.
NazarethCollege,AcademicAssessment101:Workbook
11
Remember,indirectevidencealoneisrarelysufficientintellingyou
whetheranSLOhasbeenmet!Ifyouareusingindirectevidence,
chooseadirectsourceaswell.
Criteria
Agoalorperformancestandardwhichwillinformyouonwhetheror
notanSLOhasbeenmetforagivenpieceofevidence.
Asyouformulateyourplan,theguidingquestionsshouldalwaysbe:WhatdoIwant
mystudentstolearninthisprogram?HowwillIknowtheyhavelearnedit?
PartII:ReportingonAssessmentResults
Onceyouhaveanassessmentplaninplaceandhavehadatleastanacademicyear
tocollectyourevidence,thecollegeasksthatyoureportannuallyonyour
assessmentfindingsandactions.Annualassessmentreportsarerequiredfromall
academicprogramsinordertoremainingoodstandingwithouraccreditor,the
MiddleStatesCommissiononHigherEducation.Theultimategoalofassessment
reporting,however,isthatitfacilitatesdata-basedimprovementstoyour
program.
A.Findings
Eachyearyouwillbeaskedtoreportyourfindings.Findingsarebasedonanalysis
ofevidence.Asyoulookatstudentworkthatyoucollectedduringtheyear,what
themesandtrendsareyounoticing?Arestudentsmeetingthecriteriayouset?
Whyorwhynot?
TheexamplebelowisbasedontheSLO:Studentswillbeabletoidentifythefour
dimensionsofliteracyandexplainhowtheyrelatetoclassroominstruction.
EvidenceSource#1:Finalpaperinacoursemid-programwherestudents
describeandanalyzethefourdimensionsofliteracy
Criteria:Allstudentswillearnascoreof3(proficient)orhigheronthe
assignmentrubricrow“KnowledgeoftheDimensions”
Findings:Outof14students,only6metthetargetof3orhigheron
“KnowledgeoftheDimensions”rubricrow.
Thefindingsneednotbelong-windedexplanationsbutsimplyabriefdescriptionof
whatyoufoundwhenyouanalyzedyourdata,asintheexampleabove.
NazarethCollege,AcademicAssessment101:Workbook
12
Werecommendmakingdataanalysisascheduledpartofadepartmentor
programmeetingeachyear.Reportingfindingsisfarmoremeaningfulwhenyou
candiscussthemwithyourcolleaguesandworktogethertounderstandwhat
studentsare(orarenot!)learning.
YouneednotassesseverysingleSLOeachyear.ManyprogramsputtheirSLOs
onacyclesothatassessmentworkremainsmanageable.Considerassessingoneor
twoSLOsfortwoorthreeyearsandthenmovingontodifferentones.Determining
yourassessmentcycleshouldbepartofyourassessmentplan.
Neitherthecollegeornorouraccreditorhavetheexpectationthatyour
assessmentresultsshouldreflectsuccessallofthetime.Themostimportantaspect
ofassessmentismakingimprovementstostrengthenprograms.Inthissense,itis
expectedthatyouwillnotalwaysmeetyourgoals.
B.ActionsandClosingtheLoop/FollowUp
Thisisthewholereasonwedoassessment:totakeinformedactionsthatmakeour
programsstrongerbyhelpingourstudentstolearnbetter.Ifwearenottaking
actionsbasedonourdata,thereisreallynoreasonforcollectingit!
Eachyear,youwillbeaskedtoreportactionsthatyouplantotakebasedonyour
findings.Youractionsshouldbeinresponsetothisquestion:Basedonour
findingsandpotentialproblemsidentified,whatcanwedoasaprogramto
helpourstudentsperformbetter?Or,ifyouareconsistentlymeetingyourtarget,
youmightask:Whatelsemightweassessinthefuture?Whatmightweassess
differentlyinordertomorefullyunderstandourstudents’
knowledge/abilities?
Basedonmyfindings,herearesomeexamplesofactionsImighttake:
Findings:Outof14students,only6metthetargetof3orhigheron
“KnowledgeoftheDimensions”rubricrow.
PossibleAction#1:Itseemsthatstudentsmaybenefitfrommoreclass
timespentreviewingeachdimensionofliteracyaswemovethroughthem.
Beginninginthefall,wewilladdsystematicreviewofeachdimension
throughoutthesemesterbeforemovingontothenextone.
PossibleAction#2:Whilewewillneedtocontinuetomonitorthis,we
suspectthatlimitedwritingproficiencyisasmuchofaproblemasthe
knowledgedeficit.First,wehavesetupameetingwiththedirectorofthe
WritingCentertodiscussstrategiesforsupport.Second,wewillimplement
NazarethCollege,AcademicAssessment101:Workbook
13
anoptionalofficehourswritingconferencewiththeinstructoroneweek
beforethepaperisdueandencouragestudentswhoappeartobestruggling
withthecontentand/orwritingtoattend.
Asyoucanseefromtheexamplesabove,actionsneednotbedifficultortime
consuming.Withyourcolleagues,considerreasonsforyourfindingsandthen,
together,thinkaboutwhattheprogramcandotoincreasestudents’success.
TheassessmentreportingformwillalsoaskyouaboutClosingtheLooporFollow
Uponyouractions.Thisasksyoutothinknotaboutwhatyouaregoingtodobased
onfindings,butwhatyouhavedoneinthepastyearbasedonpreviousyears’
findings.
Asyourespondtothispromptconsiderthefollowingquestion:Whatworkhave
wedoneinthepastyeartoimproveourprogrambasedonpreviousyears’
assessment?Again,thesedonotneedtobemajor,time-consumingorexpensive
actions.Thinkaboutthe“smallthings”andtalktoyourcolleaguestomakesure
you’renotmissinganything!
● Haveyouchangedanyassignmentsorgradingcriterialinkedtoanyofyour
SLOs?
● Haveyoubeguntoofferreviewclassesoradditionalone-on-onemeetings
withstrugglingstudents?
● Haveyoumodifiedlectures,readings,instructionaltechniquesorother
resourcestohelpyoutodeliverstrongerinstruction?
● Haveyouorotherfacultyinyourprogramattendedworkshopsorseminars
relatedtoimprovingqualityofteaching?
● Haveyoubeguntoimplementmid-semesterfeedbackformstobetter
understandstudents’experiencesinclasspriortotheendofthesemester?
IflinkedtoyourSLOs,theseareallpartofanimportantstorythatshouldbeshared
inyourassessmentworkeachyear!
C.RoleoftheAcademicAssessmentCommittee
EachyearonOctober1,youareaskedtosubmitanacademicassessmentreportto
thecollege.ThesereportsarecompiledbytheCoordinatorofSLOAssessmentand
sharedwiththeelectedAcademicAssessmentCommittee.Thecommitteereviewsa
sampleofassessmentreportseachyearandprovidesfeedbacktoprogramfaculty
basedontherubricbelow:
NazarethCollege,AcademicAssessment101:Workbook
14
Inadditiontoauditingourinstitutionallearningoutcomesassessmentsystem,the
goalofthecommitteeistoprovideprogramfacultywithfeedbackontheir
assessmentworkthatmayhelptostrengthentheirprocesses.Thefeedback
youreceiveisnotintendedtobeevaluativeorpunitive.
D.RecommendedAssessmentResources
MiddleStatesCommissiononHigherEducation–Publications:
https://www.msche.org/publications_view.asp?idPublicationType=5
NazarethCollegeAcademicAssessmentWebsite:
https://www2.naz.edu/academic-affairs/academic-assessment/
NazarethCollege,AcademicAssessment101:Workbook
15
NationalInstituteforLearningOutcomesAssessment:
http://www.learningoutcomesassessment.org/
Walvoord,B.E.(2010).Assessmentclearandsimple:Apracticalguidefor
institutions,departments,andgeneraleducation(2nded.).SanFrancisco,CA:JosseyBass.
PleasealsomakeuseoftheTeachingIntegrationandInnovationLab,which
supportsmeaningfuluseofassessmentdatathroughopportunitiesforteaching
development!
NazarethCollege,AcademicAssessment101:Workbook