Academic and
Administrative
Assessment Information
Session
University of South Florida
Office of Institutional Effectiveness and Review
Welcome and thank you!
The Office of Institutional Effectiveness, Academic Planning
and Review
Programs
Conduct
Internal
Assessments
Plans/Reports
submitted
SAM Database
Reviewed,
Analyzed, and
Compiled by
OIE
Melanie Wicinski, PhD
Assistant Director
Institutional Effectiveness & Assessment
Required
reports
submitted to
BOG,
SACSCOC
Agenda
 Academic assessment reports/SAM academic report

Components and requirements of an
assessment plan

Administrative assessment reports

System for Assessment Management (SAM)
Academic Assessment Plans/Reports

Annual report of each degree and certificate program.

Program learning goals and learning outcomes

Plans for future program development

Formative assessment of the degree or certificate program, not
summative assessment of students.

Non-comprehensive

Non-linear

Program-focused

Looking for opportunities for growth and development

Continuous
Components of an Assessment Plan in SAM
1. Program/Department Mission Statement
2. Program Learning Goals (LGs)
3. Learning Outcomes (1 or more per LG)
i. Statement of Learning Objective
ii. Method of Assessment
iii. Performance Targets
iv. Results
v. Use of Assessment Results
Components of an
Assessment Plan in SAM
Breakdown
Mission
Learning Goals
Learning
Outcome
Statement
Method of
Assessment
Learning
Outcome
Performance
Targets
Learning
Outcome
Assessment
Results
Learning
Outcome
Use of
Assessment
Results
Learning Goals v. Learning Outcomes
Learning Goals
Learning Outcomes

Not always measureable, broad
statements

A demonstrable skill, measureable
change in behavior

Skills, Content, knowledge, or tasks
students will have

Specific Statements

Example: “Development of Critical
Thinking Skills.”

What students will know and
demonstrate after the course(s)

Undergraduate Requirements

Example: “Majors in this program
will be able to conduct original
research in this discipline using
appropriate methods.”

Includes 5 detailed parts

Content Specific Knowledge

Critical Thinking Skills

Communication Skills
*Graduate degrees and Certificates:
no minimum reporting requirement
Method Of Assessment Section

Should be the most detailed section under the Learning Outcome

Less flexible, has several requirements of what should be included

Examples of Assessment Methods
 Written
Student Work
 Student Presentation/Performance
 Portfolio of Student Work
 Part
of a Lab Report
 Embedded test questions
 Part of a thesis or dissertation
 Standardized
tests*
 Internship/Practicum evaluation*
Method of Assessment: Requirements
1. A clear description of the assessment method
2. A statement on how the assessment specifically measures
the task, information or competency stated in the Learning
Outcome
3. Context of the Assessment
4. How the assessment will be scored
5. Which students in the program will be assessed
Method of Assessment: Requirements
Cont.
6. If a sample of student work will be analyzed in lieu of all
students, include information on the sample
7. If employing a rubric, information is needed on the rubric used
and how it was developed and validated

E.g. How Inter-rater reliability will be addressed
8. Information on how the rubric will be calibrated
9. Information on who will be reviewing and rating the
assessment.
Method of Assessment:
Inappropriate Assessments

Grades CANNOT be used as outcomes (summative assessment of
student work, not formative assessment of program work)

Summative Student Assessments (the passing or failing of a
culminating exam, course or program) CANNOT be used (summative
vs. formative)

Student/faculty/Alumni Surveys CANNOT be used (indirect measures)

Publication in an academic journal CANNOT be used as a student
learning outcome (external factors)

Job placement CANNOT be used (external factors)
Performance Targets:
Target for measured program performance

SHORT statement on what benchmark(s) the program wants to meet


This is not how we think our students will perform, but what we want our program
to do
Should be worded in terms of performance on the rubric or rating instrument
 “Performance
target will be considered met if 75% of students
achieve an overall score of 4/5 or higher”
 “Performance
target will be considered met if 80% of students
assessed will receive a final score of “Commendable” or higher.
Assessment Results

SHORT statement on the final results of the assessment

Stated in terms of the rubric or grading instrument

Does not need to include commentary about the results
and why or why not the performance targets were met

Does not need to be, and should not be, perfect

Don’t be afraid to report that the target was not met
Use of Assessment Results: Do
Should include specified, actionable “next
steps” the program will take, using the
assessment results, to develop or improve on an
programmatic level:
Curriculum
Revisit
Mapping/redevelopment
or revise the assessment method/rubric
Revisions
to plan of study/program structure
Development
Faculty
of new modules/courses
development
Use of Assessment Results: Don’t
 Remember:
The responsibility is on the
program, not its students. Use of results
section should indicate programmatic
improvements
 This
is not an assessment report of student achievement
Do
not include: tutoring effort, sending
students to the writing center, counseling
students, providing students feedback, or
improved recruitment
Certificates
The following three possibilities apply to certificates:



If ALL of the students in the certificate program are non-degree
seeking, non-financial aid students, then you can state that in SAM
and do not have to complete a program assessment.
If even one student who is degree-seeking and receives financial aid is
enrolled in the certificate program then you have to provide an
assessment (I know some certificates can be taken as electives in a
degree-seeking program and this is where this may come into play).
If your certificate is a sub-set of a full degree program (the classes in
the certificate are really just a portion of a full degree program), then
your certificate assessment can be identical or a subset of your
assessment reported for the degree program.
Rubric for Academic Program Assessment
Cycle
Element
Mission
Planning
May be the departmental mission
Quality Level
Missing Data
Mission statement is missing
Undergraduate
Any one portion of the three
Three must be covered
required Goals are not completed
(Content Specific
Knowledge,
(Content Specific Knowledge,
Program Goals
Communication, Critical Communication, Critical Thinking).
Should match the Mission
Thinking)
Statement and each should
Graduate and
be one complete statement
Certificate
No required
No Goal present
amount/content
program goals
Emerging
Developed
Highly Developed
A statement is present, but is not a mission, but a
description of the program
Mission statement is present and appropriate
A statement is present, but not a Goal
Goals are succinctly and clearly stated
Not a learning Goal
Learning Goals are listed, but
not aligned with mission
statement
Learning Goals are listed and clearly aligned with the
mission statement
The Learning Outcome is
The statement is present,
The Learning Outcome is
present, but may not be an The Learning Outcome may
but not stated as a learning
present and well-developed
appropriate student learning
need minor revisions
Should be actionable and realistic
outcome
and clearly stated
outcome
Method is appropriate, but
Method is inappropriate; not
Method of Assessment
may need further
Method is clear, concise and
No stated method of assessment or connected or specific to the
Method may need minor
Should include type of assessments, how the
description, including more
well-defined. Meets all
not revised from previous year
learning outcome; use of
clarification
program is being measured, and should provide
information about rubric,
reporting requirements
IDM, GO, external factors
information specific to the stated measure
raters, or context
Performance targets
Performance Targets
No performance target present or
Perfomance targets are
present, but stated in terms
Should be thought of as PROGRAM benchmarks.
targets do not correspond with
present, but may need
Performance targets are appropriate and well-stated
of assignment, course, or
Indicate what numbers would indicate that the
methods
minor clarification
degree completion
program has met their programmatic goal
Learning Outcome Statement
No Learning Outcome present or
not revised from previous year
Assessment Results
Reporting
Initial
Include specific findings, including number of
students included in the assessment, and any
other appropriate statistical information
No Assessment Results present
Use of Assessment Results
Should include actionable items for PROGRAM
improvement
No Use of Results present
Assessment results present
Assessment results present,
Assessment results present
Assessment results present, and clear, but missing minor
but unclear how they relate
and clear. All data reported
but not specific
data (such as: number of
to methods
appropriately
students)
Use of Results statement
indicates no changes are
necessary
Use of Results statement
Use of Results statement is
Use of Results statement is
does not indicate what
present and clear and
present and clear; may need
changes will be made based
specifically indicates the
minor clarification
off data
changes that are being made
The Phases of the Academic Reporting Cycle
(Note: For all colleges, except Education and Business)
Important Notes:
• Changes to your plan can be made at
any time.
• Reviews by OIE are intended to be
helpful – this is a conversation.
Adapted from Rodriguez & Frederick, Miami Dade College 2013
Academic Reporting Cycle Dates
Note: For all colleges except Education and Business
Academic Assessment Reporting
Plan Year
Plan Due Dates
Plan Approval Date
Final Report Date
2016
March 31, 2016
April 15, 2016
December 15, 2016
2017
January 15, 2017
January 30, 2017
December 15, 2017
2018
January 15, 2018
January 30, 2018
December 15, 2018
The Phases of the Academic Reporting Cycle
(Note: For Education and Business ONLY)
Important Notes:
• Changes to your plan can be made at
any time.
• Reviews by OIE are intended to be
helpful – this is a conversation.
Adapted from Rodriguez & Frederick, Miami Dade College 2013
Academic Reporting Cycle Dates
Note: For Education and Business ONLY
Academic Assessment Reporting
Plan Year
Plan Due Dates
Plan Approval Date
Final Report Date
2015-2016
N/A
N/A
October 1, 2016
2016-2017
October 15, 2016
October 31, 2016
October 1,2017
2017-2018
October 15, 2017
October 31, 2017
October 1,2018
Administrative Assessment Plans

Very similar to Academic plans

They are different in the following ways:

Learning Outcomes vs. Objectives

Same type of cycle, but different due dates

Not nearly as strenuous and detailed – RIGHT NOW!

May include graduation rates, alumni surveys, employer surveys, etc.

Ask: What are the goals for your support area? What can you improve? What goals might
you have been given from leadership?
Example of Administrative Assessment
Objective

Objective: Encourage funded and unfunded research involving communitybased partnerships.

Method: Track grant applications captured by Office [name] by using
[software].

Performance Target: Increase community-based grant applications by 5%.

Results: There were ### community-based grant proposals submitted this
past calendar year, representing a #.#% increase in proposals. There were ###
community-based grant awards in that year, representing an #.#% in awards,
to a total of $#. Already in 2016, there is a total of $# in awards.

Use of Results: These data are also self-identified on the … Form of the
Office [name]. A task force is working on easier methods of gathering and
recording community-based grants and funding, as well as community-based
research that is done without external funding.
The Phases of the Administrative Reporting Cycle
Important Notes:
• Changes to your plan can be made at
any time.
• Reviews by OIE are intended to be
helpful – this is a conversation.
Adapted from Rodriguez & Frederick, Miami Dade College 2013
Administrative Reporting Cycle Dates
Administrative Assessment Reporting
Plan Year
Plan Due Dates
Plan Approval Date
Final Report Date
2015-2016
N/A
N/A
August 31, 2016
2016-2017
September 30, 2016
October 15, 2016
August 31, 2017
2017-2018
September 30, 2017
October 15, 2017
August 31, 2018
How does USF organize academic and
administrative program assessment plans?
 “SAM”
The System for Assessment
Management

Database maintained by the Office of
Institutional Effectiveness

Available online with NetID/USF password
Accessing the System for
Assessment Management
The pink (or orange) paper provided in your training
materials has detailed instructions for logging in.
Recommendation: Once you get to the login page,
bookmark it!
How do you know what your plan
was rated?
Overall Plan Rating Scale
Missing Data
Initial
Emerging
Developed
Highly Developed
• If a plan receives a yellow or red overall rate, we will
select that is “not complete”
• Programs will be notified and asked to make changes
to sections that still need clarification or modification
• When secondary adjustments have been made, the
“Plan complete?” button should be changed to “Yes”
Take Away Points

Formative of program, not summative of students

Not comprehensive assessment of all program efforts, make it manageable

Use samples when possible

Not linear, if one area has met the performance target and is going well, move
on to another area

Develop a tentative five year assessment progression

Brainstorm possible area of improvement to tackle over the next 5 years

University-wide 5 year strategic plans will be coming to the college and unit
level.
Questions?
PLEASE CONTACT OUR OFFICE ALONG THE WAY!
We are here to help!