Program Quality Assurance-Clements

10/17/14 Program Quality Assurance !
Accreditation Conference!
October | 2014!
!
!
!
!
!
!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
1 10/17/14 TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
what students are going to learn, measure what they learn, and provide the feedback loop to improve a student’s comprehension ar*culate exactly provide ways to and understanding. TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
2 10/17/14 Public displeasure!
!
Increasing federal scrutiny over accreditors!
!
Federal funding to institutions!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
3 10/17/14 Iden*fy what/
who you want your graduates to be TRANSFORMING
Iden*fy the measurable components THE
PROFESSION
Apply Measurement Tools THROUGH
QUALITY
E D U C A T I O N!
Components of an assessment plan!
Mission Statements!
Expected outcomes/goals/aspirations!
Objectives!
Evaluation Tools & Benchmarks!
Assessment of Results!
Use of Results!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
4 10/17/14 Outcome Objectives One row for
each outcome Evaluation
Tools and
List all Objectives of
the given outcome in Benchmarks
same row
Note: The same (1.1,1.2,1.3). instrument/tool (e.g., an external exam, a capstone project, or other cumula*ve experience) may be used to assess mul*ple objec*ves. Assessment Use of
Results
Results
Summarize the results found for each objec*ve, related to the benchmark. Summarize the planned program changes intended to improve student learning. TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Broad statement of purpose and aspiration!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
5 10/17/14 TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Broad discipline-related characteristics of a
graduate of the program.!
!
Encompass key
aspects of learning desired!
!
Align with institutional outcomes, knowledge, skills,
and abilities designated by CAATE, and anything else as
determined by program stakeholders!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Outcome?!
!Entry-level professional
knowledge, comprehension, and
proficient skills of an orthopedic
clinical examination and
diagnosis of the knee!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
6 10/17/14 !Students will implement a clinical practice
approach that emphasizes interpreting
research and answering clinical questions
to guide patient care.!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
7 10/17/14 Measurable, demonstrable components of an
outcome.!
!
Foundation for assessments at the course
and program level.!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
QUALITY
E D U C A T I O N!
Writing Objectives!
•  Bloom’s Taxonomy!
•  SMART format!
–  Specific!
–  Measurable!
–  Attainable!
–  Realistic!
–  Time-Oriented!
TRANSFORMING
THE
PROFESSION
THROUGH
Objectives?!
Outcome: Entry-level professional knowledge, comprehension, and
proficient skills of an orthopedic clinical examination and diagnosis
of the knee!
Objectives:!
!Taking a patient's history; inspection of knee along with
observation of gait; palpation of bony and soft-tissues; joint motion
assessment using a goniometer; muscle function assessment
using manual muscle tests; circulatory assessment; neurological
function assessment including sensory, motor, and reflex testing;
selective tissue testing utilizing appropriate special tests; clinical
diagnosis of common injury, condition, illness, and/or disease
based on clinical examination; professional behaviors exhibited
during the clinical examination.!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
8 10/17/14 Outcome: Students will implement a clinical practice
approach that emphasizes interpreting research and
answering clinical questions to guide patient care.!
!
Objectives:!
1.!Successfully access and accurately interpret a variety of
healthcare related research data.!
2.!Demonstrate contemporary skill in comprehensive care of
patients with injuries.!
3.!Demonstrate contemporary skill in comprehensive care of
patients with illnesses.!
4.!Develop patient-centered treatment plans that consider
the impact of the condition on the patient’s life (eg,
socioeconomic, religion).!
5.!Incorporate patient-centered outcome measures to
evaluate the quality of care provided.!
6.!Develop a comprehensive plan for continuing education.!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Direct Assessments Indirect Assessments •  Samples of student work •  Indicators other than student work •  Feedback from students or others •  Alumni surveys; exit interviews; focus groups •  Specific exam ques*ons; papers; presenta*ons; porXolios; licensure exams TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Tools/strategies may be used to cover multiple
objectives!
!
Class grades don’t tell the story!
Specific, targeted assignments !
Projects / Presentations!
Practicum evaluations!
Rubrics used to gauge learning of cohorts!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
9 10/17/14 Outcome: Students will implement a clinical practice approach
that emphasizes interpreting research and answering clinical
questions to guide patient care.!
!
Objectives:!
1.!Successfully access and accurately interpret a variety of
healthcare related research data.!
2.!Demonstrate contemporary skill in comprehensive care of
patients with injuries.!
3.!Demonstrate contemporary skill in comprehensive care of
patients with illnesses.!
4.!Develop patient-centered treatment plans that consider the
impact of the condition on the patient’s life (eg, socioeconomic,
religion).!
5.!Incorporate patient-centered outcome measures to evaluate the
quality of care provided.!
6.!Develop a comprehensive plan for continuing education.!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
EBP Project Rubric TRANSFORMING
THE
Professional Behaviors Assessment © Excellent Consistently demonstrates traits in this column; requires no assistance with any traits Adequate Usually demonstrates traits in this column; requires minimal assistance with one or more traits Requires Improvement Occasionally demonstrates traits in this column; requires moderate assistance to effect change with one or more traits Requires Change Consistently demonstrates traits in this column; requires detailed assistance to effect change with most traits Integra>on of Evidence TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
10 10/17/14 TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Percent of a cohort demonstrating a predetermined
level of proficiency!
!
Degree to which you desire improvement from a
previous year!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
11 10/17/14 80% of students will score above a 70 on their
senior case study according to the rubric.!
!
“We met our benchmark of 80% - no further
action needed at this time”!
!
Challenges?!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Outcome Excep>onal Very Good Adequate Inadequate Evidenced-­‐based Clinical Prac*ce 15% 70% 15% 0% TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Boston University Professional Behaviors Assessment © Excellent Consistently demonstrates traits in this column; requires no assistance with any traits Adequate Usually demonstrates traits in this column; requires minimal assistance with one or more traits Requires Improvement Occasionally demonstrates traits in this column; requires moderate assistance to effect change with one or more traits Requires Change Consistently demonstrates traits in this column; requires detailed assistance to effect change with most traits Integra>on of Evidence TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
12 10/17/14 TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Outcome Excep>onal Very Good Adequate Inadequate Evidenced-­‐based Clinical Prac*ce 6% 40% 50% 4% TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
13 10/17/14 Curriculum changes!
Faculty development!
Technology!
!
Library Services!
Academic advisement and counseling!
Tutoring !
Co-curricular opportunities!
Budgeting!
Academic review!
Improve relationships / learning at clinical sites!
!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
14 10/17/14 TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
What assessment strategies did you use
to measure each objective?!
q Alumni Survey!
q Case Study!
q Class Presentation!
q Employer Survey!
q Exam!
q Exit Interview!
q IPE!
q Objective Structured Clinical Exam (OSCE)!
q Practical Examination!
q Preceptor Evaluation!
q BOC Exam!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
15 10/17/14 Annual Report Section V: Outcomes!
34.  Provide one outcome for your program:!
35.  Provide the objectives that you measured for this
outcome: (2-7)!
36.  What assessment strategies did you use to
measure each objective: !
37.  Provide the benchmark(s) you set to determine if
your students are competent based on your
assessment strategy: !
38.  Which benchmarks were met/not met: !
39.  What changes did you make in the program as a
result of the outcomes measurement: !
TRANSFORMING
THE
PROFESSION
Section V: Outcomes – not
THROUGH
QUALITY
E D U C A T I O N!
suggested!
34.  Outcome: Pass the BOC …!
35.  Objectives: Learn all competencies and proficiencies!
36.  Assessment strategies: Class Grades!
37.  Provide the benchmark(s): Each Student will … !
38.  Which benchmarks were met/not met: All. !
39.  What changes did you make in the program as a
result of the outcomes measurement: None!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
16 10/17/14 TRANSFORMING
THE
Outcome: Objec>ves: 1. 2. 3. 4. 5. PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
Assessment Descrip>on (tools, benchmarks) TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
What tools do you use in your program to measure
whether objectives are being met? !
!
Papers, Projects, Evaluations, Surveys!
!
How do you measure? Rubric……!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
17 10/17/14 Outcome Objectives One row for
each outcome Evaluation
Tools and
List all Objectives of
the given outcome in Benchmarks
same row
Note: The same (1.1,1.2,1.3). instrument/tool (e.g., an external exam, a capstone project, or other cumula*ve experience) may be used to assess mul*ple objec*ves. Assessment Use of
Results
Results
Summarize the results found for each objec*ve, related to the benchmark. Summarize the planned program changes intended to improve student learning. TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
TRANSFORMING
THE
PROFESSION
THROUGH
QUALITY
E D U C A T I O N!
QUALITY
E D U C A T I O N!
Chad Clements!
[email protected]!
!
TRANSFORMING
THE
PROFESSION
THROUGH
18