1
Dr. Melissa S. Medina
Presidential Professor and Assistant Dean for Assessment and Evaluation, & Director of Preparing Future Faculty
The University of Oklahoma Health Sciences Center
AACP Assessment SIG Webinar
December 17, 2014
Writing Measurable Objectives
Session Objectives
1. Discriminate between goals and objectives
2. Contrast lower and higher cognitive level of Bloom’s taxonomy
3. State 3 common mistakes made in objective writing
4. Identify at least 1 observable & measurable verb for each level of learning to use when writing objectives
5. Create or reconstruct your own objectives in accordance with the guidelines discussed during the workshop
OBJECTIVES – FORMAT
1.
2.
3.
4.
Objectives listed in a numbered format
1
Objectives listed in a paragraph format
0.5
Objectives are written as a topic list only
0
All objective sentence starts with the verb
1
Some objective sentences starts with verb
0.5
All objective sentences starts with “the student is able to” or “to”
0
All verbs used in each objective are measurable
2
Some objectives not measurable (e.g.know/understand)
1
No measurable objectives (all verbs are know, understand)
0
No objective verbs overlap
1
Some of the objective verbs overlap
5.
Possible
Points
0.5
Same verb used for all objectives
0
All objectives written using quantified language (using numbers or amounts)
2
Some of the objectives use quantified language
1
None of the objectives do not use quantified language
0
Points Earned/Comments
Pts Earned _____ / 1
Pts Earned _____ / 1
Pts Earned _____ / 2
Pts Earned _____ / 1
Pts Earned _____ / 2
OBJECTIVES - ALIGNMENT
1.
All objectives written at a Bloom’s level that matches level of student or audience
Some objectives written at a too high or too low level for students
2.
1
0.5
All objectives written at a too high or too low level for students
0
All objectives written at level that aligns with level content delivered
2
Some objectives written match content delivery
1
No objectives match the level the content was delivered
0
Pts Earned _____ / 1
Pts Earned _____ / 2
Total score = ________/10
2
I.
Writing educational objectives
A.
Terminology
1.
Goals
a.
b.
c.
Objectives
a.
What do you expect them to learn specifically?
b.
Cover specific expectations
c.
Write as specific, measurable outcomes in terms of learner
2.
B.
Initial Considerations
1.
Who is your learner?
a.
Expertise categories
1.
Naïve
2.
Novice
3.
Expert
b.
Grade level
1.
Freshman-Senior
2.
Didactic curriculum vs. experiential curriculum
c.
Program type
1.
Undergraduate vs. Graduate
2.
Professional program
3.
Mixed types of students (interdisciplinary program)
2.
C.
What should they learn overall?
Cover broad expectations
Difficult to measure
What is your teaching setting?
a.
Traditional classroom
b.
Clinical
c.
Other experiential setting
d.
Laboratory
Structuring the Objectives
1.
Bloom’s Taxonomy Classification system
a.
Offers framework for writing objectives and planning assessments
b.
Emphasizes alignment of teaching, learning and assessment
c.
Tool for evaluating if wide range of learning covered & areas of neglect
Table 1. Three Domains of Bloom’s Taxonomy of Educational Objectives
Cognitive Domain
Cognitive Domain
Psychomotor Domain
Affective Domain
(historical)
(revised)
Knowledge
Lower Level
1. Knowledge
2. Comprehension
3. Application
Higher Level
4. Analysis
5. Synthesis
6. Evaluation
Bloom, 1956
Emphasizes knowledge
building
1.
2.
3.
Skills
Lower Level
Remember
Understand
Apply
Higher Level
4. Analyze
5. Evaluate
6. Create
Anderson, et al., 2001
Emphasizes problem
solving & critical
thinking
1.
2.
3.
4.
Observing
Imitating
Practicing
Adapting
Bloom, 1956
Attitudes
1.
2.
3.
4.
5.
Receiving
Responding
Valuing
Organization
Characterization by a
value
Bloom, 1956
3
Table 2. Categories of Bloom’s Taxonomy Aligned with Measurable Verbs, Teaching Activities and Assessment Options
Cognitive
(intellectual)
Higher ----------------------------------------------- Lower
Domain
Level
Knowledge
Comprehension
Higher --------------------------- Lower
Psychomotor
(skills, abilities)
Higher --------- Lower
Affective
Understanding
Application
Applying
Analysis
Detecting
Synthesis
Constructing
Evaluation
Judging
Observing
(interest, attitudes, appreciation)
Key word focus
Recall, remember,
recognize
Selected verbs
Define, Identify, List, Recall,
Recognize, Select, State
Defend, Describe, Distinguish,
Discuss, Explain, Estimate, Identify,
Interpret, Select, Summarize
Apply, Calculate, Construct,
Demonstrate, Interpret, Manage,
Modify, Solve, Use, Write
Analyze, Categorize, Calculate,
Compare, Contrast, Differentiate,
Distinguish
Compose, Construct, Create,
Develop, Formulate, Integrate,
Modify, Organize, Revise, Plan
Assess, Compare, Contrast, Defend,
Evaluate, Infer, Judge, Justify,
Recommend, Support
Watching
Attend to, observe, watch
Imitating
Mirroring
Copy, imitate, model, operate
Practicing
Basic to complex
skills
Adapting
Adapting
Receiving
Awareness
Responding
Participating
Valuing
Valuing
Organizing
Prioritizing
Internalizing
values
Consistency
Adjust, Assemble, Calibrate,
Construct, Complete, Compound,
Counsel, Implement, Manipulate,
Measure, Mix, Operate, Select,
Weigh
Describe, Identify, Locate, Select
Assist, Cooperate, Contribute,
Discuss, Perform, Respond, Select
Appreciate, Believe, Demonstrate,
Describe, Differentiate, Explain,
Justify, Persuade, Propose, Seek
Compare, Contrast, Examine,
Formulate, Identify, Integrate,
Recognize
Conclude,Demonstrate, Maintain,
Qualify, Resolve, Revise, Verify
Example learning objective
State 3 indications for initiating a
specific medication to treat nocturia.
Example learning activities
Lecture, visuals, examples,
illustrations, analogies
Explain the basic pharmacology of
atenolol.
Lecture, questioning, discussion,
writing, presenting
Use information systems to prevent
medication errors.
Distinguish between minor and major
drug interactions.
Create a process to facilitate internal
communication of medication
shortages.
Using JNC-7 guidelines, recommend
a therapeutic alternative for patients
with drug-disease interactions.
Watch a training video to see the 6
steps of preparing an IV infusion.
Assemble completed prescription
orders according to procedures for
verification.
Measure nutrients and additives to
complete a parenteral nutrition order.
Revise compounding methods
according to source materials or
complex orders.
Identify the roles of other healthcare
providers in the environment.
Respond to target medication reports
on a daily basis.
Justify the use of various
communication methods for
clarification of medication orders.
Formulate a plan to proactively
manage competing priorities when
providing services.
Demonstrate respect for patients,
staff and providers.
Demonstration, role play, simulation,
projects, presenting, problem
sets/scenarios, writing
Case studies, discussion, journal
club, problem sets/scenarios,
projects, writing
Case studies, discussion, journal
club, problem sets/scenarios,
projects, simulations, writing
Case studies, discussion, journal
club, problem sets/scenarios,
projects, simulations, writing
Practice exercises, problem sets,
simulations
Practice exercises, problem sets,
simulations
Practice exercises, problem sets,
simulations
Evaluation method(s)
Observation
(assessment)
Verbal quizzes or exams
Written quizzes or exams
Written work
Rubrics
Observation
(assessment)
Rubrics
Lecture, practice scenarios,
discussion, writing
Practice scenarios, discussion
Practice scenarios, discussion,
writing
Practice scenarios, discussion,
writing
Practice scenarios, discussion,
writing
Dennis, V. (September 2010). OUHSC College of Pharmacy Preceptor conference.
American Council on Pharmaceutical Education Continuing Education Providers. Standards 4, 7, 9. Accessed July 22, 2010. Available at http://www.acpe-accredit.org/ceproviders/resources.asp#6.
English, DE, et al. Goals of Experiential Teaching. In: Cuellar LM, Ginsburg DB, editors. Preceptor’s Handbook for Pharmacists, 2nd edition. Bethesda: American Society of Health-System Pharmacists; 2009. p. 77-92.
MacKeracher D, Gross P. Writing learning objectives. Centre for Enhanced Teaching and Learning, UNB Fredricton. Accessed August 4, 2010. Available at http://www.unb.ca/fredericton/cetl/_resources/pdf/writinglearningobjectives
Observation
(assessment)
Written work
Self-assessment
Rubrics
4
D.
Writing Effective Objectives (3 most common mistakes)
1. Format
a. #1 common mistake is to use “know or understand”
1.
Number objectives vs. writing in paragraph form
2.
Omit starting the sentence with “the student is able to” or “to”
3.
Avoid offering just a list of topics
4.
Write objective in terms of learner, not the teacher
5.
Start the sentence with an action verb that specifies observable, measurable
performance (avoid see, know, understand)
6.
Lower Level Verbs
a. identify, describe, define, choose, explain, state, translate, list, apply,
b. select, use, estimate, calculate
Higher Level Verbs
a. contrast, distinguish, infer, analyze, utilize, relate, formulate, prepare, develop
b. generate, judge, evaluate
7.
2.
Content
a. #2 common mistake is to use vague or unquantified language
1. Use specific, quantified language
2. Avoid word magic or jargon that sounds good but is difficult to measure – “Critically
think about ……..”
3. Alignment
a. #3 common mistake is objectives do not match what or how emphasized in talk
1. Should match major concepts discussed
2. Write ~ 3-5 objectives for 50 minute lecture
3. Write at level that is achievable by audience – “if discuss”, is audience discussing
4. Should align with how information is assessed
References:
• Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., &
Wittrock, M.C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of
educational objectives. NY, NY: Longman.
• Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational
objectives: Handbook I: Cognitive domain. New York: David McKay.
• Davis, B.G. (1993). Tools for Teaching. San Francisco, CA: Jossey-Bass.
th
• Gronlund, N.E. (2004). How to write and use instructional objectives: 7 edition. Upper Saddle River, NJ:
Prentice Hall.
• Munro, A., & Rigney, J.W. (1977). A schema theory account of some cognitive processes in complex learning
(Technical Report No. 81). Los Angeles, CA: University of Southern California Behavioral Technology Labs.
• Woolfolk, A. (2001). Educational Psychology 8th edition. Boston, MA: Allyn & Bacon.