Disclaimer Statement
The presenter(s) attest that they do not have a conflict of
interest, bias, or commercial support related to the
educational activity. The presenter nor any member of their
immediate family, have any financial relationship in which
they receive salary, royalty, fees, honoraria, ownership
interest or other financial benefits from an organization
addressed in the educational materials. They do not promote
any services or products during the activity and neither the
presenter, nor any member of their immediate family, have
any financial interest in the topics presented during the
educational activity.
Attendees must submit a completed evaluation of the
educational activity to receive their CNE certificate.
1
1
Test Blueprinting for Standard and Concept-Based
Curricula
ATI Education Summit 2017
Lois Churchill, MN, RN
Dianna Johnston, DNP, RN, NE-BC
2
Objectives
 Explain the benefits of using test blueprinting.
 List the steps in test blueprinting development.
 Explain how to set the specifications for the elements at the course and program levels.
 Explain how to plan for remediation activities after conducting an item analysis.
3
Purpose of Test Blueprints
 Tool used to determine parameters of a test prior to its development.
 Program blueprints outline by semester the percent of each element on tests delivered during
the semester.
 Course blueprints outline the percent of each element on course tests.
4
Steps in Test Blueprint Development
 Identify learning elements to be measured
― Determine percent of items for each difficulty level (Bloom/Anderson & Krathwohl)
― NCLEX® Content Area
― Nursing Process
― QSEN
― BSN Essentials
― Massachusetts Nurse of the Future
 Determine the weight to be assigned to selected elements
― Program
― Course
 Conduct item analysis
― 55
5
Percent of Items for Each Cognitive Level
 Bloom’s Taxonomy - Knowledge, Comprehension,
Application, Analysis, Synthesis, Evaluation
(Bloom, 1956)
 Anderson & Krathwohl’s Taxonomy - Remember, Understand, Apply,
Analyze, Evaluate, Create
(Anderson & Krathwohl, 2001)
 Foundational Level
― Remember
― Understanding
 Critical Thinking / Clinical Judgment
― Apply
― Analyze
6
RN NCLEX® Test Plan 2016
Updated every 3 years Based on practice analysis (First 12 months of practice)
Client Needs
Percentage of Items from Each
Category/Subcategory
Safe and Effective Care Environment
• Management of Care
17-23%
• Safety and Infection Control
9-15%
Health Promotion and Maintenance
6-12%
Psychosocial Integrity
6-12%
Physiological Integrity
• Basic Care and Comfort
6-12%
• Pharmacological Therapies
12-18%
• Reduction of Risk Potential
9-15%
• Physiological Adaptation
11-17%
7
PN NCLEX® Test Plan 2017
Updated every 3 years Based on practice analysis (First 12 months of practice)
Client Needs
Percentage of Items from Each
Category/Subcategory
Safe and Effective Care Environment
• Coordinated Care
18-24%
• Safety and Infection Control
10-16%
Health Promotion and Maintenance
6-12%
Psychosocial Integrity
9-15%
Physiological Integrity
• Basic Care and Comfort
7-13%
• Pharmacological Therapies
10-16%
• Reduction of Risk Potential
9-15%
• Physiological Adaptation
7-13%
8
Obtaining Information about NCLEX ® Test Plans
The detailed test plan for item writers/item reviewers and nurse educators can be accessed at the
web sites below:
https://www.ncsbn.org/RN_Test_Plan_2016_Final.pdf
https://www.ncsbn.org/10136.htm- PN 2017 Detailed Test Plan
9
Level of Difficulty
 Bloom’s Taxonomy
―
―
―
―
Knowledge
Comprehension
Application
Analysis
 Anderson & Krathwohl’s Taxonomy for Learning
―
―
―
―
―
―
Remember
Understand
Apply
Analyze
Evaluate
Create
 Foundational Thinking vs. Critical Thinking
10
Nursing Process
11
QSEN Competencies
 Quality and Safety Education for Nurses Competencies
 Program with goal of preparing future nurses KSA necessary to improve quality of
care- 6 competencies
 Massachusetts Nurse of the Future integrates the QSEN competencies and IOM
recommendations
12
AACN Essentials of Baccalaureate Education for
Professional Nursing Practice
13
AACN BSN Essentials (continued)
14
Massachusetts Nurse of the Future Core Competencies©
Competency
Definition
Patient-Centered Care
The Nurse of the Future will provide individualized care that recognizes patient’s preferences, values and needs and
respects the patient or designees as a full partner in providing compassionate, coordinated, appropriate, safe care.
Professionalism
The Nurse of the Future will demonstrate accountability for practicing nursing within established moral, legal, ethical,
regulatory and humanistic principles.
Leadership
The Nurse of the Future will influence the behavior of individuals within their environment in a way that will facilitate the
establishment and acquisition/achievement of shared goals.
Systems-Based
Practice
The Nurse of the Future will demonstrate an awareness of and responsiveness to the larger context and system of health
care and the ability to effectively call on system resources to provide care that is of optimal quality and value.
Informatics &
Technology
The Nurse of the Future will use information and technology to communicate, manage knowledge, mitigate error and support
decision-making.
Communication
The Nurse of the Future will interact effectively with patients, families, and colleagues, fostering mutual respect and shared
decision-making, to enhance patient satisfaction and health outcomes.
Teamwork &
Collaboration
The Nurse of the Future will function effectively within nursing and interdisciplinary teams, fostering open communication,
mutual respect, shared decision-making, team learning and development.
Safety
The Nurse of the Future will minimize risk of harm to patients and providers through both individual performance and system
effectiveness.
Quality Improvement
The Nurse of the Future will use data to monitor outcomes and care processes, and use improvement methods to design
and test changes to continuously improve the quality and safety of healthcare.
Evidence-Based
Practice
The Nurse of the Future will identify, evaluate and use the best current evidence coupled with clinical expertise and
consideration of patient’ preferences, experience and values to make practice decisions.
15
Foundational Thinking
Ability to remember and understand basic information
 Clinical manifestations
 Expected reference range of laboratory results
 Causes/risk factors for an alteration in health
 Expected/side/adverse effects of medication or treatment
 Medication purpose/mechanism of action
16
Critical Thinking/Clinical Judgment
Ability to apply knowledge and use higher level critical thinking skills
(assess, analyze, and evaluate) when making decisions about clinical
problems a client scenario poses.
• Define the problem accurately
• Make inferences based on data
• Identify and examine alternative solutions
• Recognize assumptions and faulty reasoning
• Develop a plan of action
• Prioritize nursing interventions
• Perform dosage calculation
• Analyze new information
• Draw conclusions about a plan’s effectiveness
• Revise a plan based on evaluation
17
Sample Program Test Blueprint
Evaluation
Implementation
Planning
Diagnosis
Assessment
Physiological Adaptation
Reduction of Risk Potential
Pharmacological & Parental
Therapies
Basic Care & Comfort
Psychosocial Integrity
Health Promotion &
Maintenance
Safety and Infection Control
Management of care
Create
Evaluate
Remember
Understand
Apply
Analyze
Year
(11-17%)
(9-15%)
(12-18%)
(6-12%)
(6-12%)
(6-12%)
(9-15%)
(17-23%)
(15 –
20%)
(40 –
45%)
(45 –
50%)
Seme
ster 2
(0 –
5%)
(65 –
70%)
(1525%)
(15 –
20%)
Seme
ster 1
15
18
18
13
6
10
10
12
Avg.
Critical Thinking / Clinical
Judgment
Foundational
Nursing Process %
NCLEX © Category %
Cognitive Level %
Sem1
10
50
40
0
5
5
7
3
15
20
25
20
25
25
20
20
10
Sem2
0
20
60
20
20
15
10
12
10
12
10
11
20
20
20
20
20
Leveling Elements
 Establish a range for the percent of items in each element
― Avoid using exact numbers
― Assign the range of percent based upon
• Placement of course in the curriculum
• Course objectives
• Degree of target difficulty
― Place increased weight on higher level processes as students progress
through the curriculum
(Oermann and Gaberson, 2014)
19
Test Blueprinting- Traditional vs. Concept-based Curriculum
 Traditional Curriculum
― Leveled by course
― Content is body system focused
 Concept-based Curriculum
― Leveled through the curriculum across the
lifespan
― Content is concept-focused
Leveling Elements are the same for either curriculum type
20
Example: Weight of Level of Difficulty/Program Level
 Recommended range (weight) for the percent of items in each difficulty level by
program level
Example:
(The above percents are for example only and are not recommendations.)
21
22
Linking Test Items to the Syllabus
Use one of the following to link items on test to syllabus
 Course objectives
 Unit objectives
 Course content
23
Number of Alternate Items
 NCSBN has been using alternate item format questions on NCLEX®-RN/PN
since 2003.
 Alternate items give students the opportunity to demonstrate competence in
alternative ways such as calculations, identifying graphics.
 Choice of item format should be based on content being tested.
 Number of alternate items should depend on the length of the test.
 Some experts recommend limiting type of alternate format items to no more
that 3 per test.
(Oermann and Gaberson, 2014)
24
Key Components of Item Analysis
 Item Difficulty: Is the item too hard or too easy?
 Item Discrimination: Who is answering the item correctly?
25
Using Data from Item Analysis to Plan for Content ReviewThe p value
 p value ( also known as difficulty level) is the per cent of students who chose the correct
answer. The goal is a p value of 30 to 80% for majority of the items on the test.
 For purposes of content review, any item with a p value of 50% or less should be
reviewed. (A p value of 50% means that half of the students who took the exam chose
the incorrect answer)
26
Using Data from Item Analysis to Plan for Content ReviewThe Point Biserial Index
 The point biserial index is an indication of the quality of an item.
 Can range from -1.00 to +1.00. It tells us whether the high-ability students or the low
ability students choose the key. We want a positive point biserial for the key. That
means high ability students chose the key.
 Goal is a 0.30 for the key.
 For purposes of content review, any item with negative point biserial for the key should
be reviewed.
27
Hint
 Write items as you prepare for class.
 Always write rationales for both the key and distractors before you administer the test.
28
Steps in Item Analysis for Content Review
Peer
Review
• Ask a peer for input about the quality of the item.
• Two sets of eyes are better than one.
• Can you link it to a course or unit objective? Is it entry-level practice? Can you link it to the NCLEX© test
plan?
Relevance • Can you identify the step of the nursing process?
• For concept-based curricula, is the item at the proper level of difficulty inline with your curriculum?
• If you have used the item with a prior group of cohorts, did the prior stats meet the acceptable ranges?
Item
Quality
• Is the stem free of extraneous content?
• Is the question in the stem clear?
• Is there only one correct answer?
29
Content Remediation
 Policies on content remediation should be incorporated into the program’s testing
policies.
 Explore with students either individually or in a group their understanding of the
content/concept.
 Assign activities for review.
 Retest the content/concept if possible.
30
Summary and Final Thoughts
 Questions and comments
 Plans for moving forward
 Aha moments
31
References
American Association of Colleges of Nursing (AACN), 2008 The essentials of baccalaureate
education for professional nursing practice. Retrieved from
http://www.aacn.nche.edu/educational-resources08.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing:
A revision of Bloom’s taxonomy of educational objectives. Boston, MA: Pearson Education
Group.
Billings, D., & Halstead, J. (2016). Teaching in nursing: A guide for faculty (5th ed.). St. Louis,
MO: Saunders.
Bloom, B.S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification
of educational goals, by a committee of college and university examiners. Handbook I: Cognitive
domain. New York, NY: Longmans, Green.
32
References (Continued)
Massachusetts Nurse of the Future Nursing Core Competencies© Retrieved from
www.mass.edu/nursing
National Council of State Boards of Nursing. (2016). NCLEX-RN® Examination: Test plan for the
National Council licensure examination for registered nurses. Retrieved from
https://www.ncsbn.org/RN_Test_Plan_2016_Final.pdf
National Council of State Boards of Nursing. (2017). NCLEX-PN® Examination: Test plan for the
National Council licensure examination for nurses. Retrieved from
https://www.ncsbn.org/PN_Test_Plan_2017_Educator_v2.pdf
Oermann, M., & Gaberson, K. (2014). Evaluation and testing in nursing education. (5th ed.). New
York, NY: Springer.
Quality and Safety Education for Nurses Institute (2013). Competencies. Retrieved from
http://qsen.org/compentencies/
33