GPC 364
Appraisal of Students
Planning
Achievement
Tests Unit 4
Dean Owen, Ph.D., LPCC
METU-NCC
Spring 2017
Citation Note

All material, unless specifically cited,
has been drawn from:
Owen, D. (2011). The role of assessment in mental health
counseling. In W.J. Weikel & A.J. Palmo (Eds.),
Foundations of Mental Health Counseling (4th ed.)
(pp.333-353). Springfiled, IL: Charles C. Thomas,
Publisher.
Lecture Note
Today’s lecture will be composed of three
parts…..
1. Planning a test
2. Selecting and writing recognition items
3. Selecting and writing supply items
….but first
Discussion and collection of homework
assignment (One page report)
and
Weekly SRI example
Owen, D. (2007). Tools for the counselor-Part III: A Self-Efficacy Scale.
Neues Perspectives, 19 (2), 12-13.
Specific Learning Outcomes
SLO
By the end of this lecture each student
will be able to:
Describe the 4 categories of learning objectives
Recall the levels of learning by Bloom and by Owen
Construct and use a table of specifications
Planning the Achievement Test

A test is NOT something that is tacked
onto the end of a period of instruction.
Ideally, a test is “built” at the same
time a unit of instruction is designed.

A test is NOT an opportunity to go
on a treasure hunt to find out what
the students have learned.

Planning the Achievement Test
As counselors in future you will be called
upon to provide many types of training for
students, faculty and even parents and
school administrators…..
This will require that you have some basic
understanding of planning and evaluating a
unit of instruction….sooooo……..Let’s plan a
trip!!
Planning a trip
1. What information do we need to plan a
trip??
How will I go…
How many are going?
How long will we stay???
Where am I now??
Where do I want to go?
How will I know if I get
there?
The 4 Types of Objectives
You can’t possibly get somewhere if you
have no idea where you want to end
up.
Ultimate vs Immediate
Process vs Product
Implicit vs Behavioral
Restricted vs Inclusive
The 4 Types of Objectives
Ultimate vs Immediate
Ultimate objectives refer to an unspecified time
somewhere in the distant future and no attempt will
be made to assess…..but they provide a general
goal…..”Someday I hope my students will come to
appreciate literature…”
Immediate: These begin with …by the end of the
week, by the end of the term…and an attempt will be
made to assess them…..”By the end of the workshop
each participant will be able to use two relaxation
techniques to reduce stress.”
The 4 Types of Objectives
Process vs Product
Process: These objectives describe a learning activity
that will assist in creating change. “Each of the
students will read chapters 4, 5, and 6 in the text
book.”
Product: These objective describe how the student
will have changed because they complete the
process above. “Each of the students will be able to
diagram and label the major parts of the human
digestive system.”
The 4 Types of Objectives
Implicit vs Behavioral
Implicit objectives describe internal states that cannot
be seem and use verbs like “to know”, “to
understand,” “to comprehend” or “to appreciate”.
Behavioral objectives describe overt behaviors that can
to seen…..to write, to build, to discuss, to label, to
perform, to select, to dance, to speak…..etc.
You can see these when they occure!!
The 4 Types of Objectives
Restricted vs Inclusive
Restrictive objectives describe specific examples of
things……”The students will be able to identify: the
stomach, spleen, liver, gal bladder, small intestine,
large intestine.
Inclusive objectives describe a category or group of
things….The students will be able to identify the major
organs of the human digestive system.
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Knowledge (most simple learning)
Comprehension
Application
Analysis
Synthesis
Evaluation (most complex)
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Knowledge
“involves the recall of specifics and
universals, the recall of methods and
processes, or the recall of a pattern,
structure, or setting.”…Simple memory
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Comprehension
“refers to a type of understanding or
apprehension such that the individual
knows what is being communicated and
can make use of the material or idea being
communicated without necessarily relating
it to other material or seeing its fullest
implications.” Simple understanding
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Application
refers to the “use of abstractions in
particular and concrete situations.”
The ability to “use” information….
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Analysis
represents the “breakdown of a
communication into its constituent
elements or parts such that the relative
hierarchy of ideas is made clear and/or
the relations between ideas expressed
are made explicit.”
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Synthesis
involves the “putting together of
elements and parts so as to form a
whole.”
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom’s taxonomy in the Cognitive Domain….
Evaluation
engenders “judgments about the value
of material and methods for given
purposes.”
Source: http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
Levels of Learning
Bloom
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
The RUT
Levels of Learning
Bloom
Knowledge
Owen
Remembering
Comprehension
Application
Understanding
Analysis
Synthesis
Evaluation
Thinking
The Table of Specifications




Represents a plan for your test
Insures adequate item sampling
Insures proportional representation of
content material on the test.
Provides a consistent and logical basis
for the structure of you assessment
tool.
Table of Specifications…An example




You want a test that can be completed
in 1 hour of testing.
You want quick, objective scoring
You want to be able to return scores to
students quickly…and go over the test.
You expect to teach the same material
during a subsequent semester…..so
Table of Specifications…An example




Let’s decide on a 30 item, 4 option multiple
guess instrument…….
You’re teaching a unit on meteorology
(weather) in a general science class…..
You’ve decided on 5 major areas to
cover…..not a bad idea…
You’re going to use 10 days, 2 class meetings
for each of the 5 major areas…This may change
later but, remember, we’re just planning now…….
The Table of Specifications
Topic
H2O Cycle
Instruments
Air Mass Stuff
Clouds
Severe Wx
R
U
T
#
The Table of Specifications
Topic
R
U
T
#
H2O Cycle
6
Instruments
6
Air Mass Stuff
6
Clouds
6
Severe Wx
6
30
The Table of Specifications
Topic
R
U
T
#
H2O Cycle
3
2
1
6
Instruments
4
1
1
6
Air Mass Stuff
3
2
1
6
Clouds
5
1
0
6
Severe Wx
4
1
1
6
19
7
4
30
Table of Specifications


The majority of questions are R’s
insuring that even the poorest students
get some correct.
There are enough T’s to be able to
discriminate among the very best
students in the class.
Item File Creation


Armed with a list of specific objectives and
goals….begin writing items as you teach each
section.
Write each item to assess a major or valued
objective and classify it as R, U, or T
depending upon what an average student
would need to answer it….Remembering,
Understanding or Thinking.
Example of an Item Card
Science
Example of an Item Card
Science / Meteorology
Example of an Item Card
Science / Meteorology / Clouds /
Example of an Item Card
Science / Meteorology / Clouds /
Which of the following cloud types is likely to develop
under conditions of warm, moist, and unstable air
conditions on summer afternoons?
a.
b.
c.
d.
Lenticular
Cirrus
Strato-nimbus
Cumulus
R
Example of an Item Card
Science / Meteorology / Clouds /
Identify the cloud type in
the photo to the right.
a.
b.
c.
d.
Lenticular
Cirrus
Stratus
Cumulus
R
Example of an Item Card
Science / Meteorology / Clouds /
Which of the following cloud formations is likely to
develop under conditions of warm, moist, and
unstable air conditions?
a.
b.
c.
d.
Lenticular
Cirrus
Strato-nimbus
Cumulus
U
Example of an Item Card
Science / Meteorology / AMP /
T
An occluded warm front running North and South is
located 70 miles west of Knoxville, Tn. The
Autumnal Bermuda high pressure area is now
displaced 200 east of it’s normal position. What will
Atlanta’s forecast likely be for the next 24 hours?
a.
b.
c.
d.
Clear skies and warm temperatures
Steady cold rain
Severe Thunderstorms
Clear changing to overcast and rain
developing late in the period
Test Assembly





By referring to the Table of specifications you
can now begin assembling the test.
Arrange items for “simple” to “difficult”
Prepare an answer sheet or use a stock one
Prepare an answer key….
Remember: What you’ve created is a
prototype and at this point you really
have no idea about how well it will
ultimately perform.
Review





Start by deciding what you want your
student to “know” or “be able to do”.
Write a list of specific objectives.
Prepare your materials and lectures
Prepare a blueprint or Table of
Specification for your assessment.
Write your items as you teach….not two
weeks later!
Review



Review the quality of your items before
including them in your test.
Assemble your test using the Table of
Specifications as a guide.
Administer and score your instrument but
don’t trust it….you’ve just “test flown” an
experimental instrument….it can probably be
made better….more later on that……
Please be
back in 15
minutes.
There is still
much to
discuss……