Describing Cognitive Level of Teacher Discourse, and Student Retention of Content,
during a Secondary Agricultural Science Unit of Instruction
DISSERTATION
Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy
in the Graduate School of The Ohio State University
By
Jeremy M. Falk
Graduate Program in Agricultural and Extension Education
The Ohio State University
2011
Dissertation Committee:
Dr. M. Susie Whittington, Advisor
Dr. Robert J. Birkenholz
Dr. Jamie M. Cano
Dr. Robert R. Hite
Copyright by
Jeremy M. Falk
2011
Abstract
The purpose of this research was twofold: to describe the cognitive level of
teacher discourse in secondary agricultural education classrooms, and to describe student
retention of content from both the perspective of cognitive level of content delivery and
cognitive level of content assessment.
Research questions of the study included:
Research Questions
1. What cognitive level of discourse do teachers exhibit in secondary agricultural
science classrooms?
2. At what rate are students retaining content in secondary agricultural science
classrooms, at the immediate and short-term intervals?
3. What patterns exist between teacher cognitive discourse and student retention of
content in secondary agricultural science classrooms?
Two secondary agricultural science teachers were videotaped during one week‟s
classes. Instruments were used to determine cognitive level of teacher discourse, and
cognitive level of student retention. Patterns were examined.
Results were used to show that teacher‟s discourse was at the lowest levels of
cognition 68 percent of the class time for one teacher, and 79 percent of the class time for
the other teacher. Additionally, student content retention scores were highest on lower
cognitive level questions on the post1, and post2 tests.
It was concluded that students retained content during the 49-day observation, and
that retention rates were highest at the levels of cognition in which the class sessions
ii
were taught. A pattern began to emerge in which students retained the same cognitive
level of questions on the post1 and post2 as the cognitive level of discourse that was
delivered by the teacher. Content, therefore, was being retained at the same levels at
which the content was taught.
Recommendations included teachers writing assessments at the levels of
cognition they wish their students to perform, and to align those assessments with
classroom discourse. A second recommendation was for teachers to clearly state the goals
for a unit of instruction and continue to check student progress along those goals during
the unit. Further recommendations included teacher educators preparing pre-service
teachers for teaching across levels of cognition.
iii
Acknowledgments
Affectionately, the investment that Dr. M. Susie Whittington has devoted toward
me has earned her the title of Mom. It has been with tireless tenacity in which Dr.
Whittington has pushed my intellectual, personal, and professional being. While there are
many role models in the profession of Agricultural Education, there are none that I hold
in higher esteem than Dr. Whittington.
Along with my advisor, my dissertation committee became an asset and helped
stimulate my scholarly thinking. Dr. Birkenholz prompted me to think bigger; Dr. Cano
instilled a sense of consistency; and Dr. Hite‟s creativity and eagerness for discovery will
forever be with me. I appreciate the helpful guidance and friendship you each have given.
My family, along this path, has been extremely supportive. It has been with their
love that I can search for the importance in the work that I do and strive to positively
impact the world. I wish to thank all of my friends, role models, and guidance-givers
during my graduate career.
iv
Vita
2003................................................................B.S. Agricultural Education, The Ohio State
University
2003-2007 ......................................................Agricultural Education Instructor,
Federal Hocking High School, Stewart, OH
2010................................................................M.S. Agricultural and Extension Education,
The Ohio State University
2007-2010 ......................................................Graduate Teaching Associate, Human and
Community Resource Development, The
Ohio State University
2011................................................................Ph.D. Human and Community Resource
Development, The Ohio State University
v
Publications
Refereed Journal Article
Connors, J. J., Falk, J. M., & Epps, R. B. (2010). Recounting the legacy: The history and
use of FFA camps for leadership and recreation. Journal of Agricultural Education, 51(1),
pp. 32-42. doi:10.5032/jae.2010.01032
Refereed Poster Proceedings
Falk, J. M., & Whittington, M. S. (2003). We took the lead, so they could lead. Poster
Proceedings of the 30th Annual National Agricultural Education Research Conference.
Orlando, FL.
Whittington, M. S., Foster, D. D., Falk, J. M., Beck, W. M., & Bookman, J. A. (2008).
Creating a conceptual framework for studying cognitive levels of teaching and learning.
Poster Proceedings of the Annual National Agricultural Education Research Conference.
Reno, NV. Third place poster.
Falk, J. M., Beck, W. M., & Whittington, M. S., (2008). The real deal. Poster
Proceedings of the Annual National Agricultural Education Research Conference. Reno,
NV.
Falk, J. M., Beck, W. M., & Whittington, M. S. (2009). The real deal: Formal preparation
for outreach education. Poster Proceedings of the Annual Ohio Extension Educator‟s
Conference. Columbus, OH.
Falk, J. M., Batts, A, N., & Whittington, M. S. (2010). Describing the cognitive level of
discourse of a secondary teacher during an animal science unit of instruction. Poster
Proceedings of the Annual National Agricultural Education Research Conference.
Omaha, NE.
Fields of Study
Major Field: Agricultural and Extension Education
vi
Table of Contents
Abstract ............................................................................................................................... ii
Acknowledgments.............................................................................................................. iv
Vita...................................................................................................................................... v
Table of Contents .............................................................................................................. vii
List of Tables .................................................................................................................... xii
List of Figures .................................................................................................................. xiii
Chapter 1: Introduction ...................................................................................................... 1
Teaching and Learning .................................................................................................... 1
Theoretical Foundation ................................................................................................... 2
Need for the Study........................................................................................................... 3
Statement of the Problem ................................................................................................ 5
Purpose of the Study ....................................................................................................... 6
Research Questions ......................................................................................................... 7
Constitutive Definitions of Terms ................................................................................... 7
Operational Definitions of Terms ................................................................................... 8
Limitations of the Study .................................................................................................. 9
vii
Delimitations ................................................................................................................. 10
Summary ....................................................................................................................... 10
Chapter 2: Review of Literature ....................................................................................... 12
A Framework for Measuring Classroom Cognition ...................................................... 12
Bloom‟s Taxonomy for the Cognitive Domain ......................................................... 12
Taxonomy of Learning, Teaching, and Assessing (Anderson & Krathwohl, 2001) . 13
Newcomb-Trefz Model ............................................................................................. 14
Teaching Techniques and Higher Order Teaching .................................................... 15
Cognition Research in Agricultural Education ............................................................. 17
Levels of Cognitive Behaviors in Agriculture Classrooms ....................................... 17
Relationships between Teaching Techniques and Content Retention ....................... 21
Conceptual Framework ................................................................................................. 25
Summary ....................................................................................................................... 27
Chapter 3: Methods ........................................................................................................... 28
Subjects ......................................................................................................................... 29
Procedures ..................................................................................................................... 30
Instrumentation – Unit Tests ......................................................................................... 32
Validity of Agricultural Products Test ...................................................................... 33
Reliability of Agricultural Products Test................................................................... 33
viii
Data Collection .......................................................................................................... 34
Data Analysis ............................................................................................................. 35
Validity of Agricultural Sales Test ............................................................................ 36
Reliability of Agricultural Sales Test ........................................................................ 37
Data Collection .......................................................................................................... 37
Data Analysis ............................................................................................................. 38
Instrumentation - FTCB ................................................................................................ 39
Validity ...................................................................................................................... 40
Reliability .................................................................................................................. 40
Data Collection .......................................................................................................... 41
Data Analysis ............................................................................................................. 41
Summary ....................................................................................................................... 42
Describing the Cognitive Level of Discourse ............................................................... 43
Describing Student Retention of Content in an Agricultural Products Unit of
Instruction...................................................................................................................... 46
Describing Student Retention of Content in an Agricultural Sales Unit of Instruction 50
Patterns between Teacher Cognitive Discourse and Student Content Retention .......... 53
Summary ....................................................................................................................... 54
Chapter 5: Conclusions ..................................................................................................... 56
ix
Executive Summary ...................................................................................................... 56
Summary of Findings .................................................................................................... 58
Conclusions Related to Cognitive Levels of Discourse ................................................ 58
Discussion Related to Cognitive Levels of Discourse .................................................. 59
Cognitive Distribution ................................................................................................... 61
Recommendations and Implications Related to Cognitive Levels of Discourse .......... 62
Conclusions Related to Student Content Retention ...................................................... 62
Discussion Related to Student Content Retention ........................................................ 63
Recommendations and Implications Related to Student Content Retention................. 64
Conclusions Related to Patterns between Cognitive Discourse and Student Content
Retention ....................................................................................................................... 65
Discussion Related to Patterns between Cognitive Discourse and Student Content
Retention ....................................................................................................................... 65
Recommendations and Implications Related to Patterns between Cognitive Discourse
and Student Content Retention...................................................................................... 66
Further Discussion on Student Content Retention and Cognitive Studies .................... 67
Assessment of Content Retention .............................................................................. 68
Additional Factors that Contribute to Cognitive Gains ............................................. 69
Summary ....................................................................................................................... 70
x
References ......................................................................................................................... 71
Appendix A: Unit of Instruction – Agricultural Production ............................................. 77
Appendix B: Unit of Instruction – Agricultural Sales .................................................... 101
Appendix C: Florida Taxonomy of Cognitive Behavior ................................................ 108
Appendix D: Correspondence ......................................................................................... 112
xi
List of Tables
Table 1: Taxonomy of Educational Objectives - Cognitive Domain ................................. 3
Table 2: Cognitive Level of Questions on an Agricultural Products Test ........................ 35
Table 3: Classification of Agricultural Sales Test Questions ........................................... 38
Table 4: Agricultural Products Unit, percent cognitive level of teacher discourse as
measured using the Florida Taxonomy of Cognitive Behavior by class session .............. 44
Table 5: Agricultural Sales Unit, percent cognitive level of teacher discourse as measured
using the Florida Taxonomy of Cognitive Behavior by class session .............................. 45
Table 6: Agricultural Products Unit Test scores ............................................................... 48
Table 7: Retention scores by cognitive level of questions on the Agricultural Products
Test .................................................................................................................................... 50
Table 8: Agricultural Sales Unit test scores ...................................................................... 51
Table 9: Retention scores by cognitive level of questions on the Agricultural Sales Unit
Test .................................................................................................................................... 53
Table 10: Patterns between teacher cognitive discourse and student content retention ... 54
xii
List of Figures
Figure 1: A Summary of Structural Changes from the Original Taxonomy to the Revision
(Anderson & Krathwohl, 2001) ........................................................................................ 14
Figure 2: A Comparison of Bloom's Taxonomy and the Newcomb-Trefz Model ........... 15
Figure 3: Conceptual Framework for Studying Cognitive Levels of Teaching and
Learning ............................................................................................................................ 26
Figure 4: A comparison of Bloom's Taxonomy, the Florida Taxonomy of Cognitive
Behavior, and the Newcomb-Trefz Model ....................................................................... 40
xiii
Chapter 1: Introduction
Teaching and Learning
Gardner (1999), a leading author, psychologist, and educator, wrote:
Whether I am traveling in the United States, or visiting Europe, Latin
America, or the Far East, I find a surprising consensus: the belief that the
quality of a nation‟s educational system will be . . . the chief determinant
of its success during the next century . . . . (p. 15)
The strength of education lies in its ability to afford a society the power to think
and solve problems -- cognitive activities that Bloom, Engelhart, Furst, Hill, and
Krathwohl, (1956) classified as higher-order activities. Yet it is widely accepted among
American educators that higher-order thinking skills are declining in the student
population. Van Gelder (2005) wrote, “Almost everyone agrees that one of the main
goals of education . . . is to help develop general thinking . . . . Almost everyone also
agrees that students do not acquire these skills as much as they could or should” (p. 1).
Kuhn (1999) speculated that the total mass of knowledge is so great that none of it
can be learned well, and that too often students are required to memorize a body of facts
that are much easier to forget than to remember. Kline (2002) further detailed the
information-overload phenomenon by writing, “The amount of available information in
all fields is growing at more than a billion times the rate it was in 1950. By the year 2010
it will be continuing to expand at more than ten times that rate” (p. xii). Kuhn offered
1
advice for teachers grappling in this information era, “Teaching for permanent learning
must go beyond dissemination of information to the development of student interest and
thinking abilities. Through thinking, students become actively involved in learning” (p.
16).
Theoretical Foundation
The theoretical foundation for this line of inquiry was Piaget‟s Theory of
Cognitive Development (1970), along with Bloom‟s Taxonomy of Educational
Objectives (Bloom, 1964). Piaget was interested in human development and outlined
several factors that influence how people think; these influences included maturation,
activity and social transmission. Teachers have little influence to develop a student‟s
level of maturation in Piaget‟s theory, but teachers can help alter the way a student thinks
through the activity factor, by providing opportunities for exploration, observation,
assessment, and structured content. The third factor, that of social transmission, can be
stimulated through learning activities that allow students to learn from others (Ewing &
Whittington, 2007).
Although Piaget‟s Theory of Cognitive Development serves as the foundation for
this research, Piaget‟s activity factor can be further refined by incorporating Bloom‟s
Taxonomy (1956) to categorize cognitive levels of educational activities. Bloom et al.
(1956) created a hierarchy of cognitive levels that categorize classroom activity,
discourse, and objectives into six levels (see Table 1). Knowledge and comprehension are
most often considered lower levels of cognition, while application, analysis, synthesis
and evaluation comprise the higher levels of cognitive functions (Miller, 1989).
2
Category
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Behavior
Remember
Translate, interpret, extrapolate
Use abstractions in specific situations
Break down concepts into components
Use parts or elements to form a whole
Judge value of materials
Table 1: Taxonomy of Educational Objectives - Cognitive Domain
Note. Bloom et al., Taxonomy of Educational Objectives, Cognitive Domain (1956)
Need for the Study
America‟s educators have been criticized for failing to teach students to think
(Halpern, 1984). Research provides evidence that instructors deliver classroom discourse
(teacher talk) at the lowest cognitive levels (Whittington & Newcomb, 1993; Ewing &
Whittington, 2007). There is a need, yet little research has been conducted, to determine
the impact of lower level teaching on long-term student retention and transfer of learning
(Foster, 2009).
Sir Ken Robinson, a creativity expert and education reformer, presented at the
2010 Technology, Entertainment, and Design (TED) conference and relates education to
a living process, saying:
We have to recognize that human flourishing is not a mechanical process, it is an
organic process. And you cannot predict the outcome of human development, all
you can do is, like a farmer, create the conditions under which they will begin to
flourish. So when we look at reforming education and transforming it, it isn‟t like
cloning a system…. it‟s about customizing them to your circumstances and
personalizing education to people you are actually teaching.
3
Linda Darling-Hammond (2006), stated that,
Teaching that aims at deep learning, not merely coverage of material, requires
sophisticated judgment about how and what students are learning, what gaps in
their understanding need to be addressed, what experiences will allow them to
connect what they know to what they need to know, and what instructional
adaptations can ensure that they reach common goals. (p. 10)
Adding complicated assessments to the student and teacher interactions can
influence how subject matter is taught. “Teaching for retention during a single academic
term to prepare students for an assessment that will be given to them in the same context
in which the learning occurs is very different from teaching for long-term retention and
transfer” (Halpern & Hakel, 2003, p. 38).
Edwards and Ramsey (2004) stated, “. . .the agriculture, food, and environmental
system could be an appropriate learning context for assisting students to think critically
. . . assuming agriculture teachers demonstrated effective instructional behaviors in a
sustained fashion and supported progressive cognitive learning by their students” (p.
164). Questions still remain regarding how to engage students in critical thought, in a
manner that supports retention and transfer.
Within Agricultural Education, a strategic plan and action agenda was created
within the initiative, Reinventing Agricultural Education for the Year 2020. Four goals
were created to assert Agricultural Education as the premier educational delivery model
in the United States (National Council for Agricultural Education, 2000). In this
document it was stated that, “In today‟s rapidly changing world, agricultural education
4
must be prepared to change constantly and make adjustments to meet new challenges and
opportunities” (p. 6). The impact that agricultural education can have on students can
advance the quality of life for everyone.
Statement of the Problem
According to Kolers and Roediger (1984), learning and memory research often
places more emphasis on what is learned while neglecting the strategies, techniques, and
methods by which the learning took place. In addition, Ewing & Whittington (2007)
suggested that the behaviors of instructors in agricultural education should be focused on
building the capacity to teach at higher cognitive levels.
Ulmer (2007) posited that, “Research has failed to reveal how characteristics of
teachers, schools, and/or classes effect teaching at higher levels of cognition or how
related disciplines compare in cognitive behavior” (p. 108). According to Farr (1987),
“. . . there is surprisingly and disappointingly little in the literature of practical use to the
learning and retention of the broad range of complex, real-world „cognitive‟ tasks” (p. 1).
Farr also noted that knowledge is forgotten and skills deteriorate when they are not used
or practiced. “All in all, we need to know a great deal more of useful information about
the factors which promote long-term retention and retard „decay‟” (Farr, 1987, p. 2).
Teachers are held accountable for the success of their students. Although, current
education systems often measure teacher success by standardized test scores, is the real
value of education the productivity that is afforded society when students use the content
they have retained to make long-term decisions and solve problems? If so, what
5
techniques and strategies are teachers using to help students retain the content for longer
periods of time? Could teaching content at higher cognitive levels lead to student
engagement of the content and consequent long-term retention?
Whittington (2003) asserted that learning experiences are enhanced when students
performed at higher levels of cognition. In addition, to achieve the goal of preparing
students to be able to contribute to society during their lives, Whittington advocates that
teachers challenge students at high levels of cognition during class sessions. Therefore,
the central problem for this study was: If teachers teach across all of Bloom‟s levels of
cognition, will student retention of content increase?
Purpose of the Study
The purpose of this research was twofold: to describe the cognitive level of
teacher discourse in two secondary agricultural education classrooms, and to describe
student retention of content from both the perspective of cognitive level of content
delivery and cognitive level of content assessment. The design of this study was
descriptive, and examined teacher discourse at each level of Bloom‟s Taxonomy and the
proportion of classroom discourse delivered at higher and lower levels of cognition. In
addition, student retention of content was assessed immediately after delivery of a fiveday unit of instruction, and again following a short-term interval (49 days after delivery
of instruction). The research subjects included two secondary agricultural science
teachers who were teaching a total of 42 students.
6
Research Questions
1. What cognitive level of discourse do teachers exhibit in secondary agricultural
science classrooms?
2. At what rate are students retaining content in secondary agricultural science
classrooms, at the immediate and short-term intervals?
3. What patterns exist between teacher cognitive discourse and student retention of
content in secondary agricultural science classrooms?
Constitutive Definitions of Terms
Definitions presented in this section provide an explanation of how each term was
used as it appears in the study. The list is meant to assist the researcher by working within
the boundaries of each term.
Cognitive discourse –Verbal statements spoken by high school teachers during class
sessions, categorized into six levels of cognition.
Higher cognitive levels – The upper four levels of Bloom‟s Taxonomy of Educational
Objectives in the cognitive domain (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956).
The levels are labeled application, analysis, synthesis and evaluation.
Lower cognitive levels – The lower two levels of Bloom‟s Taxonomy of Educational
Objectives in the cognitive domain (Bloom et al., 1956). The levels are labeled
knowledge and comprehension.
Student immediate retention – The amount of knowledge that the student retained
immediately after the unit of instruction, as evidenced by the unit test score post1.
7
Student retention score – A numerical value, used to represent the change in knowledge
between the post-test (post1), and the post-post test (post2).
Student short-term retention – The amount of knowledge that the student retained 49
days after the unit of instruction, as evidenced by the unit test score post2.
Operational Definitions of Terms
Student immediate retention – The sum of a student‟s score on the unit test. Measured,
using unit tests, immediately after the unit of instruction was taught.
Student short-term retention – The sum of a student‟s score, on each test item, from the
second test in relation to the first test. Students‟ retention scores increase when they
correctly answer a test item on both tests. Measured, using unit tests, 49 days after the
unit was taught.
Student retention score – Student retention scores were calculated by comparing post-test
(post1) and post-post test (post2) scores. If a student answered a question correctly on
post1, and also correctly on post2, the student received a retention score of 2.0. Students
who answered correctly on post1, but incorrectly on post2 received a retention score of
-1.0. Students, who incorrectly answered a question on both tests, received a retention
score of 0.0. Finally, a student who did not answer the question correctly on post1, but got
the question correct on post2, received a score of 1.0. Scores were averaged for each
class, and a mean retention score was assigned to each teacher.
Teacher cognitive discourse – Observed teacher statements were measured using The
Florida Taxonomy of Cognitive Behavior (FTCB) (Webb, 1970). Frequencies were
8
measured every six minutes of a class session. Measured frequencies of observed
behaviors were totaled for each level of cognition, during each class session, yielding a
subtotal of teaching behaviors at each cognitive level for each class session. Then,
subtotals for each cognitive level were summed for each class session for each teacher.
The sum for each level of cognition was divided by the total number of class sessions
observed for each teacher, resulting in a percentage of discourse for each teacher at each
level of cognition.
Limitations of the Study
Acknowledged limitations were found in this study, which occurred from the
procedures used; these limitations effect how the conclusions can be generalized.
Convenience sampling was used to select participants for this study, which limits the
ability to generalize beyond the sample of teachers used as subjects.
The collection of evidence is also a limitation. Classroom videotapes were used
by the researcher instead of direct observation of the classes every day, limiting the
ability to fully understand the context and conditions of the learning environment.
Additionally, using a pencil and paper test to measure student achievement and
retention has limitations. Quantifiable tests are practical for research purposes, but can be
over-generalized, therefore, not taking into account the individual student. Further, Tsui
(2002) stated that,
On the whole, research studies on critical thinking have not displayed great
variation in research design. There appears to be an overwhelming reliance on
9
quantitative data of a certain sort. More specifically, researchers tend to use
standardized multiple-choice tests to measure critical thinking and students‟
responses on questionnaire surveys to measure classroom and out-of-class
experiences. Yet, any single research method is necessarily limited in its
capability and endowed with its own particular shortcomings. (p. 742)
Delimitations
The delimitations of this study established the boundary between what is included
in this research, and what is not a part of this study. The purpose of this research was
twofold: to describe the cognitive level of teacher discourse in secondary agricultural
education classrooms, and to describe student retention of content from both the
perspective of cognitive level of content delivery and cognitive level of content
assessment. The study was designed to focus on a sample of two teachers and their
classroom behaviors. The study was not intended to describe students in the classrooms.
To be clear, this study was not written to describe the reasons students do or do
not retain content. The study was written to focus on the teacher and the cognitive level
of teacher discourse that is possibly related to student content retention.
Summary
Chapter One was designed to guide the reader through the introduction of a study
that described teacher cognitive discourse and student retention of content. Although the
10
results of the study were not intended to be generalized to a population of teachers, the
results may further develop lines of inquiry into student retention of content.
The results of this study can be beneficial to practicing teachers, as well as teacher
educators. Practicing teachers can use the findings to lead them to reflect upon their
personal discourse for teaching at higher levels of Bloom‟s Taxonomy and what
consequences their discourse may have on students retaining information. Teacher
educators can use the findings of this study to base their line of research on student
retention of knowledge, as well as to encourage pre-service candidates to be aware of the
relationship between teacher discourse at higher levels cognition and student retention of
content.
Additionally, the results of this study will be useful in the professional
development of teachers. Persons responsible for providing ongoing training and
education of teachers can use the findings to create awareness about the impact of teacher
discourse across all levels of cognition on student retention of content.
11
Chapter 2: Review of Literature
A Framework for Measuring Classroom Cognition
Benjamin Bloom, then Associate Director of the Board of Examinations of the
University of Chicago, initiated the idea of creating a framework for classifying
statements of what we expect students to learn as an outcome of instruction (Krathwohl,
2002). Bloom and his colleagues created the Taxonomy of Educational Objectives: The
Classification of Educational Goals. Handbook I: Cognitive Domain. Bloom‟s
Taxonomy created common language for learning goals, established standards for
courses and curriculum, and determined comparable educational objectives, assessments
and activities.
Bloom’s Taxonomy for the Cognitive Domain
Bloom‟s Taxonomy (1956) consists of a six-level classification scale: knowledge,
comprehension, application, analysis, synthesis, and evaluation. According to Bloom et
al. (1956), higher cognitive thinking is represented at the application, analysis, synthesis,
and evaluation levels of the Taxonomy. In this model, the levels build upon each other
and the higher levels require deeper, fuller, understanding of the content. Bloom et al.
(1956) posit that knowledge emphasizes remembering; comprehension involves making
use of the material, yet not relating it to other implications; application involves transfer
12
of abstractions to concrete situations; analysis breaks learned material and concepts into
categories for processing; synthesis involves putting together parts to make a whole idea;
and evaluation includes making judgments, while using given knowledge to inform the
decision. According to Tsui (2002), “Higher-order cognitive skills, such as the ability to
think critically, are invaluable to students‟ futures” (p. 740).
Taxonomy of Learning, Teaching, and Assessing (Anderson & Krathwohl, 2001)
Bloom‟s original Taxonomy (Bloom, Engelhart, Furst, Hill & Krathwohl, 1956),
was revised 45 years later by Anderson and Krathwohl in 2001. The original taxonomy
consists of six categories, arranged in a cumulative hierarchical framework. The revised
taxonomy is a two-dimensional framework, eliminating sub-categories from the original,
instead using them in the form of a table for increased function for classifiers (Krathwohl,
2002). One important change is in the higher levels of cognition, where Synthesis is
changed to Create, and is ranked above Evaluate in the revised Taxonomy. See Figure 1
for a visual model of the revised Taxonomy. The researcher in this study chose to use
Bloom‟s original Taxonomy because there were no changes in the meanings, or
classification of high and low levels of cognition. Additionally, the original Taxonomy is
still frequently cited in the body of literature and the familiarity for researchers and
practicing teachers is beneficial in this descriptive research.
13
Figure 1: A Summary of Structural Changes from the Original Taxonomy to the Revision
(Anderson & Krathwohl, 2001)
Newcomb-Trefz Model
Classroom learning behaviors and activities can be classified to determine
whether or not class objectives are being met. Newcomb and Trefz (1987) developed a
model, simplifying the six levels of Bloom‟s Taxonomy, see Figure 2. Newcomb and
Trefz identify the knowledge level of Bloom‟s Taxonomy as remembering in their model.
The remembering level requires no understanding of the information, only recall. The
next level of the Newcomb-Trefz Model is named processing and it combines the
comprehension, application, and analysis levels of Bloom‟s Taxonomy. The processing
level requires the learner to use facts to reach answers for given situations. Creating, the
next level of the Newcomb-Trefz Model requires the development of a product. The final
level of the Newcomb-Trefz Model is evaluating. When operating at the evaluation level,
learners must make judgments based on criteria to defend an answer.
14
Bloom‟s Taxonomy The Newcomb-Trefz Model
Knowledge
Remembering
Comprehension
Processing
Application
Analysis
Synthesis
Creating
Evaluation
Evaluating
Figure 2: A Comparison of Bloom's Taxonomy and the Newcomb-Trefz Model
Teaching Techniques and Higher Order Teaching
Gardiner (1998) reviewed several studies about college students. Combined, the
studies showed that students‟ college experiences included “loosely organized, unfocused
curriculum, with undefined outcomes, classes that emphasize passive listening, lectures
that transmit low-level information, and assessments of learning that frequently demand
only the recall of memorized material or low-level comprehension of concepts”
(Gardiner, p. 72).
According to Halpern & Hakel (2003), lectures are not the best teaching tool to
promote deep learning. Many lecture-based learning environments are associated with
multiple-choice tests, which “tap only lower-level cognitive processes” (Halpern &
Hakel, 2003).
Ewing & Whittington (2007), in a study of 12 professors, and 21 class sessions in
a Midwestern College of Agricultural Sciences, found that student cognition is lessened
by increased amounts of lecture during class sessions. Lecture alone generally does not
15
allow students to be active in the learning process (Mangurian, Feldman, Clements, &
Boucher, 2001), but teachers can use other group, and individualized teaching techniques
to increase active student learning. Lectures that require students to simply recall facts, a
lower cognitive skill, promote students to operate at lower levels of cognition during
class sessions and during individual review for exams. Class discussions and debates can
lead to higher-order thinking because information is actively processed (instead of merely
recorded), is more readily retrieved from memory, has more application to new events,
and is less prone to being forgotten (Tsui, 2002). Teachers are being encouraged to teach
with their mouths shut and to encourage students to participate in discourse, drawing, and
role playing (Finkle, 2000; Klemm, 2007).
Nunn (1996) conducted a study with 20 university professors and 579 students to
examine teaching techniques that encouraged student participation in class and the
researcher found that asking questions and probing for elaboration from student
responses was significant in promoting student participation. Asking questions and
probing for elaboration can often promote higher-order thinking for students.
In a study of 12 professors, with 138 student subjects, Smith (1977) discovered
that critical thinking was positively linked to three kinds of teacher-influenced
interactions: the extent of encouragement, or praising student ideas; the amount and
cognitive level of student participation in class; and the amount of peer interaction in a
course.
Newmann (1990) has linked teacher behaviors to opportunities for students to be
active at higher cognitive levels. Newmann proposed that for students to exercise higher16
order thinking, teachers can model higher-order thinking during classroom discourse.
Additionally, Newmann recommended that teachers allow students to question authority
and pose challenging questions, practice problem solving using experiences, and
carefully analyze conclusions.
Cognition Research in Agricultural Education
Edwards (2004) conducted a review of literature of cognitive learning in
Agricultural Education and implied that the impact of Agricultural Education on student
achievement must be identified to secure future support. The researcher conducting the
current study categorizes past research in Agricultural Education into two categories:
Level of cognition in classrooms, and how levels of cognition have been linked to teacher
behaviors or student retention scores. The review of literature cited in this study is
intended to inform the methods used, and to provide rationale for continuing the line of
inquiry in student cognition.
Levels of Cognitive Behaviors in Agriculture Classrooms
In a study of 11 high school horticulture teachers, Cano and Metzger (1995)
sought to describe the cognitive levels of instruction used in horticulture classes. It was
found that “teachers taught 84 percent of the time at the lower levels (knowledge,
translation, interpretation, application) of cognition” (p. 40). The Florida Taxonomy of
Cognitive Behavior (Webb, 1970) was used to classify cognitive behaviors of the
teachers. Cano and Metzger (1995) stated that the FTCB is a derivative of Bloom‟s
17
Taxonomy (1956), and “can be considered valid in identifying behaviors at various levels
of cognition” (p. 37). Cano and Metzger also reported that students of agriculture tend to
have higher percentage scores at the higher levels of cognition than students in science
and other disciplines, which support the findings of several studies in Agricultural
Education in the 1990s (Newcomb & Trefz, 1987; Cano & Martinez, 1991; Pickford,
1988; Rollings, Miller & Kayler, 1988).
Torres and Cano (1995) emphasized the importance of developing higher-order
thinking in students, beginning in the freshmen year of college. In addition, the
researchers advocated that teachers should teach at higher cognitive levels, forcing
students to do more than simply restate learned facts (Torres & Cano, 1995). Torres and
Cano (1995) also stated that tests and assignments should be written at higher levels of
cognition.
Ewing and Whittington (2007) studied the variables that influenced student
cognition during college class sessions. Ewing and Whittington, in a study of 21 total
class sessions with 12 different instructors in an agricultural college, described the
professor discourse, teaching techniques, cognitive level of questions asked by the
professor, student engagement in class, and cognitive level of student questions during
class sessions. Ewing and Whittington used researcher-developed instruments to describe
the variables during class sessions and used think-aloud protocols to describe the
thoughts of the students.
Furthermore, 62 percent of student thoughts and questions pertaining to class
content occurred at Bloom‟s (Bloom et al., 1956) knowledge and comprehension levels
18
of cognition (Ewing & Whittington, 2007). Approximately 9 percent of student thoughts
occurred at Bloom‟s application level, 17 percent at analysis, 5 percent at synthesis, and 6
percent at evaluation (Ewing & Whittington). Sixty percent of student thoughts in general
were unrelated to the class content (Ewing & Whittington). Therefore, Ewing and
Whittington recommended that professors analyze the cognitive levels of student
thoughts to ensure the course is at the most appropriate cognitive level, and that if the
student thoughts were at lower-cognitive levels, professors should consider changing
course objectives, delivery, and discourse to enhance student cognition.
In a descriptive case study by Falk, Batts, and Whittington (2010), it was found
that, during the 18 days of a secondary animal science unit of instruction, the teacher in
the study taught at the two lowest levels of cognition 46.93 percent of the time. In this
descriptive study, the researcher sought to describe instructor behaviors and student
retention of classroom content. Specifically, the purpose the of the study was to describe
the instructor‟s discourse, attitude, and aspiration toward teaching at higher cognitive
levels during an animal science unit of instruction, and to describe student immediate,
short-term, and long-term cognitive retention of class session content. The teacher in this
study aspired to teach across all levels of Bloom‟s Taxonomy, and wanted to write
assignments that challenged students at the higher levels of cognition, the highest
percentage of time. It was also found that the teacher had a positive attitude toward
teaching at higher cognitive levels. Concerning student retention, the researcher had the
teacher administer a unit test immediately after the unit was taught, 42 days after the unit,
and 182 days after the unit. It was found that students were able to retain the majority of
19
the content assessed by the final unit test immediately following instruction, yet the final
unit test was written at the lowest levels of cognition.
Using think-aloud protocols, in a study of 16 professors and 64 students,
Whittington, Lopez, Schley, and Fisher (2000), found that teachers “were generally
teaching at lower cognitive levels” (p. 621). Additionally, students were primarily having
random, nonsense thoughts during lectures. The researchers asserted that professors need
to become aware of student cognition to select the correct teaching methods for their
students, impacting the level of performance (Whittington, Lopez, Schley & Fisher).
Ulmer (2007) concluded that agriculture and science teachers were not exhibiting
different cognitive behavior. Ulmer‟s study was designed to describe differences, if they
existed, between teachers of biology and high school agriculture teachers, in teacher
attitude toward teaching at higher levels of cognition, and level of cognitive behavior
displayed. A sample of 18 teachers (9 science, and 9 agricultural education teachers)
were the subjects of the study, who were observed using the FTCB. It was found that
“Both agriculture and science teachers exhibit lower-order (knowledge and
comprehension) teaching behaviors the vast majority of the time (83 percent and 84
percent, respectively)” (p. 113). Ulmer (2007) questioned the value of providing both
biology and agricultural courses if agriculture teachers do not utilize the opportunities
within a complete agricultural education program to increase higher-order thinking.
20
Relationships between Teaching Techniques and Content Retention
In a study of the effectiveness of simulation in an agricultural mechanics class, no
significant difference was found between simulation and the real-life, conventional
teaching of mechanical concepts (Agnew & Shinn, 1990). Agnew and Shinn (1990)
conducted an experimental study of 230 students and 11 teachers by assessing the
students with the Science Research Associate‟s Test of Mechanical Concepts after five
school days of instruction in agricultural mechanics. The questions on the instrument
were identified as being low cognitive level or high cognitive level questions; no
significant differences were reported between scores. It is important to note that low level
questions were identified as knowledge, comprehension or application on Bloom‟s
Taxonomy, while high cognitive level questions were at the analysis, synthesis, or
evaluation levels.
Boone (1990), in a study of 99 high school agriculture students with six
vocational agriculture teachers, sought to determine the relationship between the use of
the problem solving approach and student achievement and retention of agricultural
knowledge. In the study, the researchers used a variation of the non-equivalent control
group design where the teachers taught two units of instruction and measured student
achievement with a forty question achievement test at three intervals. Boone concluded
that the problem solving approach to teaching increased the level of student retention of
agricultural knowledge, yet also stated that student prior knowledge and teacher
characteristics impacted student achievement.
21
In contrast, Flowers and Osborne (1987), in a study of 126 high school vocational
agriculture students and 20 agriculture teachers, found that “the problem solving
approach is no more or less effective than the subject matter approach in producing
higher scores on the delayed retention test, regardless of the cognitive level of questions”
(p.26). Dyer and Osborne (1999) shared similar conclusions; retention scores of students
were not impacted by the problem solving approach. In a similar study, Boone and
Newcomb (1990) found that teachers did not fully teach using the problem solving
approach, but rather blended teaching techniques. The Boone and Newcomb (1990) study
held inconclusive evidence of a relationship between teaching techniques and student
retention scores.
Another study in Agricultural Education was designed to describe the effect of
writing-to-learn activities on student retention of subject matter, as opposed to traditional
learning activities. In their study, Reaves, Flowers, and Jewell (1993) conducted a quasiexperimental study with 199 students of agriculture and 13 teachers. They found that,
“Students taught by writing-to-learn activities appeared to have lower scores on the initial
achievement test, but higher scores on the retention test administered three weeks later
than did students taught by lecture and discussion methods” (p. 37).
In a quasi-experimental study of 352 high school students enrolled in introductory
Agricultural Science courses, Myers and Dyer (2006) concluded that there is a concern
with the great amount of time spent teaching a unit of instruction that results in little
student knowledge gain. Using pre-test and post-test instruments, Myers and Dyer
concluded that students with less prior knowledge had higher content knowledge gain
22
scores at the conclusion of the instruction. In addition, students with higher science
processing skill achievement prior to the instruction, had higher content knowledge gain
at the conclusion. Myers and Dyer recommended that further research be conducted to
find the effect of teaching methods on student attitude and long-term and short-term
content knowledge retention.
Falk, Beck, and Whittington (2009) conducted a descriptive case study, with a
high school agriculture teacher, and found that the teacher taught 40 percent of the time
using lecture, and 33 percent of the time using discussion. Falk et al., postulated that,
“Students may have retained the content better if more diverse teaching techniques were
used” (p. 56), but caution must be exercised since the case study (n=1) lacked the
necessary variability to measure relationships between teaching techniques and student
content retention.
In another descriptive study, Ball and Garton (2005) used a convenience sample
of seven teacher educators at a university to study the levels of cognition modeled in the
course objectives, classroom discourse, and assessments in pre-service teacher
preparation courses, and to examine the alignment among the cognitive levels of course
objectives, the cognitive levels of classroom discourse, and the cognitive levels of
assessment activities. It was discovered that the teacher educators in the study created
course objectives at the application and synthesis levels of Bloom‟s Taxonomy 76
percent of the time, which are considered, by many cognitive researchers, to be higher
levels of cognition. The teacher educators modeled lower levels of thinking in classroom
discourse 61 percent of the time, which is consistent with the findings of Whittington
23
(1995) and Miller (1989), where teachers conducted classroom discourse most often at
the knowledge and comprehension levels of cognition.
The results of the Ball and Garton (2005) study imply that teacher educators in the
study were not modeling the use of higher levels of Bloom‟s Taxonomy to the teacher
candidates in their courses. Eraut (1997) suggested that teachers teach the way they were
taught, which may indicate that the teacher educators are teaching their students to
conduct classroom discourse at lower levels of cognition (Ball & Garton). The
researchers recommended that “Further research should be conducted regarding the
relationship between the cognitive levels of teaching and student performance at different
levels of cognition” (Ball & Garton, 2005, p. 67).
Time Spent on Instruction and between Instruction
Summer break has been criticized for erasing much of the educational progress
gained during the school year (Klemm, 2007), noting that rehearsal is important for
activating memories and moving them into long-term storage. Eliminating the summer
vacation could increase student memory and retention, yet in a study of 18 middle school
classrooms in Maryland, Karweit and Slavin (1981) found that extending the hours in
school without first improving quality of teaching and time-on-task, produced
disappointing results for student achievement.
Through research regarding time spent on teaching and learning, it was suggested
that attention to individual differences, as well as attention to quality of instruction, effect
student achievement more than increasing the quantity of time (Hite, 2001). Hite (2001)
24
speculated that, for most students, the more time spent on instruction, the greater the
achievement.
A meta-analysis of time spent on teaching and learning, conducted by Cooper,
Nye, Charlton, Lindsay, and Greathouse (1996), indicated that the overall decrease in
student achievement after summer vacation, masked dramatic differences in the effect of
summer vacation on different skill areas. Notably, the results indicated that summer loss
was more dramatic for math-related subject areas than for reading or language.
Educational researchers have suggested that children‟s home and community
environments can provide more opportunity to practice reading skills and to learn new
words than to practice and learn mathematics (Murname, 1975; Cooper, Nye, Charlton,
Lindsay, & Greathouse, 1996).
Conceptual Framework
Given an extensive review of the related literature, the following conceptual
framework is provided to guide this and future cognition studies. Piaget‟s theory of
learning emphasizes that 1) knowing is grounded in activity, 2) development occurs
gradually and progressively to make sense of the environment, and 3) learning occurs
when an individual acts to resolve conflicts between existing beliefs and new data which
does not fit the existing beliefs (Jordan, 1993). Bruner, (1965) believed that learning is
more likely to be successful when the learner can connect to the material, which supports
Piaget‟s work. Since Agricultural Education is grounded in application-based,
experiential learning, it fits well within Piaget‟s Theory of Cognitive Development.
25
Piaget (1970) theorized that teachers have little impact on the maturation
influence, but that teachers can provide exploration, observation, testing, and information
organization, all of which are likely to alter thinking processes, thereby effecting Piaget‟s
activity influence. In addition, teachers can impact the social transmission influence,
learning from others, depending on the stage of cognitive development the student has
reached (Whittington, Foster, Falk, Beck & Bookman, 2008).
There exists, in Agricultural Education, an on-going line of inquiry to study
instructor, student, and learning environment variables that explain student cognition
during class sessions (Whittington et al., 2008). Studying patterns that exist between
teacher classroom discourse and student content retention adds to the knowledge base of
this line of inquiry. In Figure 3, an illustration is presented of the conceptual framework
that guided this study.
Figure 3: Conceptual Framework for Studying Cognitive Levels of Teaching and
Learning
26
Summary
“Two of the most important educational goals are to promote retention and to
promote transfer (which, when it occurs, indicates meaningful learning),” (Mayer, 2002,
p. 226). To continue exploring how teaching at various levels of cognition impacts
student retention of content, this study was designed to further describe the
recommendations of Falk, Batts, and Whittington (2009), stating, “A larger study should
be conducted to collect information from more subjects, in more classrooms, in more
communities” (p.11).
27
Chapter 3: Methods
The purpose of this research was twofold: to describe the cognitive level of
teacher discourse in two secondary agricultural education classrooms, and to describe
student retention of content from both the perspective of cognitive level of content
delivery and cognitive level of content assessment. The design of this study was
descriptive, and examined teacher discourse at each level of Bloom‟s Taxonomy and the
proportion of classroom discourse delivered at higher and lower levels of cognition. In
addition, student retention of content was assessed immediately after delivery of a fiveday unit of instruction, and again following a short-term interval (49 days after delivery
of instruction). The research subjects included two secondary agricultural science
teachers who were teaching a total of 42 students.
Research Questions
1. What cognitive level of discourse do teachers exhibit in secondary agricultural
science classrooms?
2. At what rate are students retaining content in secondary agricultural science
classrooms, at the immediate and short-term intervals?
3. What patterns exist between teacher cognitive discourse and student retention of
content in secondary agricultural science classrooms?
28
Subjects
Two secondary agricultural science teachers in Ohio participated in the study. The
subjects were selected by an Associate Professor at The Ohio State University who had
previously taught each of the subjects in the Methods of Teaching Agriculture course (see
Correspondence in Appendix D).
The researcher allowed subjects to volunteer to participate because volunteering
could have potentially necessitated changing the order in which he or she delivered the
agricultural science curriculum. The subjects were also selected based upon their
willingness to work with the researcher. No incentives were provided for teachers to
participate in this study. The researcher acknowledges that this sample of convenience
cannot be generalized beyond the participants in the study, and that readers should
consider this limitation when interpreting the results.
The teacher of the Agricultural Products Unit of Instruction was selected because
of this teacher‟s master‟s thesis. The teacher studied the levels of Bloom‟s Taxonomy and
saw a need for variability of cognitive levels in the classroom. The researcher felt that
this teacher was very well prepared in the area of instructional design and discourse
across levels of Bloom‟s Taxonomy. The Agricultural Products teacher had fewer than
three years teaching experience.
The teacher of the Agricultural Sales Unit of Instruction was selected because of
this teacher‟s typical understanding of Bloom‟s Taxonomy compared to other recent
agricultural science graduates. While this teacher was considered by the researcher to be
an effective teacher, the researcher felt that this teacher did not consider Bloom‟s
29
Taxonomy when developing course instruction and teacher discourse and, therefore, was
more typical among agricultural science teachers as a whole. The Agricultural Sales
teacher had fewer than three years teaching experience.
Procedures
Five-day units of instruction were taught, and video-recorded, in two high school
agricultural science classrooms for this study. The researcher selected two secondary
agricultural science teachers to participate in the study, and allowed each teacher to
develop his/her own units and assessments for this research. Units were teacherdeveloped to provide a real description of what is being taught in agricultural science
classrooms. Units were taught the week of November 15-19, 2010.
An Agricultural Products Unit of Instruction was developed and taught at a
secondary school in Northwestern Ohio. The unit was written for a class of 27 9th grade
students. Goals of the unit were written to address student interaction with raw and
processed agricultural products, animal products, and plant products. The unit was
written by the teacher, who was purposefully selected due to his/her background with
research on Bloom‟s Taxonomy and the Florida Taxonomy of Cognitive Behavior.
The Agricultural Sales Unit of Instruction was developed and taught at a school in
Northeastern Ohio. The unit was written for a class of 13 agricultural science students.
Goals of the unit addressed qualities of good sales persons, determining client needs and
wants, techniques used in horticultural sales, and effective customer service. The unit was
30
written by the teacher, who was purposefully selected due to his/her traditional
background in teaching methods.
For each unit, a test was created across all levels of Bloom‟s Taxonomy by the
teacher of the unit and then edited specifically by the researcher for the ability of the
instrument to test across the levels of Bloom‟s Taxonomy. The tests were used three
times: As a pre-test before the unit was taught, as post1 immediately after the unit was
taught, and as post2 shortly (49 days) after the unit was taught.
The respective teachers collected the tests from students, scored them for use in
their classes, and returned the tests to the researcher. The tests were then re-scored by the
researcher to calculate scores at each level of cognition. The researcher also collected the
video-recordings for each classroom, which were used to evaluate teacher discourse using
the Florida Taxonomy of Cognitive Behavior (FTCB).
Forty-nine days later, on January 7, 2011, the teachers gave the test a third time
(post2) to their students. Students were not given advance notice of post2. January 7th was
selected because it was at the end of a full week when students returned from winter
break. The break allowed an opportunity for students to not receive related instruction
from the teacher. The teachers agreed that high school students would likely be in
attendance for class, and that the 49-day interval was acceptable for testing retention of
content. During a study by Falk, Beck, and Whittington (2009), the intermediate retention
of student content knowledge was measured at 42 days. In this study, 42 days would have
been during a break from school, therefore, a week was added to the short-term retention
timeframe.
31
The third test was graded by each teacher and sent to the researcher to be rescored at each level of cognition. Scores were evaluated for a change in correct and
incorrect answers. Mean values for each teacher‟s class were calculated at each level of
cognition.
It should be noted that the teachers provided written feedback on post1.
Comments from the teachers were used to clarify why a student got an answer incorrect,
as well as confirm correct answers. It appeared that each teacher provided comments on
the questions that were frequently answered incorrectly. All constructive comments
related to the content of the units that were taught.
Instrumentation – Unit Tests
Miller (1989) found that teachers often wrote tests at lower levels of cognition,
but that course assignments typically reached the higher levels of cognition. Miller
suggested that teachers and students placed more value on test scores rather than on
course assignments, which, he posited, led to emphasizing low-level thinking.
To ensure cognitive variability across the levels of Bloom‟s Taxonomy in the
tests, the researcher edited each test that was created by the teachers (see Unit Tests in
Appendix A and B). The tests were revised to represent higher cognitive and lower
cognitive levels of Bloom‟s Taxonomy. A current secondary agricultural education
instructor and an Associate Professor in Agricultural Education at The Ohio State
University, both of whom had conducted research involving Bloom‟s Taxonomy, were
used to validate the levels of Bloom‟s Taxonomy, at which each test question was
32
written. When there was disagreement between the researcher and the experts on the level
of Bloom‟s Taxonomy, consensus was reached through discussion.
Validity of Agricultural Products Test
Validity was established for each test. The agricultural products test was deemed
valid using a panel of experts, including a faculty member of Agricultural and Extension
Education at The Ohio State University, a current secondary agriculture teacher, and four
graduate students in Agricultural and Extension Education at The Ohio State University.
Content validity was established using the panel of experts by asking them to determine if
each question was written with appropriate content for freshmen in a high school
agricultural science classroom. One question was identified as being redundant and was
removed. Face validity was also established using the panel of experts to assess the tests
for their ability to test secondary agricultural science students regarding the content that
was taught. The panel suggested that one question was confusing to 9th graders; therefore
the question was reworded, according to suggestions from a high school teacher, to
clarify the question and to more appropriately assess content from the unit of instruction.
Reliability of Agricultural Products Test
Reliability was assessed using a test-retest procedure using a comparison group
during a pilot test. The comparison group was a class of nine 9th graders enrolled in a
secondary agricultural science classroom. A minimum reliability coefficient was set a
priori for each test item at .75. This coefficient indicated that the best estimate that 75
33
percent of observed variance in a test item is true variance, and 25 percent is error
variance (Ary, Jacobs & Razavieh, 1985). The reliability coefficient was set at .75 to
account for variance in a classroom size of nine students, allowing for two students to
record a different response between tests. A higher coefficient could have been chosen a
priori, but the researcher chose to allow for student variability in pre-determined
classrooms.
Data Collection
The Agricultural Products Test was given on the first day of a five-day unit of
instruction as a pre-test for students. Post1 test was given on the final day of a five-day
unit on agricultural products. The teacher administered the test under his/her typical class
operations, and students were to do their own work. Completed tests were collected by
the researcher. Post2 was given 49 days after delivery of the unit test as a measure of
short-term retention on January 7th. This date was selected because it included a winter
break from school, yet also allowed for students to be back in a school routine for a week
before the re-test. Having a stable routine, instead of an irregular occurrence, was
important for this study in order to describe the students on a typical week. Students were
not told that the three tests were the same test each time, and were not forewarned prior to
the administration of the post2 test.
34
Data Analysis
Questions on the Agricultural Products Test were classified by their level of
cognition; six questions at the knowledge level, two questions at the comprehension
level; zero questions at the application level, two questions at the analysis level, two
questions at the synthesis level, and two questions at the evaluation level. Specifically,
question numbers 1, 3, 6, 8, 12, and 13 were written at the knowledge level. Question
numbers 2 and 10 were written at the comprehension level. Question numbers 4 and 5
were written at the analysis level. Question numbers 11 and 14 were written at the
synthesis level. Question numbers 7 and 9 were written at the evaluation level.
An answer key was provided to the teachers to score their tests and the teachers recorded
the score as part of their course grade. The tests were re-scored for cognition by the
researcher to remove discrepancies between scoring of each of the teachers. To receive
full credit for cognition for a question, the student had to correctly answer the question.
In Table 2, the classification of each question on the Agricultural Products Test is
displayed.
Level of cognition
Numbers of test questions
Knowledge level questions
1, 3, 6, 8, 12, 13
Comprehension level questions 2, 10
Application level questions
n/a
Analysis level questions
4, 5
Synthesis level questions
11, 14
Evaluation level questions
7, 9
Table 2: Cognitive Level of Questions on an Agricultural Products Test
35
The second test was scored identically as the first test. If a student answered a
question correctly on post1, and also correctly on post2, then the student received a
retention score of 2.0. Students, who answered correctly on post1, but incorrectly on
post2, received a retention score of -1.0. Students, who incorrectly answered a question
on both tests, received a retention score of 0.0. Finally, a student who did not answer the
question correctly on post1, but got the question correct on post2, received a score of 1.0.
Scores were averaged for each class, and a mean retention score was assigned to each
teacher.
Validity of Agricultural Sales Test
Validity was established for each test. The Agricultural Sales Test was deemed
valid using a panel of experts, including a faculty member in Agricultural and Extension
Education at The Ohio State University, a current high school agriculture teacher, and
four graduate students in Agricultural and Extension Education at The Ohio State
University. Content validity was established using the panel of experts and asking them
to determine if each question was appropriate for freshmen in a high school agricultural
education classroom. No changes were made to the test after review from the panel of
experts. Face validity was also established using the panel of experts, resulting in all
questions being assessed for their ability to measure what a secondary student in
agricultural science should be able to answer following an agricultural sales unit of
instruction.
36
Reliability of Agricultural Sales Test
Reliability was established for the Agricultural Sales Test differently than the
Agricultural Products Test. There was neither sufficient time allowed, nor a consenting
comparison group to perform test-retest reliability for the Agricultural Sales Test before it
was administered to students. Reliability for homogeneity of items on the test was
determined using the Kuder-Richardson 21 formula, and was calculated as 0.366. Ary,
Jacobs, and Razavieh (1985) noted that low reliability coefficients may be acceptable for
research in cognitive domains if the results are not used to make irreversible decisions for
the students.
Data Collection
The Agricultural Products Sales was given on the first day of a five-day unit of
instruction as a pre-test for students. Post1 test was given on the final day of a five-day
unit on agricultural products. The teacher administered the test under his/her typical class
operations, and students were to do their own work. Completed tests were collected by
the researcher. Post2 was given 49 days after delivery of the unit test as a measure of
short-term retention on January 7th. This date was selected because it included a winter
break from school, yet also allowed for students to be back in a school routine for a week
before the re-test. Having a stable routine, instead of an irregular occurrence, was
important for this study in order to describe the students on a typical week. Students were
not told that the three tests were the same test each time, and were not forewarned prior to
the administration of the post2 test.
37
Data Analysis
Questions on the Agricultural Sales Test were classified by their level of
cognition; nine questions at the knowledge level, three questions at the comprehension
level; zero questions at the application level, three questions at the analysis level, two
questions at the synthesis level, and two questions at the evaluation level. Specifically,
question numbers 1, 2, 3, 4, 5, 6, 12, 13 and 14 were written at the knowledge level.
Question numbers 15, 16 and 17 were written at the comprehension level. Question
numbers 7, 9 and 11 were written at the analysis level. Question numbers 8 and 18 were
written at the synthesis level. Question numbers 10 and 19 were written at the evaluation
level.
An answer key was provided to teachers to score their students and the teachers
recorded that score as part of the course grade for the students. The tests were re-scored
for cognition by the researcher to remove discrepancies between scoring of each of the
teachers. To receive full credit for cognition for a question, the student had to correctly
answer the question. In Table 3, it can be seen how each test question was classified for
the Agricultural Sales Test.
Level of cognition
Numbers of test questions
Knowledge level questions
1, 2, 3, 4, 5, 6, 12, 13, 14
Comprehension level questions 15, 16, ,17
Application level questions
n/a
Analysis level questions
7, 9, 11
Synthesis level questions
8, 18
Evaluation level questions
10, 19
Table 3: Classification of Agricultural Sales Test Questions
38
The second test was scored identically as the first test. If a student answered a
question correctly on post1, and also correctly on post2, then the student received a
retention score of 2.0. Students, who answered correctly on post1, but incorrectly on
post2, received a retention score of -1.0. Students, who incorrectly answered a question
on both tests, received a retention score of 0.0. Finally, a student who did not answer the
question correctly on post1, but got the question correct on post2, received a score of 1.0.
Scores were averaged for each class, and a mean retention score was computed for the
class.
Instrumentation - FTCB
Webb (1970) used Bloom‟s Taxonomy to create the Florida Taxonomy of
Cognitive Behavior (FTCB) to assess the cognitive level of discourse in classrooms
(Appendix C). The FTCB utilizes 55 observable behaviors, potentially present in a
classroom, that are indicators of cognitive levels of processing. There are 17 observable
behaviors in the knowledge level, 12 behaviors listed for the comprehension level, the
application level has four behaviors listed, analysis has 11 behaviors indicated, synthesis
has nine associated behaviors, and evaluation has two observable behaviors listed on the
FTCB (Whittington, Lopez, Schley & Fisher, 2000). The FTCB was used to identify and
quantify cognitive behaviors observed during teacher discourse in the classroom. As can
be seen in Figure 4, a comparison is provided for describing the levels of cognition used
by Bloom‟s Taxonomy, the FTCB, and the Newcomb-Trefz Model (Ewing, 2006).
39
Figure 4: A comparison of Bloom's Taxonomy, the Florida Taxonomy of Cognitive
Behavior, and the Newcomb-Trefz Model
Validity
According to Whittington, Lopez, Schley, and Fisher (2000), validity for the
FTCB is “based upon its direct development from Bloom‟s Taxonomy and the support
generally given to this hierarchy of cognitive behaviors” (p. 617).
Reliability
Inter-rater and intra-rater reliability for the FTCB, in this study, was established
by coding video-taped classroom observations between an expert in using the FTCB, a
research expert, and the researcher. The videos watched came from a collection of 18
video-taped, secondary classroom sessions available to the researcher. A Pearson
reliability coefficient was set a priori of r = .90 for inter-rater and intra-rater reliabilities.
The researcher re-assessed a random selection of a video, one week after the initial
scoring, which resulted in an intra-rater reliability of r = .93. Inter-rater reliability was
40
established between the FTCB expert and the researcher by each watching the same
video and scoring the FTCB separately. The inter-rater reliability was calculated by
dividing the number of FTCB observations of the researcher by the number of FTCB
observations from the FTCB expert, which equaled r = .95.
Data Collection
Both teachers in this study video-recorded each day of instruction. The videos
were collected by the researcher and were analyzed by the researcher using the FTCB.
Verbal discourse across cognitive levels was noted at six-minute intervals for each class
session. Frequency counts were calculated for each cognitive level, as well as a percent
for verbal discourse at each level of cognition for each day and for the entire unit of
instruction.
Data Analysis
Frequencies of observed behaviors were totaled for each level of cognition, during
each class session, yielding a subtotal of teaching behaviors at each cognitive level for
each class session. Then, subtotals for each cognitive level were summed for each class
session for each teacher. The sum for frequencies observed at each level of cognition
over the entire unit was divided by the total number of class sessions observed, resulting
in a percentage of discourse for the teacher at each level of cognition.
41
Summary
In this study of teacher classroom discourse, and student content retention, two
secondary agricultural science teachers were video-recorded as they each taught a fiveday unit of instruction of their choosing. For cognitive level of discourse, the Florida
Taxonomy of Cognitive Behavior was used while observing the video-taped sessions, to
capture frequencies of teacher behaviors at various cognitive levels.
Acquiring data for student content retention required the use of teacher-created,
researcher-edited, unit tests. Students took the test as a pre-test, post-test (post1) the final
day of instruction and as a post-post test (post2) 49 days following instruction. Gain
scores were calculated to describe student short-term content retention.
Teacher classroom discourse proportions and student retention scores were
examined. Potential patterns that might exist were related to frequencies of behaviors and
amount of time spent teaching at various cognitive levels, and frequency of content
retained at various levels of cognition.
42
Chapter 4: Findings
Presented in this chapter are the findings of the study, which was designed to
answer the following research questions guiding this descriptive study:
Research Questions
1. What cognitive level of discourse do teachers exhibit in secondary agricultural
science classrooms?
2. At what rate are students retaining content in secondary agricultural science
classrooms, at the immediate and short-term intervals?
3. What patterns exist between teacher cognitive discourse and student retention of
content in secondary agricultural science classrooms?
Describing the Cognitive Level of Discourse
The cognitive level of discourse for two teachers, over the five-day unit of
instruction, was assessed using the Florida Taxonomy of Cognitive Behavior (FTCB).
Results from the FTCB for the Agricultural Products Unit of Instruction are reported in
Table 4, and the results from the FTCB for the Agricultural Sales Unit of Instruction are
reported in Table 5.
43
Class
session
Total
Knowledge
Comprehension Application
Analysis
Synthesis
Evaluation frequencies
%
f
%
f
%
f
%
f
%
f
%
f
f
1
65.22
15
17.39
4
4.35
1
8.7
2
4.35
1
0
0
23
2
21.92
16
32.88
24
9.59
7
23.29
17
12.33
9
0
0
73
3
40.32
25
32.26
20
1.61
1
16.13
10
6.45
4
3.23
2
62
4
35.06
27
23.38
18
3.9
3
23.38
18
11.69
9
2.6
2
77
5
41.38
12
27.59
8
6.9
2
17.24
5
6.9
2
0
0
29
Average
40.78
19
26.7
14.8
5.27
2.8
17.75 10.4
8.34
5
1.17 0.8
52.80
St.Dev.
15.70 6.60
6.47
8.43
3.06 2.49
6.07
7.09
3.49
3.81 1.61 1.10
25.16
Table 4: Agricultural Products Unit, percent cognitive level of teacher discourse as measured using the Florida Taxonomy of
Cognitive Behavior by class session
44
44
Class
session
1
2
3
4
5
Average
St.Dev.
Knowledge Comprehension
%
f
%
f
56.00
14
28.00
7
42.42
28
28.79
19
40.74
22
35.19
19
30.51
18
30.51
18
77.78
7
22.22
2
49.49 17.8 28.94
13
18.23 7.95 4.68
7.97
Application
%
f
0.00
0
6.06
4
9.26
5
6.78
4
0.00
0
4.42 2.6
4.21 2.41
Analysis
%
f
8.00
2
15.15
10
11.11
6
16.95
10
0.00
0
10.24 5.6
6.70 4.56
Total
Synthesis
Evaluation frequencies
%
f
%
f
f
4.00
1
0.00
0
25
4.55
3
0.00
0
66
3.70
2
0.00
0
54
11.86
7
3.39
2
59
0.00
0
0.00
0
9
4.82
2.6 0.68 0.4
42.60
4.32
2.70 1.52 0.89
24.41
Table 5: Agricultural Sales Unit, percent cognitive level of teacher discourse as measured using the Florida Taxonomy of
Cognitive Behavior by class session
45
45
During the Agricultural Products Unit of Instruction, 67.5 percent of the teacher
discourse was at the two lowest levels of cognition (knowledge and comprehension) over
the five days of instruction. Teacher discourse was at the two highest levels of cognition
(synthesis and evaluation) an average of 9.5 percent of the time. However, the evaluation
level of cognition was not evidenced using the FTCB on days one, two, or five. In
addition, on the opening day of the unit, 82 percent of the teacher discourse was delivered
at Bloom‟s two lowest levels of cognition.
The teacher of the Agricultural Sales Unit of Instruction delivered discourse at the
two lowest levels of cognition (knowledge and comprehension) 78.4 percent of the time
over the five days of instruction. The discourse for this instructor was at the two highest
levels of cognition (synthesis and evaluation) an average of 5.5 percent of the time.
However, the evaluation level was not reached on the first, second, fourth, or fifth days of
teaching the unit. In addition, on the opening day of the unit, 84 percent of the teacher
discourse was delivered at Bloom‟s two lowest levels of cognition.
Describing Student Retention of Content in an
Agricultural Products Unit of Instruction
The Agricultural Products Unit of Instruction was administered to 28 students in a
secondary agricultural education program. Students completed a pre-test on day one,
completed the same test as post1 on day five, and completed the same test again after 49
days. The test was teacher-created and was written with 14 questions, totaling a possible
46
71 points. Within Table 6 is included the results of the Agricultural Sales pre-test, post1,
and post2.
The average proportion of content retained by the class from post1 to post2 was 96
percent. It is important to note that the standard deviation from the average proportion of
content retained was 9.5 points. Twenty-eight student scores were used to calculate the
average for the class; one student did not take post2 at the same time as the other students.
47
Pre-Test
Student
#
Score
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
26
27
28
29
St. Dev.
Mean
Post1
Post2
%
Score
%
39
56
37
30
46
35
28
26
42
30
55%
79%
52%
42%
65%
49%
39%
37%
59%
42%
29
50
34
54
28
50
47
37
21
45
39
49
43
38
62
48
42
10.11
40.19
41%
70%
48%
76%
39%
70%
66%
52%
30%
63%
55%
69%
61%
54%
87%
68%
59%
14%
57%
51
59
52
54
59
57
48
53
60
48
58
58
64
57
61
50
63
64
54
41
21
61
52
52
48
65
59
58
8.76
54.54
72%
83%
73%
76%
83%
80%
68%
75%
85%
68%
82%
82%
90%
80%
86%
70%
89%
90%
76%
58%
30%
86%
73%
73%
68%
92%
83%
82%
12%
77%
Score
Gain or loss
from post1
to post2
Percent
retention
from post1
to post2
0
-6
-1
100%
90%
98%
+2
-6
-14
-9
-10
-1
-10
-9
-2
-3
-11
+1
-5
-11
-4
-9
+36
+8
+5
-7
+3
+1
-4
+4
9.50
-2.28
103%
90%
71%
83%
83%
98%
83%
85%
97%
95%
82%
102%
92%
83%
93%
78%
271%
113%
110%
87%
106%
102%
93%
107%
36%
96%
%
51 72%
53 75%
51 72%
61
51
34
44
50
47
48
49
62
54
50
51
58
53
50
32
57
69
57
45
51
66
55
62
8.25
52.26
86%
72%
48%
62%
70%
66%
68%
69%
87%
76%
70%
72%
82%
75%
70%
45%
80%
97%
80%
63%
72%
93%
77%
87%
12%
74%
Table 6: Agricultural Products Unit Test scores
Note. There were a total of 71 points possible on the test.
48
The average pre-test score for the Agricultural Products Unit of Instruction was
57 percent. The average post1 score was 77 percent. The average post2 score was 74
percent. Student #4 did not complete the post2, and student #11 did not complete the pretest. Additionally, student #25 was officially enrolled in the course during the time of the
tests, but was not attending class during the data collection. The overall change in score
from post1 to post2 was a loss of 2.28 points. The range for the change in score was from
a gain of 36 points to a loss of 14 points for the students enrolled in the course.
Retention of content was measured by comparing student scores on their post1 to
their post2 score. A retention score of 2.00 for a question, meant that the answer was
correct on the post1, and was also correctly answered on post2. A retention score of -1.00
meant that the question was answered correctly on the post1, but was answered
incorrectly on the post2. The average retention score for the knowledge level questions
was 1.49, indicating that much of the content was retained by the students. The average
retention score for comprehension level questions was 1.37, indicating that students
retained less of the content compared to knowledge level questions. The average
retention score for analysis level questions was 1.20, which indicates that students
retained less synthesis level content, compared to knowledge and comprehension
questions. The average retention score for synthesis level questions was 0.81, indicating a
general loss of content retention for synthesis level questions. The average retention score
for evaluation level questions was 1.17, indicating that evaluation level content was
retained less often than lower cognitive levels. The cognitive levels of questions, and
49
their average retention scores, are presented in Table 7, as well as the test questions on
the test that corresponded to each cognitive level.
Cognitive level
Knowledge level questions
Comprehension level questions
Application level questions
Analysis level questions
Synthesis level questions
Evaluation level questions
Average student retention score
1.49
1.37
n/a
1.20
0.81
1.17
Test numbers of
questions
1, 3, 6, 8, 12, 13
2, 10
n/a
4, 5
11, 14
7, 9
Table 7: Retention scores by cognitive level of questions on the Agricultural Products
Test
Describing Student Retention of Content in an Agricultural Sales Unit of Instruction
The Agricultural Sales Unit of Instruction was administered to 13 students in a
secondary agricultural education program. Students completed a pre-test on day one,
completed the same test as post1 on day five, and completed the same test again after 49
days. The test was teacher-created and was written with 19 questions, totaling a possible
44 points. Within Table 8 is included the results of the Agricultural Sales pre-test, post1,
and post2.
The average proportion of content retained by the class from post1 to post2 was 87
percent. It is important to note that the standard deviation from the average proportion of
content retained was 6.43 points. Thirteen student scores were used to calculate the
average for the class.
50
Pre-Test
Post1
Post2
Gain or loss
from post1 to
post2
Student
#
Score
%
Score
%
Score
%
1
2
3
4
5
6
7
8
9
10
11
12
13
St. Dev.
Mean
33
40
25
31
42
42
31
42
15
39
32
30
35
7.82
33.62
75%
91%
57%
70%
95%
95%
70%
95%
34%
89%
73%
68%
80%
18%
72%
38
44
37
38
44
44
40
44
15
41
42
40
44
7.75
39.31
86%
100%
84%
86%
100%
100%
91%
100%
34%
93%
95%
91%
100%
18%
84%
36
41
20
20
44
43
37
42
13
43
30
35
42
10.41
34.31
82%
93%
45%
45%
100%
98%
84%
95%
30%
98%
68%
80%
95%
24%
74%
-2
-3
-17
-18
0
-1
-3
-2
-2
+2
-12
-5
-2
6.43
-5
Percent
retention from
post1 to post2
95%
93%
54%
53%
100%
98%
93%
95%
87%
105%
71%
88%
95%
17%
87%
Table 8: Agricultural Sales Unit test scores
Note. There were a total of 44 possible points on the test.
The average pre-test score for the Agricultural Sales Unit of Instruction was 72
percent. The average post1 score was 84 percent. The average post2 score was 74 percent.
Student #9 was absent on days two and three of this unit of instruction. Student #13 was
absent on day four of the unit, but this student made-up the missed content on day five,
and took the post1 during a lunch period. The average overall change in score from post1
to post2 was a loss of 5.00 points. The range for the change in score was from a gain of 2
points to a loss of 18 points for the students enrolled in the course.
51
Retention of content was measured by comparing students‟ scores on their post1
to their post2. A retention score of 2.00 for a question meant that the answer was correct
on post1, and also correctly answered on post2. A retention score of -1.00 meant that the
question was also answered correctly on the post1, but was answered incorrectly on the
post2. The average retention score for the knowledge level questions was 1.50, indicating
that much of the knowledge-level content was retained after 49 days. The average
retention score for comprehension level questions was 1.64, indicating that students
retained much of the comprehension level content. The average retention score for
analysis level questions was 1.38, indicating that students retained less analysis-type
questions compared to knowledge and comprehension content. The average retention
score for synthesis level questions was 1.38, indicating that students retained less
synthesis-type questions compared to knowledge and comprehension content. The
average retention score for evaluation level questions was 1.23, indicating that the least
amount of content was retained at the evaluation level. The cognitive level of questions
and their average retention scores are reflected in Table 9, along with the corresponding
question numbers that represent each level of cognition.
52
Cognitive level
Knowledge level questions
Comprehension level questions
Application level questions
Analysis level questions
Synthesis level questions
Evaluation level questions
Average student retention score
1.50
1.64
n/a
1.38
1.38
1.23
Test numbers of
questions
1, 2, 3, 4, 5, 6, 12, 13, 14
15, 16, 17
n/a
7, 9, 11
8, 18
10, 19
Table 9: Retention scores by cognitive level of questions on the Agricultural Sales Unit
Test
Patterns between Teacher Cognitive Discourse and
Student Content Retention
A pattern emerged in each teacher‟s discourse, and the students‟ content retention
scores. Both teachers taught a majority of their units at the lower levels of cognition,
knowledge and comprehension, over a five-day unit of instruction. Short-term retention
rates were measured using post2, 49 days after post1. Lower cognitive level questions on
the tests were retained at higher frequencies than higher cognitive level questions. The
pattern reflects that the content most likely to be retained by students, is content asked on
the test at the same cognitive level in which it was taught during the unit of instruction
(see Table 10).
53
Lower cognitive levels
Higher cognitive levels
Agricultural
Products Unit
teacher
discourse
68%
32%
Agricultural
Products Unit
retention scores
1.43
1.06
Agricultural
Sales Unit
teacher
discourse
79%
21%
Agricultural
Sales Unit
retention scores
1.57
1.33
Table 10: Patterns between teacher cognitive discourse and student content retention
Note. Lower cognitive levels included: Knowledge and Comprehension. Higher cognitive
levels included: Application, Analysis, Synthesis, and Evaluation.
The teacher of the Agricultural Products Unit of Instruction delivered content 68
percent of the time at the knowledge level and comprehension levels, and students earned
an average retention score of 1.43 at the knowledge and comprehension levels. The
teacher of the Agricultural Products Unit of Instruction taught at the higher cognitive
levels (application, analysis, synthesis, and evaluation) levels 32 percent of the time, and
students earned an average retention score of 1.06 at these higher cognitive levels.
The teacher of the Agricultural Sales Unit of Instruction delivered content 79
percent of the time at the knowledge level and comprehension levels, and students earned
an average retention score of 1.57 at the knowledge and comprehension levels. The
teacher of the Agricultural Sales Unit of Instruction taught at the higher cognitive levels
(application, analysis, synthesis, and evaluation) levels 21 percent of the time, and
students earned an average retention score of 1.33 at these higher cognitive levels.
Summary
Teacher discourse was measured using the FTCB (Webb, 1970). Both teachers in
this study delivered classroom content at the lowest levels of Bloom‟s Taxonomy a
54
majority of the observed class time. In addition, related to student retention of content,
students retained content between post1 and post2. Some patterns began to emerge
between the teacher and student variables: Students appeared to retain content at the
various cognitive levels, in proportion to the cognitive levels at which the teachers
delivered the content. Also, it appears that the amount of time a teacher spends at the
various cognitive levels influences the students‟ rate of retention of content.
55
Chapter 5: Conclusions
Executive Summary
Previous researchers have recommended that teachers create learning situations
that teach students at higher levels of cognition (Cano & Newcomb, 1990). In addition,
Myers and Dyer (2006) recommended that further research be conducted on the effect of
teaching methods toward student attitude and long-term and short-term content
knowledge retention.
The purpose of this research was twofold: to describe the cognitive level of
teacher discourse in two secondary agricultural education classrooms, and to describe
student retention of content from both the perspective of cognitive level of content
delivery and cognitive level of content assessment. The design of this study was
descriptive, and examined teacher discourse at each level of Bloom‟s Taxonomy and the
proportion of classroom discourse delivered at higher and lower levels of cognition. In
addition, student retention of content was assessed immediately after delivery of a fiveday unit of instruction, and again following a short-term interval (49 days after delivery
of instruction). The research subjects included two secondary agricultural science
teachers who were teaching a total of 42 students. The specific research questions guiding
this study included:
56
Research Questions
1.
2.
What cognitive level of discourse do teachers exhibit in secondary agricultural
science classrooms?
At what rate are students retaining content in secondary agricultural science
classrooms, at the immediate and short-term intervals?
What patterns exist between teacher cognitive discourse and student retention
of content in secondary agricultural science classrooms?
3.
To describe the cognitive level of teacher discourse in secondary agricultural
education classrooms, the researcher purposefully selected and observed two secondary
agricultural science teachers during five-day units of instruction. One subject of this
study was familiar with Bloom‟s Taxonomy in his/her teaching, while the other teacher
was selected to represent a typical, early-career agriculture teacher. Classroom
instruction was not dictated by the researcher; rather, that which was observed, was that
which would have naturally been taught. Each class session, for each teacher, was videorecorded and later analyzed using the Florida Taxonomy of Cognitive Behavior for
teacher discourse. Frequency of teacher discourse during each class session was
documented across various levels of cognition, and then proportions were reported at
each level of cognition.
In order to describe at what rate students retained content in secondary
agricultural science classrooms, the students enrolled in agriculture courses completed a
pre-test before the units were taught; post1 was completed immediately following the
unit; and post2 was administered 49 days following the completion of the unit. The
teacher-developed tests were edited by the researcher to ensure variability across levels of
Bloom‟s Taxonomy.
57
Summary of Findings
Two purposefully-selected, secondary agricultural science teachers taught a fiveday unit of instruction in this study. During the Agricultural Products Unit of Instruction,
67.5 percent of the teacher‟s discourse was delivered at the two lowest levels of
cognition. Similarly, during the Agricultural Sales Unit of Instruction, 78.4 percent of the
teacher‟s discourse was delivered at the two lowest levels of cognition.
For both units of instruction, content was retained by students at the greatest
frequency for questions that were asked at the lowest levels of cognition. Rates at which
content was retained decreased at each higher level of Bloom‟s Taxonomy. A decrease in
the rate of retention indicated that students retained more content at the lowest levels of
cognition.
A pattern was observed between teacher cognitive discourse and student content
retention. The data were used to reveal that both teachers in the study taught the majority
of their respective units of instruction at lower levels of cognition, and that lower
cognitive level content was retained at higher frequencies. A pattern began to emerge in
which students retained the same cognitive level of questions on the post1 and post2 as the
cognitive level of discourse that was delivered by the teacher. Content, therefore, was
being retained at the same levels at which the content was taught.
Conclusions Related to Cognitive Levels of Discourse
It was concluded that the teacher of the Agricultural Products Unit of Instruction
taught a secondary agricultural science class at the lowest levels of cognition. This
58
teacher was selected to participate in this study due to the expectation of the researcher
that the teacher would teach across all levels of Bloom‟s Taxonomy.
It was concluded that the teacher of the Agricultural Sales Unit of Instruction also
taught a secondary agricultural science class at the lowest levels of cognition. This
teacher was selected to participate in this study because s/he was likely a good
representative of many of the early career agriculture teachers in Ohio. The teacher who
was selected to be purposeful about his/her use of cognitive levels in the classroom had
comparable results to a teacher who was not aware of his/her classroom discourse
reaching varying levels of cognition
Discussion Related to Cognitive Levels of Discourse
The researcher based this study on the premise that Bloom‟s Taxonomy
emphasizes the importance of offering lower level information to students as a base on
which to build to the higher levels of cognition. The researcher suspected that secondary
agricultural science educators may be delivering the greater proportion of content at
lower levels of cognition. Although this study had a sample size of two teachers, the
findings of this study are consistent with previous studies (Pickford, 1988; Miller, 1989;
Whittington & Newcomb, 1993; Cano & Metzger, 1995). The teacher who was selected
to be purposeful about his/her use of cognitive levels in the classroom, had comparable
results to the teacher who was not aware of his/her classroom discourse reaching varying
levels of cognition, which is troubling to the researcher because the subjects were
59
expected to create variability in the levels of cognitive discourse that were used to deliver
content.
Understanding why a teacher did not evidence variability of discourse across
levels of Bloom‟s Taxonomy will be important in future research. A subject in this
research was purposefully selected due to his/her awareness of Bloom‟s Taxonomy and
how to engage students across all levels through teacher discourse. Several possible
reasons occurred to the researcher through observation, including: the teacher wanted to
maintain control of the classroom; the teacher was anxious to progress through content
efficiently; and the teacher was answering student questions to be helpful to students.
Maintaining control of a classroom, for an early career teacher, was a way for the
teacher to manage student behavior, resulting in fewer opportunities for students to act
unexpectedly. The researcher felt that many of the educational activities presented were
teacher-directed. Teacher discourse that incorporated more autonomous, student activities
could have given students an opportunity to reach higher cognitive levels, including
synthesis and evaluation. Efficient use of class time is important to teachers, and each
teacher in this study was efficient, possibly at the cost of having allowed students to
create higher cognitive level questions. A third possible reason for the teacher to exhibit
lower cognitive level discourse, appeared to be that the teachers would answer student
questions immediately, often before students committed higher cognitive level thinking
to the question. Teachers could have increased the cognitive level of discourse by asking
student-developed questions to the rest of the students enrolled in the class. The three
60
reasons listed above could have been the most immediate concerns of an early-career
teacher, rather than purposefully engaging in discourse at higher cognitive levels.
Why teachers continue to teach at lower cognitive levels is not known, even
though teacher educators over the past 25 years have created an awareness of cognitive
levels of teaching and learning in the agricultural education profession. Early career
educators may possibly need more practice developing and delivering lessons that
challenge students, despite the preparation the teachers had in their university
coursework.
Cognitive Distribution
There is a lack of research that has prescribed a distribution of discourse across
cognitive levels during a class session. Teaching methods, such as inquiry-based
instruction, problem-solving, or didactic instruction are likely to offer different
proportions of discourse at each cognitive level. Additionally, the timing and order of
delivery of discourse to a group of students is likely to vary. For example, in the problemsolving approach, students may be presented with higher cognitive level discourse early
in the unit of instruction, and finish the unit of instruction with a greater proportion of
lower cognitive level discourse. It is possible, that a more appropriate way to think about
distribution of cognitive level of discourse is, rather than across each class session, across
each unit of instruction, across each area of study, or across each department‟s total
course offerings. The point is, one must ask the question, “When are learners receiving
61
opportunities to engage in thinking at higher cognitive levels?”, and then to be purposeful
about planning for addressing the question.
Recommendations and Implications Related to Cognitive Levels of Discourse
A recommendation regarding discourse, is for practicing teachers to further
develop intentional opportunities for students to learn across levels of cognition.
Professional development opportunities should be made available through universities,
state departments of education, and local school districts. Darling-Hammond (2006)
advocated for clinical experiences for teachers to become familiar with research theory,
while being firmly rooted in practical application.
Conclusions Related to Student Content Retention
The Agricultural Products Unit Test was written across five of the six levels of
cognition, omitting the application level of cognition. Students were able to retain a
majority of the content from the five-day unit of instruction. Average student scores for
the Agricultural Products Pre-test indicated that students were very unfamiliar with the
content that was to be taught. The post-test scores were 77 percent and 74 percent.
Therefore, it is concluded that students retained content that was taught during the
Agricultural Products Unit of Instruction.
The Agricultural Sales Unit of Instruction was also written across five of the six
levels of cognition, omitting the application level of cognition. Average student scores on
the Agricultural Sales Pre-test indicated that students were moderately familiar with the
62
content that was about to be taught. Students earned 84 percent of the points immediately
following the teaching of the unit of instruction (post1), and then earned 74 percent of the
points on post2 49 days later. The researcher concluded that the average student retained
nearly the same amount of content across a short period of time.
Discussion Related to Student Content Retention
After a five-day unit of instruction, students were expected to learn content, as
measured by a written test. Students who retained the content after seven weeks would
have been more likely to apply what they learned to new topics taught in class, as well as
transfer that knowledge to real-world decisions. The researcher observed students
interacting with their environment, listening to the instructor, and visually experiencing
many events in the classroom, but many of these activities were teacher-directed and
permitted students to only commit to doing what the teacher said.
The researcher was alarmed to compare the average pre-test score (72 percent) to
the post2 score (74 percent) on the Agricultural Sales Unit Test. One might question the
value of teaching the Agricultural Sales Unit, as was taught to the 13 specific students in
the study, when the test score after 53 days was similar to before it was ever taught.
Additionally, in this same unit, the standard deviations (7.82 points on the pre-test; 7.75
points on post1; 10.41 points on post2) indicated that there was a wide range of student
content knowledge. It is possible, that not all of the students in the study were motivated
to perform well on the post2.
63
Student content retention may have been influenced by threats to external validity,
specifically including interactions with a history effect. The history effect could have
existed, meaning that students could have had opportunities to engage with the content of
the lesson outside of the five-day units of instruction. The history interaction could have
influenced student retention scores. Teachers could have caused a history effect by
continually referring to the content that had been taught over the seven weeks between
post1 and post2. Students may have interacted with similar content in their lives, outside
of the classroom; the amount of a history effect, or lack of a history effect, could have
influenced how strongly a student retained the content taught during a five-day unit of
instruction.
Recommendations and Implications Related to Student Content Retention
Student assessments should be carefully constructed to measure the desired
outcomes of the curriculum. Test-creators must consider assessment across all levels of
Bloom‟s Taxonomy, only after the have teachers taught content across all levels of
Bloom‟s Taxonomy. If teachers teach at all levels of cognition, then students will have
increased opportunity to retain the content across all levels, potentially increasing student
achievement. Only after students have been exposed to the content across all levels,
should evaluations assess the content across all levels.
A second recommendation is for teachers to clearly state the goals for a unit of
instruction at the beginning of the unit, and then to check student progress along those
goals during the unit. If students knew that they were capable of performing the goals of
64
the unit, they would have likely performed even better on the tests. Finally, future studies
should consider student motivation for multiple testing scenarios.
Conclusions Related to Patterns between Cognitive Discourse and Student Content
Retention
The following pattern was concluded to have emerged: When more content was
taught at lower levels of cognition, lower-level questions were answered correctly at
higher frequency. The secondary Agricultural Science instructors in this study were
teaching at the lowest levels of Bloom‟s Taxonomy, therefore, it is concluded that
students retained content in proportion to the cognitive levels in which it was delivered
by the teacher.
Discussion Related to Patterns between Cognitive Discourse and Student Content
Retention
Teachers with more than three years of teaching experience were not included in
this study, but each teacher in this research commented that they would teach their unit
differently in the future. Experienced teachers may be reaching levels of cognition
through teacher discourse at a different frequency than early career teachers. Levels of
teacher experience could impact a teacher‟s comfort toward managing the classroom
environment, a teacher‟s sense of comfort toward meeting learning objectives, and a
teacher‟s comfort toward allowing student questions to remain unanswered by the
teacher.
65
Recommendations and Implications Related to Patterns between Cognitive Discourse and
Student Content Retention
Teacher educators should provide opportunities for college students to practice
teaching across all levels of Bloom‟s Taxonomy, and should assess students across all
cognitive levels. Practice would afford opportunities for pre-service teachers to modify
classroom discourse and then, to assess content knowledge across the cognitive levels at
which the teacher taught the content.
If teacher educators would increase preparation of pre-service teachers to teach
and assess across all levels of cognition, then pre-service teacher candidates could benefit
from practicing spent on delivering teaching activities across the levels of cognition.
Consequently, the pattern that could potentially exist could be one in which more
discourse was delivered at higher cognitive levels; therefore more higher cognitive level
questions could be answered correctly. Consequently, more content could be retained for
longer periods of time, affording opportunities to transfer the learning to other scenarios
and environments.
A second recommendation was that teachers of agriculture should teach across all
levels of cognition during a unit of instruction. A pattern emerged in the data, whereby
when levels of teacher cognitive discourse were low, retention of higher level content
was also low. Since students are expected to perform at higher levels of cognition (Kuhn,
1999; Van Gelder, 2005), it is recommended that teachers teach across all levels of
cognition, including the higher levels of analysis, synthesis, and evaluation.
66
Based on the pattern that emerged between teacher cognitive discourse and
student retention of content, if teachers teach across all cognitive levels, students will
have more opportunities to experience content at all levels, which could possibly lead to
longer-term content retention. Specifically, students might have the potential to retain
information at the evaluation level, when teachers provide opportunities for students to
learn at the evaluation level. The goal, ultimately, is for students to be more successful at
transferring knowledge learned in the classroom environment to other parts of their lives
and careers, when they have been given the opportunity to engage in the content at higher
cognitive levels.
Further Discussion on Student Content Retention and Cognitive Studies
Both research subjects in this study delivered classroom content at similar
cognitive levels of discourse. Additionally, the teachers had similar retention scores, even
though the teachers were different, the content was different, and students were different.
In order to better understand relationships between teacher discourse and student content
retention, future research should include a greater number of subjects. A study with an
appropriate number of teachers as subjects in treatment groups, randomly selected from
the population, would allow results to be generalized to the population. Further,
researchers should design common assessments and units of instruction to control for
more variables. Common assessments could be designed to incorporate questions across
all levels of Bloom‟s Taxonomy. The advantage of teaching common units of instruction
67
across all subjects is that variables could be isolated, including the proportions of teacher
discourse at various levels of cognition.
Assessment of Content Retention
Other forms of communication, besides discourse, as well as different modes of
assessment, may add to the body of literature for measuring student retention of content.
Out-of-class assignments, student projects, and student presentations are examples of
common learning activities for teachers to engage students at higher levels of cognition
(application, analysis, synthesis, and evaluation). A goal of this research was not to
describe teaching methods employed in the classroom, but to observe teachers using
lecture and teacher-led questioning as learning opportunities for students. The teachers
may have selected these teaching methods, which largely gave the teacher control,
because of a five-day timeframe for observation. Assessing cognitive levels of other
forms of classroom communication, besides only teacher discourse, could help to
describe what is present in secondary agricultural science classrooms.
Assessing student mastery of content is often limited to paper tests, likely due to
the relative ease in grading. However, students might actually have earned higher
retention scores, had they learned the content in class across all levels of cognition, and
then been given a real-life problem as the post2 to solve. Student projects, where they
create, perform, analyze and evaluate, could more accurately describe what a student is
able to know and do.
68
Additional Factors that Contribute to Cognitive Gains
The researcher observed that student learning was often disrupted during the unit
of instruction due to a student being tardy, leaving early, or not attending class. The
burden of having a student be prepared equally with other students was put on the
teacher, due to the teacher assuming that all students needed to be equally prepared.
Online education possesses great opportunity for students to make-up what was missed,
as do other individualized teaching techniques. During this study, many group teaching
techniques were witnessed by the researcher, including lecture, discussion, and skits.
When a student missed class, they often missed the opportunity to learn at the same depth
as when it was taught in the group setting. Teachers could consider effective teaching
strategies, and development of learning communities, in on-line course components, to
limit disruption of learning.
Affective variables should be considered, not just cognitive variables, when
considering classroom climate and variables that affect student learning. The videorecorded lessons that were observed and analyzed by the researcher revealed a noticeable
difference in the number and kind of praises offered during a class session. Many
affirmations were given in one class, while few were offered in the other classroom
environment. The Florida Taxonomy of Cognitive Behavior was used in this study, which
only measured the frequency of discourse at various cognitive levels. Questions were
raised in the mind of the researcher regarding the effect of the affective domain on
student retention of content. For example, how do different students, in different
classrooms, feel compelled or motivated to achieve? How can a teacher offer the
69
optimum amount of affective feedback to students to achieve positive results? Student
motivation, which effects achievement, should be considered in future research.
Summary
While teachers have been aware of Bloom‟s Taxonomy (1956) for over 50 years,
some teachers need to purposefully use the different levels in the classroom for student
learning. Knowledge and comprehension were the most-used levels of Bloom‟s
Taxonomy in this study, but all levels are important in student cognitive development.
Teachers should focus on building upon the levels of cognition to better teach students,
and then assess the students at all levels of Bloom‟s Taxonomy.
Ewing & Whittington (2007) recommended that teacher behaviors in Agricultural
Education be developed that increase capacity to teach purposefully at higher cognitive
levels, with the goal of enhancing student learning. Teachers should focus on teaching
and assessing students at varying levels of cognition to match the preparation and desired
outcome for the students.
70
References
Agnew, D. M., & Shinn, G. C. (1990). Effects of simulation on cognitive achievement in
agriculture mechanics. Journal of Agricultural Education, Summer 1990, p. 1216.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and
assessing: A revision of Bloom‟s taxonomy of educational objectives. New York:
Addison Wesley Longman, Inc.
Ary, D., Jacobs, L. C., & Razavieh, A. (1985). Introduction to research in education (3rd
ed.). New York: CBS College Publishing.
Ball, A. L., & Garton, B. L. (2005). Modeling higher order thinking: The alignment
between objectives, classroom discourse, and assessments. Journal of
Agricultural Education, 46(2), 58-69.
Beck, W. (2009). Describing time spent using various teaching techniques and student
immediate, short-term, and long-term cognitive retention. Unpublished master‟s
thesis, The Ohio State University.
Bloom, B. S. (1964). Taxonomy of educational objectives: Handbook 1, cognitive domain
(2nd ed.). New York, NY: Longman.
Bloom, B. S. (Ed.), Engelhart, M. D, Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956).
Taxonomy of educational objectives handbook 1: Cognitive domain. Longman,
NY: Longman.
Boone, H. N. (1990). Effect of level of problem solving approach to teaching on student
achievement and retention. Journal of Agricultural Education, Spring 1990, p. 1826.
Boone, H. N., & Newcomb, L. H. (1990). Effects of approach to teaching on student
achievement, retention, and attitude. Journal of Agricultural Education, Winter
1990, p. 9-14.
Bruner, J. (1965). Growth of mind. American Psychologist, 20, 1007-1017.
71
Cano, J. M. & Marinez, C. (1991). The relationship between cognitive performance and
critical thinking abilities among selected agricultural education students. Journal
of Agricultural Education, Spring 1991, p. 24-29.
Cano, J. M., & Metzger, S. (1995). The relationship between learning style and levels of
cognition of instruction of horticulture teachers. Journal of Agricultural
Education, 36(2), 36-43.
Cooper, H., Nye, B., Charlton, K., Lindsay, J. & Greathouse, S. (1996). The effects of
summer vacation on achievement test scores: A narrative and meta-analytic
review. Review of Educational Research, Fall 1996, 66(3), pp. 227-268. doi:
10.3102/00346543066003227
Darling-Hammond, L. (2006). Powerful teacher education: Lessons from exemplary
programs. San Francisco: Jossey-Bass.
Dyer, J. E., & Osborne, E. W. (1999). Effects of student learning styles on short and long
term retention of subject matter using various teaching approaches. Journal of
Agricultural Education, 40(2), p. 11-18.
Edwards, M. C. (2004). Cognitive learning, student achievement, and instructional
approach in secondary agricultural education: A review of literature with
implications for future research. Journal of Vocational Education Research,
29(3), p. 225-244.
Edwards, M. C., & Ramsey, J. W. (2004). Cognitive learning, student achievement, and
instructional approach in secondary agricultural education: A review of literature
with implications for future research. Proceedings of the 31st Annual National
Agricultural Education Research Conference, St. Louis, 158-169.
Eraut, M. (1997). Developing professional knowledge and competence. Washington,
D.C.: The Falmer Press.
Ewing, J. C. (2006). Teaching techniques, and cognitive level of discourse, questions,
and course objectives, and their relationship to student cognition in college of
agriculture class sessions. Unpublished dissertation, The Ohio State University,
Columbus, OH.
Ewing, J. C. & Whittington, M. S. (2007). Describing the cognitive level of professor
discourse and cognitive level of student engagement in college of agriculture class
sessions. Journal of Agricultural Education, 48(3), 91-99.
72
Falk, J. M., Batts, A, N., & Whittington, M. S. (2010). Describing the cognitive level of
discourse of a secondary teacher during an animal science unit of instruction.
Poster Proceedings of the Annual National Agricultural Education Research
Conference. Omaha, NE.
Falk, J. M., Beck, W. M., & Whittington, M. S. (2009). Describing student cognitive
retention in an animal science unit. NACTA 2009 Abstracts and Awards, p. 43-44.
Farr, M. J. (1987). The long-term retention of knowledge and skills: A cognitive and
instructional perspective. Springer-Verlag, New York, NY.
Foster, D. D. (2009). Instructor variables, student variables, and class session
environment variables: Describing their relationship to student cognition during
class sessions. Unpublished dissertation, The Ohio State University, Columbus,
OH.
Finkel, D. L. (2000). Teaching with your mouth shut. Portsmouth, N.H.: Boynton/Cook
Publishers.
Flowers, J., & Osborne, E. W. (1987). The problem solving and subject matter
approaches to teaching vocational agriculture: Effects on student achievement and
retention. Journal of Agricultural Education, p. 20-26.
Gardner, H. (1999). The disciplined mind. New York: Simon & Schuster.
Gardiner, L. F. (1998). Why we must change: The research evidence. The NEA Higher
Education Journal, 71-88.
Halpern, D. F. (1984). Thought and knowledge. Hillsdale, NJ: Lawrence Erlbaum
Associates.
Halpern, D. F., & Hakel, M. D. (2003). Applying the science of learning to the university
and beyond: Teaching for long-term retention and transfer. Change, 35(4), 36-41.
Hite, W. R. (2001). Extending the school year: Student achievement, attendance, and
student, teacher, and parent satisfaction. Unpublished dissertation, Virginia
Polytechnic Institute and State University, Blacksburg, VA.
Jordan, L. E. (1993). Human development theories and their applicability to the middle
school program: A position paper. (ERIC Document Reproduction Service No.
ED 380 395)
Karweit, N. & Slavin, R. E. (1981). Measurement and modeling choices in studies of
time and learning. American Educational Research Journal, 18(2), 157-171.
73
Klemm, W. R. (2007). What good is learning if you don‟t remember it? The Journal of
Effective Teaching, 7(1), p. 61-73.
Kline, P. (2002). Why America’s children can’t think. Maui, HI: Inner Ocean Publishing,
Inc.
Kolers, P. A., & Roediger, H. L. (1984). Procedures of mind. Journal of Verbal
Learning and Verbal Behavior 23, p. 425–449.
Krathwohl, D. R. (2002). A revision of Bloom‟s taxonomy: An overview. Theory into
Practice, 41(4), pp. 212-218.
Kuhn, D. (1999). A developmental model of critical thinking. Educational Research,
28(2), 16-25.
Mangurian, L., Feldman, S., Clements, J., & Boucher, L. (2001). Analyzing and
communicating scientific information. Journal of College Science Teaching,
30(7), 440-445.
Mayer, R. E (2002). Rote versus meaningful learning. Theory into Practice, 41(4), p.
226-232.
Miller, C. (1989). Cognitive levels of instruction and student performance in college of
agriculture courses. Unpublished doctoral dissertation, The Ohio State
University, Columbus.
Murname, R. (1975). The impact of school resources on inner-city school children.
Cambridge, MA: Ballenger.
Myers, B. E. & Dyer, J. E. (2006). Effects of investigative laboratory instruction on
content knowledge and science process skill achievement across learning styles.
Journal of Agricultural Education, 47(4), 52-63.
National Council for Agricultural Education (2000). The national strategic plan and
action agenda for agricultural education: Reinventing agricultural education for
the year 2020. National Council for Agricultural Education, Indianapolis, IN.
Newcomb, L. H., & Trefz, M. K. (1987). Levels of cognition of student tests and
assignments in the College of Agriculture at The Ohio State University.
Proceedings of the National Agricultural Education Research Meeting, Las
Vegas, NV.
74
Newmann, F. M. (1990). Higher-order thinking in teaching social studies: A rationale for
the assessment of classroom thoughtfulness. Journal of Curriculum Studies, 22,
41-56. doi: 10.1080/0022027900220103
Nunn, C. E. (1996). Discussion in the college classroom. Journal of Higher Education,
67, p. 243-266.
Piaget, J. (1970) Piaget‟s Theory. In P. Mussen, (Ed), Handbook of Child Psychology,
(3rd Ed.), (Vol. 1, p. 703-732). New York: Wiley.
Pickford, J. C. (1988). Selected student and professor variables related to cognitive
achievement in college of agriculture courses. Unpublished master‟s thesis, The
Ohio State University, Columbus, OH.
Reaves, R. R., Flowers, J. L., & Jewell, L. R. (1993). Effects of writing-to-learn activities
on the content knowledge, retention, and attitudes of secondary vocational
agriculture students. Journal of Agricultural Education, Fall 1993, p. 34-40.
Robinson, K. (2010, February). Sir Ken Robinson: Bring on the learning revolution.
Retrieved from http://www.ted.com/talks/sir_ken_robinson_bring_on_the_
revolution.html
Rollins, T. J., Miller, W. W., & Kayler, A. A. (1988). Critical thinking skills of
agriculture students. Proceedings of the 1988 National Agriculture Education
Research Meeting, St. Louis, MO.
Smith, D. G. (1977). College classroom interactions and critical thinking. Journal of
Educational Psychology, 69, p. 180-190.
Torres, R. M., & Cano, J. M. (1995). Examining cognition levels of students enrolled in a
college of agriculture. Journal of Agricultural Education, 36(1), 46-54.
Tsui, L. (2002). Fostering critical thinking through effective pedagogy. The Journal of
Higher Education, 73(6), p. 740-763.
Ulmer, J. D. (2007). A comparison of the cognitive behaviors exhibited by secondary
agriculture and science teachers. Journal of Agricultural Education, 48(4), p. 106116.
Van Gelder, T. (2005). Teaching critical thinking: some lessons from cognitive
science, College Teaching, 45, p. 1–6.
75
Webb, J. N. (1970). The Florida taxonomy of cognitive behavior, In A. Simon and E.G.
Boyer, (Eds.), Mirrors for behavior: An anthology of classroom observation
instruments, 1(6), Philadelphia: Research for Better Schools.
Whittington, M. S. (1995). Higher order thinking opportunities provided by professors in
college of agriculture classrooms. Journal of Agricultural Education, 36(4), 3238.
Whittington, M. S. (2003). Improving the cognitive capacity of students by fully
engaging professors in the teaching and learning process, Unpublished
Manuscript. The Ohio State University.
Whittington, M. S., Foster, D. D., Falk, J. M., Beck, W. M., Bookman, J. A. (2008).
Creating a conceptual framework for studying cognitive levels of teaching and
learning. Proceedings of the North American Colleges and Teachers of
Agriculture Meeting, Logan, UT.
Whittington, M. S., Lopez, J., Schley, E., & Fisher, K. (2000). Using think-aloud
protocols to compare cognitive levels of students and professors in college
classrooms. Proceedings of the 27th Annual National Agricultural Education
Research Conference, San Diego, CA.
Whittington, M. S., & Newcomb, L. H. (1993). Relationship of attitude toward teaching
at higher cognitive levels to aspired and assessed cognitive level of instruction.
Journal of Agricultural Education, 34(2), 55-62.
76
Appendix A: Unit of Instruction – Agricultural Production
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Appendix B: Unit of Instruction – Agricultural Sales
101
102
103
104
105
106
107
Appendix C: Florida Taxonomy of Cognitive Behavior
108
1
2 3 4
5 6
7 8 9 10
1.1 Knowledge of specifics
1.
Reads
2.
Spells
3.
Identifies something by name
4.
Defines meaning of term
5.
Gives a specific fact
6.
Tells about an event
1.2
Knowledge of ways and means of dealing with specifics
7.
Recognizes symbol
8.
Cites a rule
9.
Gives chronological sequence
10.
Gives steps of process, describes method
11.
Cites trend
12.
Names classification system or standard
13.
Names what fits given system or standard
1.3 Knowledge of universals and abstracts
14.
States generalized concept or idea
15.
States a principle, law, theory
16.
Tells about organization or structure
17.
Recalls name of principle, law, theory
2.0 Translation
18.
Restate in own words or briefer terms
19.
Gives concrete examples of an abstract idea
20.
Verbalizes from a graphic representation
21.
Translates verbalization into graphic form
22.
Translates figurative statements into literal statements or
vice versa
23.
Translates foreign language to English or vice versa
109
1
2
3 4 5
6
7
8 9 10
3.0 Interpretation
24.
Gives reason (tells why)
25.
Shows similarities, differences
26.
Summarizes or concludes from observation of evidence
27.
Shows cause and effect relationship
28. Gives analogy, simile, metaphor
29.
Performs a directed task or process
4.0
Application
30.
Applies previous learning to new situations
31.
Applies principle to new situation
32.
Applies abstract knowledge in a practical situation
33.
Identifies, selects and carries out process
5.0
Analysis
34.
Distinguishes fact from opinion
35.
Distinguishes fact from hypothesis
36.
Distinguishes conclusion from statements which support it
37.
Points out unstated assumption
38.
Shows interaction or relation of elements
39.
Points out particulars to justify conclusions
40.
Checks hypotheses with given information
41.
Distinguishes relevant from irrelevant statements
42.
Detects error in thinking
43.
Infers purpose, point of view, thoughts, feelings
44.
Recognizes bias or propaganda
110
6.0
Synthesis (Creativity)
45.
Reorganizes ideas, materials, processes
46.
Produces unique communication, divergent idea
47.
Produces a plan, proposed set of operations
48.
Designs an apparatus
49.
Designs a structure
50.
Devises a scheme for classifying information
51.
Formulates hypotheses, intelligent guesses
52.
Makes deductions from abstract symbols, propositions
53.
Draws inductive generalization from specifics
7.0
Evaluation
54.
Evaluates something from evidence
55.
Evaluates something from criteria
The Florida Taxonomy of Cognitive Behavior (Webb, 1970)
111
Appendix D: Correspondence
112
May 31, 2011
Anthony Wayne High School
5967 Finzel Rd.
Whitehouse, OH 43571
Request for participation in educational research study
Dear Mrs. Jeri Hoellrich:
In an attempt to contribute to the body of knowledge in cognitive research in the
classroom, a research study will be conducted to describe teacher behaviors in an
agriculture classroom and the resulting student content retention. It is the desire of the
researchers at The Ohio State University to include Mrs. Whitney Short as a participant
in this research.
If permitted to participate, Mrs. Short will be asked to allow us to video record her as she
teaches a unit of instruction, of her choosing, for a minimum of five days. The video
recorder will be positioned in the classroom to focus on the teacher, not the students. The
videos will be analyzed for teacher behaviors. At the end of the unit, Mrs. Short will
assess the students using an examination that is written to test their content knowledge
across the levels of Bloom‟s Taxonomy. Copies of the examinations, without student
identification, will be collected by the researchers. Again, during this research, the names
of students will not be recorded. Any student work that is collected (quizzes, worksheets,
tests, etc.) will be coded with a number to remove student names. During this study, Mrs.
Short will communicate with the researchers at Ohio State, share lesson plans and
teaching documents, and video record the class sessions. Anthony Wayne High School
will not need to furnish any equipment for this study. The five consecutive days of
observation will be selected by Mrs. Short.
The results of this study will be used to describe current teaching behaviors and how they
are related to student retention of content.
If consent for Mrs. Short to participate in this research study is granted, please express
this interest by responding to me via email. I can be reached at [email protected]. Further
questions and comments should also be directed to me.
Respectfully,
Jeremy M. Falk
PhD Candidate, Graduate Teaching Associate
Agricultural and Extension Education
113
May 31, 2011
Jackson High School
7600 Fulton Drive NW
Massillon, Ohio 44644
Request for participation in educational research study
Dear Mrs. Cindy Glass
In an attempt to contribute to the body of knowledge in cognitive research in the
classroom, a research study will be conducted to describe teacher behaviors in an
agriculture classroom and the resulting student content retention. It is the desire of the
researchers at The Ohio State University to include Mr. Ryan McMichael as a participant
in this research.
If permitted to participate, Mr. McMichael will be asked to allow us to video record him
as he teaches a unit of instruction, of his choosing, for a minimum of five days. The
video recorder will be positioned in the classroom to focus on the teacher, not the
students. The videos will be analyzed for teacher behaviors. At the end of the unit, Mr.
McMichael will assess the students using an examination that is written to test their
content knowledge across the levels of Bloom‟s Taxonomy. Copies of the examinations,
without student identification, will be collected by the researchers. Again, during this
research, the names of students will not be recorded. Any student work that is collected
(quizzes, worksheets, tests, etc.) will be coded with a number to remove student names.
During this study, Mr. McMichael will communicate with the researchers at Ohio State,
share lesson plans and teaching documents, and video record the class sessions. Jackson
High School will not need to furnish any equipment for this study. The five consecutive
days of observation will be selected by Mr. McMichael.
The results of this study will be used to describe current teaching behaviors and how they
are related to student retention of content.
If consent for Mr. McMichael to participate in this research study is granted, please
express this interest by responding to me via email. I can be reached at [email protected].
Further questions and comments should also be directed to me.
Respectfully,
Jeremy M. Falk
PhD Candidate, Graduate Teaching Associate
Agricultural and Extension Education
114