4/23/13 Will this be on the test? Characterizing the cognitive levels of undergraduate biology Jennifer Momsen, North Dakota State University Undergraduate Biology Classes Facts Facts More facts Why? 1 4/23/13 Student expectations of biology 1. 2. What is the nature of the knowledge I am learning? What should I do to learn in biology? Will this be on the test? Will this be on the test? Will this be on the test? Hall et al 2011 Course expectations of students 1. Learning goals and outcomes Students will… define, describe, explain, identify, label, etc 2. Nature of assessments Students will… memorize terms, processes, examples bring a number 2 pencil 3. Classroom activity Students will… take copious, accurate notes (and maybe complete the crossword) 2 4/23/13 Characterizing assessments Concept and content knowledge o o Vision and Change Core Concepts Map to a textbook Skills Cognitive skills using Bloom’s taxonomy, Webb’s depth of knowledge o Vision and Change Core disciplinary practices o Bloom’s taxonomy, cognitive domain o o o Identifies 6* levels of cognitive skills Nested or hierarchical Higher levels require more complex cognitive processes Synthesis & Evaluation Analysis Application Comprehension Knowledge *Note: Synthesis and Evaluation are collapsed into one level as they are not truly hierarchical. 3 4/23/13 Applying Bloom’s taxonomy Independent raters assign a Bloom’s level to Assessment items weight by Point value discuss to Reach consensus on each item Does the MCAT stress rote memorization? N AP Biology Undergraduate GRE MCAT Medical School 157 111 108 109 101 Weighted ratings 2.35 ± 0.12 2.43 ± 0.14 2.32 ± 0.10 2.57 ± 0.09 1.93 ± 0.08 Zheng et al 2008 4 4/23/13 Study objectives Characterize and compare the cognitive skills routinely assessed in o o Introductory biology courses (Study 1, 2) Upper division biology courses (Study 2) Characterize the relationship between cognitive skill level and difficulty in biology (Study 2) Study 1 Introductory biology, nationally 56 faculty (from FIRST II or SI) taught 77 introductory biology courses 2 Independent Raters assigned a Bloom’s level to Exam/quiz quesKons (n=9,317) IRR = 0.65 RaKngs averaged 5 4/23/13 Study 1 Course demographics Momsen et al 2010 Study 1 Introductory biology, nationally 100% Syllabus Goals Assessment Items 80% 60% 40% 20% 0% 1 Knowledge 2 3 4 Bloom's Level 5 6 Synthesis Momsen et al 2010 6 4/23/13 Study 2 Characterizing the biology major Large, public university, very high research activity o Single department of biological sciences • • • 17 research-active faculty ~500 undergraduate majors Courses serve science and engineering majors Focus on a single academic year (2010-11) o Collected all exams, term papers, portfolios, high stake homework from courses within major o No. Courses No. Items Intro biology 5 1100 Upper division 21 2051 Study 2 Applying Bloom’s taxonomy 3 independent raters assigned a Bloom’s level to Assessment items (n=3151) Agreement of 83% discussed to Reach consensus on each item 7 4/23/13 Study 2 Just the facts?! Introductory 100% Upper Division 80% Level 60% N Average Level Introductory 1100 1.55 ± 0.67 Upper Division 2051 1.42 ± 0.87 40% 20% Take-home exams Course portfolio 0% 1 2 Knowledge 3 4 5 6 Synthesis Bloom's Level Study 2 Bloom’s level versus performance* 1 course, n = 3 exams Percent Correct 100 80 60 40 20 0 0 1 Knowledge *For introductory biology only 2 3 4 Bloom's Level 5 6 Synthesis Momsen et al 2013 8 4/23/13 Study 3 (bonus!) Good news? 100% Items with figures Items without figures 80% 60% Still a lot of identify questions… 40% But a small yet significant shift towards higher-level cognitive tasks 20% 0% 1 Knowledge 2 3 4 Bloom's Level 5 6 Synthesis Good news? Study 3 (bonus!) Questions with figures ask students to label, interpret, reason with, create, evaluate, etc. 9 4/23/13 Assessment drives learning Provides practice with • Disciplinary skills • Concept understanding • Metacognitive skills New information Perceived Will this be on the test? Organize Working memory Elaborate Visualize Processed Retrieval Long-term memory Based on Kolb 1984 Moving beyond the facts Synthesis & Evaluation Create, integrate, assess, convince Analysis Separate, classify, order, interpret Application Show, solve, modify, relate, calculate Comprehension Knowledge 10 4/23/13 Ready to LEVEL UP? Coding assistance Erika Offerdahl Lisa Montplaisir Elizabeth Anderson Amy Williams Method development Tammy Long Sara Wyse Elena Bray Speth Diane Ebert-May Scott Freeman Thanks to over 100 faculty who’ve graciously shared exams, quizzes, and homework assignments. NSF-DUE #0833268 11
© Copyright 2026 Paperzz