Assessment of Student Learning Module 4 Module 4 ASSESSING KNOWLEDGE AND SIMPLE UNDERSTANDING Objectives Differentiate the different methods of assessing knowledge and simple understanding Discuss the specific guidelines in constructing completion, short answer and selected-response items Critique sample test items and improve the noted limitations of the test items Construct sample items for completion, short answer and selectedresponse items Introduction In this lesson, you will see how different learning targets can be conceptualized and how assessment methods can be used to measure each type of learning. The logic here is that the nature of the learning target is what influence which assessment method is used. As was pointed in the last lesson, some assessment methods measure certain targets better than others. Your job is to refine your learning targets, select the most appropriate type of assessment, Page 1 of 27 Assessment of Student Learning Module 4 prepare the assessment so that it will meet the criteria of high quality, administer it, and then score and interpret the results. Knowledge and understanding learning targets are the ones most commonly assessed in tests and quizzes given after instruction. As you can recall from your own experiences, most tests require students to remember facts, definitions, concepts, places and so on. Some tests go beyond simple memorization and assess understanding. We will begin the module with some important considerations for preparing for any type of assessment of any type of target. This is to review what you think you want to do in light of the criteria for ensuring high-quality assessments and therefore answer the following questions: 1. Do I have clear and appropriate learning targets? 2. What method of assessment will match best with the targets? 3. Will I have good evidence that the inferences from the assessments will be valid? 4. How can I construct an assessment that will minimize error? 5. Will my assessment be fair and unbiased? Have students had the opportunity to learn what is being assessed? 6. Will the assessment be practical and efficient? Types of Knowledge and Understanding Targets The simple phrase “what students should know” is used frequently as a concept for inclusion of important learning outcomes and standards. But this phrase is also pretty Page 2 of 27 Assessment of Student Learning Module 4 vague. We need to be more specific about what is meant by “know” and “Knowledge”, so that appropriate assessment methods can be selected to foster as well as measure the type of learning target that is desired. With Bloom’s taxonomy, knowledge is the first and lowest level of categories in the cognitive domain, in which knowledge is defined as remembering something. This requires students to recall or recognize facts, definitions and terms but in the contemporary view of knowledge or “knowledge representation”, remembering is only part of what occurs when students learn. There are three types of knowledge representations but only two have direct application to assessment: declarative and procedural knowledge. Declarative Knowledge and Understanding Declarative knowledge is information that is retained about something and is similar to Bloom’s first level- remembering and is represented by simple association, such as rote memory. At a higher level, declarative knowledge consists of concepts, ideas and generalizations that are more fully understood and applied. This type of knowledge involves “understanding” in the form of comprehension or application, the next two levels in Bloom’s taxonomy. Thus, declarative knowledge can exists as recall or understanding, depending on the intent of the instruction and how the information is learned. For example in Geometry, students learn the concept of “rectangle” at the level Page 3 of 27 Assessment of Student Learning Module 4 of recall, and then they simply memorize a definition or identify rectangles from a set of different shapes. If students “understand” the concepts of rectangle, however, they will be able to give original examples and identify rectangles of different sizes and shapes they have never seen before. Procedural Knowledge and Understanding Procedural knowledge is “knowing how to do something”. What is demonstrated is knowledge of strategies, procedures, and skills students must engage in, and this can be demonstrated at different levels. At the recall level, students simply identify or repeat the needed steps. Understanding is indicated as students explain in their own words (comprehension) and actually use the steps in executing a solution (application) Table 1 presents the definitions and examples of the two major types of knowledge and examples Level Recall knowledge Declarative Procedural Defines, identifies “specific facts, Defines, identifies “ correct procedure, concepts, principles or theories steps, skills or strategies Ex. Is able to define democracy Understanding (comprehension) Ex. Is able to identify, in correct order , steps in scientific method Converts, translates, distinguishes, Converts, translates, distinguishes, explains, provides examples of explains, provides examples of “ correct “essential meanings of concepts procedure, steps, skills or strategies and principles” Ex. Is able to give three examples Ex. Is able to explain whether a set of of countries that are democracies. procedures follows the scientific method Page 4 of 27 Assessment of Student Learning Module 4 Understanding (application) Uses existing knowledge of concepts, principles and theories in new situations, to solve problems, interpret information, and construct responses Uses existing knowledge of correct procedures, steps, skills, or strategies in new situations, to solve problems, interpret information, and construct responses Ex. Is able to determine whether a new country is a democracy by its Ex. Is able to demonstrate in writing the description correct use of the scientific method to solve a novel problem ASSESSING RECALL KNOWLEDGE Both declarative and procedural recall knowledge are best assessed with completion, short-answer, and selected response items (e.g., matching items, binarychoice items, and multiple choice items). Each type has its own strengths and weaknesses in relation to your teaching situation and personal likes and dislikes. You need to be comfortable with whatever method you will use, and this consideration is probably more important that other factors like ease of construction and scoring. Completion and Short-Answer Items The most common and effective way to assess recall knowledge is simply to ask a question and require the students to answer it from memory. Items for which the students respond to an incomplete statement are completion items; a brief response to a question is a short-answer item. Page 5 of 27 Assessment of Student Learning Module 4 Completion items. This type offers the least freedom of student response, calling for one answer at the end of the sentence. Responses may be in the forms of words, numbers or symbols. Strengths: 1. They are easy to construct 2. Their short response time allows a good sampling of different facts 3. Guessing contributes little to error 4. Scorer reliability is high 5. They can be scored quickly that short-answer or essay items 6. They provide more valid results that a test with an equal number of selectedresponse items (e.g., multiple choice) Limitations: 1. It is difficult to phrase statement so that only one answer is correct 2. Scoring is contaminated by spelling ability when responses are verbal. 3. Scoring is tedious and time consuming Page 6 of 27 Assessment of Student Learning Module 4 Guidelines in constructing completion items 1. Paraphrase sentences from textbooks and other instructional materials. Statements in textbooks, when taken out of context are often too vague or too general. Likewise, students tend to memorize phraseology in the text. Paraphrase or restate facts in words that are different from those students have read. Example: the textbook statement is “The criterion that refers to the extent to which the test yields consistent, dependable and stable scores is called reliability”. Poor item: The criterion that refers to the extent to which the test yields consistent, dependable and stable scores is called _____________. Improved: The test yields consistent, dependable and stable scores, the test is said to be _. 2. Word the sentence so that only one brief answer is correct. The single greatest error in writing completion items is to use sentences that can be legitimately completed with more than one response. This is true when the sentence is openended. Poor: Columbus first landed on “ America” _________. Improved: Columbus first landed on “America” in _______. Page 7 of 27 Assessment of Student Learning Module 4 Columbus first landed on “America” in the year _______. Better: 3. Place one or two blanks at the end of the sentence. If blanks are placed at the beginning or at the middle, it may be difficult for the students to understand what response is called for. It is easier to first read the sentence and then determine what will complete it correctly. (That is why it is called completion item. Poor: In 1945, ______ decided to have the atomic bomb dropped on Japan. Improved: The name of the president who decided to have the atomic bomb dropped on Japan in 1945 was ___________. 4. Do not include several blanks in a single sentence. This will confuse students and measure reasoning skills as much, if more, than recall. Poor: 5. The name of the ______________ who decided to have the ___________________ dropped on ___________ in 1945 was _________________. If answered in numerical units, specify the unit required. Poor: The distance between the moon and the Earth is __________. Improved: The distance between the moon and the earth is ___________miles. Page 8 of 27 Assessment of Student Learning Module 4 6. Do not include clues to the correct answer. The common wording errors are using single or plural verbs and wording the sentence so that blank is preceded by “ a “ or “an”. Poor: The supply –type used to measure the ability to organize and integrate material is called an _____. Improved: Supply-type items used to measure the ability to organize and integrate materials are called ______. Short –Answer Items. This type, in which students supply an answer consisting of one word, a few words, or a sentence or two, is generally preferred to completion items for assessing recall targets. First, this type is similar to how teachers phrase questions and direct student behavior during instruction, making questions more natural for the students. Second, it is easier for teachers to write these items to more accurately measure knowledge. Short-answer items are usually stated in the form of a question (e.g., “Which state is surrounded by three large bodies of water?”). They can also be stated in general directions (e.g., “Define each of the following terms’), and they can require responses to visual stimulus materials (e.g., “Name each of the countries identified with arrows A-D’). Strengths: Similar with the completion items. Page 9 of 27 Assessment of Student Learning Module 4 Limitations: 1. Scoring takes longer 2. Scoring is more subjective. Guidelines in constructing short-answer items 1. State the item so that only one correct answer is correct. Be sure that the question or directions are stated so that what is required in the answer is clear. Poor: Where if Eifel Tower located? (answers could be Europe, Paris, France) Improved: In what country is the Eifel Tower located? 2. State the item so that the required answer is brief. Keep students responses to a word or two, or a short sentence or tow if necessary, by properly wording the item, offering clear directions and providing spaces or blanks that indicate the length of the responses. Poor: What does the term reptile mean? ____________________________________________________________ ____________________________________________________________ ________________________ Improved: Name three characteristics of reptiles. 1. ________________ 2. ________________ 3. ________________ Page 10 of 27 Assessment of Student Learning Module 4 3. Do not use questions verbatim from the textbooks and other instructional materials. This will discourage students from rote memorization 4. Designate units required for the answer. This will avoid the time students may take to try figure out what is wanted. Poor: When was Dr. Jose Rizal shot at Bagumbayan? Improved: In what year Dr. Jose Rizal was shot at Bagumbayan? 5. State the items succinctly with words students understand. Avoid using words or phrases that may be difficult for some students to understand. Poor: What was the name of the extraordinary president of the United States who earlier had used his extensive military skills in a protracted was with exemplary soldiers from another country? Improved: What United States general defeated British and later became president? Matching Items. Matching items effectively and efficiently measure the extent to which students know related facts, associations, and relationships. In a matching item, the items on the left are called the premises or the question column, and the right is the responses or the option column. The students’ task is to match the correct response with each of the premises. Page 11 of 27 Assessment of Student Learning Module 4 Strengths: 1. The teacher can obtain a very good sampling of recall knowledge. 2. It is easily and objectively scored. 3. Easier to construct than multiple choice items 4. Reading and response time is short Limitations: 1. This item type is largely restricted to simple knowledge outcomes based on association 2. When there is insufficient material to include in the item, the items are weak measures because irrelevant information is added. Guidelines in constructing matching items 1. Make sure directions are clear to students. It is helpful to indicate in writing the basis for the matching and where and how responses should be recorded. Generally, letters are used for each response in the right-hand column. It is also important in the directions to indicate that each response may be used once, more than once or not all. This will lessen guessing. 2. Include homogeneous premises and responses. It is not a good idea to contain both dates and men’s names as responses. Testing homogeneous material with matching is effective for fairly fine discrimination among facts. Page 12 of 27 Assessment of Student Learning Module 4 3. Limit matching exercises to 10-15 items only. Use for to eight premises. A relatively short list will probably be more homogeneous and will be perceived by students as more fair. 4. Keep responses short and logically ordered. Students will be more accurate in their answers if the responses are in logical order. Thus, if responses are dates they should be rank ordered by years; words or names should be alphabetized. 5. Avoid grammatical clued to correct answers. As with completion items, you need to be careful that none of your matches are likely because of grammatical clues, such as verb tense agreement. 6. Put the entire matching item on the same page. This will prevent the distraction of flipping pages back and forth, and prevent students from overlooking responses on another page. Example: The following is an example of a good matching set. Notice the complete directions, responses on the right in logical order, and homogeneous content (Hollywood actors and roles) Directions: Match an actor’s name in Column A with a film role of the actor played in Column B. Write the letter of the film role next to the number of the corresponding actor. Each film in Column B may be used once, more than once, or not at all. Column A ___1. Anthony Hopkins ___2. Jim Carrey ___3. Wesley Snipes ___4. Mike Myers ___5. Dustin Hoffman Column B a. Ace Ventura b. The Jackal c. Captain Jack Aubrey d. Hannibal Lecter e. Austin Powers Page 13 of 27 Assessment of Student Learning Module 4 ___6. Jack Black ___7. George Lazenby ___8. Robin Williams ___9. Sigourney Weaver ___10. Michelle Yeoh ___11. Rusell Crowe f. g. h. i. j. k. l. m. Blade Yu Shu Dewey Finn Professor Brainard Benjamin Braddock James Bond Ellen Ripley John Book True or False and Other Binary-Choice Items When students select and answer from only two response categories, they are completing a binary-choice item o sometimes called alternative response. The most popular binary-choice item is true/false question; other types of options can be right/wrong’ correct/incorrect, yes/no, fact/opinion, agree/disagree, and so on. Binarychoice items are constructed in a form of a propositional statement and one of the two choices must be absolutely true or false, correct or incorrect, and so on. Strengths: 1. Students are familiar with the items because such questions are similar to what is asked in class. 2. Short binary items provide for an extensive sampling of knowledge because students are able to answer many items in a short time 3. Items can be written in short, easy-to-understand sentences. 4. Scoring is objective and quick. Page 14 of 27 Assessment of Student Learning Module 4 Limitations: 1. It is susceptible to guessing. 2. It is difficult to write items beyond the knowledge level that are free from ambiguity 3. No diagnostic information is provided by the incorrect answers. Guidelines in constructing binary-choice items 1. Include only one central idea in each statement. The decision should not depend on some subordinate point or trivial detail because students tend to be confused and the answer is more apt to be influenced by reading ability than the intended outcome. Poor: T F* called an Improved: T* item. F The true-false item, which is favored by test experts, is also alternative-response item. The true-false item is also called an alternative-response 2. Keep the statement short and use simple vocabulary and sentence structure. This will increase the likelihood that the point of the item is clear. Poor: T* F The true-false item is more subject to guessing but it should be used in place of a multiple-choice item, if well constructed, when there is a dearth of distracters that are plausible. Improved: T* F The true-false item should be used in place of a multiple choice item when only two alternatives are possible. Page 15 of 27 Assessment of Student Learning Module 4 3. Word the statement so precisely that can be unequivocally be judged true or false. Specific determiners that are most likely to be true (some, may, possible, etc.) and most likely to be false (no, none, never, etc.) should be avoided because it provides clues. Poor: T* F Some objective tests are prone to guessing. Improved: T* F Objective test such as alternative-response items is prone to guessing. 4. Use negative statement sparingly and avoid double negatives. The “no” and/or “not” in negative statement are frequently overlooked and they are read as positive statement. Statements including double negatives tend to be so confusing that they should be restated in positive form. Poor: T* F Correction-for-guessing is not a practice should never be used in testing. Improved: 5. T* F Correction-for-guessing is a practice that should sometimes be used in testing. When cause-effect relationships are being measured, use only true propositions. When used for this purpose, both propositions should be true, and only the relationship judged true or false. Poor: T F* True-false items are classified as objective items because students must supply the answer. Page 16 of 27 Assessment of Student Learning Module 4 Improved: 6. T F* True-false items are classified as objective items because there are only two possible answers. Do not try to trick students. This happens when a word that changes the meaning of an idea is included. This practice undermines your credibility, frustrate students, and provide less valid measures of knowledge. Poor: T F* “The Raven” was written by Edgar Allen Poe. Improved: T* F “The Raven” was written by Edgar Allan Poe. 7. Commands cannot be true or false. They do not state or assert anything; they simply direct. Poor: T F Eat the four basic foods. Improved: T F To keep us healthy, eating the four basic food is helpful. Multiple-Choice Items Multiple choice items are used widely in schools, even though they may not be the best method for assessing recall knowledge. Multiple-choice items have a stem, in the form of a question or incomplete statement, and there are three or more alternatives. The alternatives contain one correct or best answer and two or more distracters. For measuring recall knowledge, it is usually best to use a direct question as the stem because it is easier to write and its format is familiar to students. Page 17 of 27 Assessment of Student Learning Module 4 Strengths: 1. Learning outcomes from simple to complex can be measured. 2. Highly structured and clear tasks are provided. 3. A broad sample of achievement can be measured 4. Incorrect alternatives provide diagnostic information. 5. Scores are less influenced by guessing that true-false items. 6. Scoring is easy, objective and reliable. Limitations: 1. Constructing good items is time consuming. 2. It is frequently difficult to find plausible distracters. 3. Scores can be influenced by reading ability. Guidelines in constructing multiple-choice items 1. Write the stem as a clearly described question or task. It is best to put as much information as possible in the stem and not the responses, as long as the stem does not become too wordy. The stem is longer than the alternatives but in the end, a good indicator of an effective stem is if students have a tentative answer in mind quickly, before reading the options. Page 18 of 27 Assessment of Student Learning Module 4 Poor: Validity refers to a. the consistency of test scores b. the inference made on the basis of the test scores c. measurement error as determined by standard deviation d. the stability of test scores Improved: 2. The inference made on the basis of the test scores refers to a. Reliability b. Validity c. Stability d. Measurement error Avoid the use of negative in stem. Using words like not and except will confuse students and create anxiety and frustration. So try to word the stem positively. If negative statement cannot be avoided, emphasized it by boldface or underline. Poor: Which of the following is not a mammal? a. b. c. d. e. Improved: Which of the following is a mammal? a. b. c. d. e. 3. Bird Dog Horse Whale Cat Bird Frog Whale Fish Lizard Write the distracters to be plausible yet clearly wrong. If the distracters are obviously wrong, they are useless because the intent of a multiple-choice items is to have students discriminate among plausible answers. Page 19 of 27 Assessment of Student Learning Module 4 Poor: Which of the following is the largest city in the United States? a. Michigan b. London c. New York d. Berlin Improved: Which of the following is the largest city in the United States? a. Los Angeles b. Chicago c. New York d. Miami 4. Avoid using “all of the above”, “none of the above”, or other special distracters. When test makers are having distracters, they frequently resort to the use these distracters. Since students are to select only one answer, they can detect “ all of the above” as the correct answer simply by noting two of the alternatives are correct. 5. Each question should have only one answer, not several possible answers. Example 1: Poor: What are the differences between invertebrates and vertebrates? ( There are many differences between them, according to structure, etc.) Improved: Animals with backbones are vertebrates. Which of the following is an invertebrate? a. Dog b. Snake c. Lizard d. Cockroach Page 20 of 27 Assessment of Student Learning Module 4 Example 2: Poor: Mosquitoes and flies are disease carriers. How can we protect ourselves from them? a. Spray insecticides b. Destroy their young c. Clean the surroundings d. Use mosquito nets and cover our food Improved: Which is the correct way of protecting ourselves from mosquitoes and flies which are disease carriers? a. Cover stagnant places b. Leave garbage cans open c. Leave empty cans around 6. Arrangement of correct answers should not follow any pattern. This happens when your purpose if to facilitate scoring. Students have the tendency to look for pattern of answers (like a,b,c,d,..a,b,c,d or a,a,a,b,b,b,c,c,c,d,d,d, etc.). If this happens, students may get the correct answers even though they do not even read the items. ASSESSING SIMPLE UNDERSTANDING: Comprehension and Application As stated earlier, comprehension and understanding are two type s of knowledge through which students demonstrate their understanding of something. Assessing comprehension Comprehension is demonstrated when students understand, in their own words, the essential meaning of a concept, principle or procedure. They show this by providing Page 21 of 27 Assessment of Student Learning Module 4 explanations and examples, by converting and translating and by interpreting and predicting – mental operation which is hierarchically categorized into three levels: translation, interpretation and extrapolation. Example 1: Recall Knowledge (short-answer): Define hypothesis: _________________________________. Comprehension (translation, completion): Sunlight is used by plants to make energy in a process called ______. Comprehension (interpretation, short answer): Explain how plants get energy from the sun. Comprehension (extrapolation, short answer): What would happen to plants if they did not receive any sunlight for a long time? Example 2: Recall knowledge (multiple choice): Which of the following is the process by which plants use light to make glucose? a. Respiration b. Photosynthesis c. Energizing d. Growing Comprehension (translation, multiple choice): In plants, sugar is made by energy from the sun from which of the following? a. b. c. d. Respiration Photosynthesis Energizing Growing Comprehension (extrapolation, multiple-choice): Which of the following is most consistent with the process of photosynthesis? Page 22 of 27 Assessment of Student Learning Module 4 a. Plants that get light do not need to make glucose. b. Plants that get less light make less glucose. c. Glucose is produced from plants before photosynthesis. d. Energy is stored in plants as glucose. Assessing application Understanding is demonstrated through application when students are able to use what they know to solve problems in a new situation. Knowing something well enough to apply it successfully to new situations is called learning for transfer. Your goal in assessing application is to construct items that contain new data or information that the student must work with to obtain the answer to create new problems or applications in which students must extend what they know in a novel way. Examples: Application 1. Rica has decided to make two magnets by wrapping wire around a nail and attaching the wires to a battery so that electric current can create a magnetic force. One magnet (A) uses thin wire and one magnet (B) uses thick wire. Which magnet will be strongest? a. A b. B c. A and B will be the same. d. Cannot be determined from the information provided 2. T F Other things being equal, an electric stove with greater resistance will be hotter than a stove with less resistance. Page 23 of 27 Assessment of Student Learning Module 4 3. To increase the heat produced from his electric iron, Mr. Reyes would _____ the resistance. PUTTING TESTS TOGETHER After you have developed test items, they need to be put together in the form of a test. The following guidelines should be considered: a. Test directions- test directions should include the purpose, time allowed for completing the test, basis for responding, procedures for recording answers, what to do with guessing and how constructed-response items will be scored. b. Test arrangement – arranging the items from easiest to most difficult has little effect on the results. The most important consideration is item type. Items answered more quickly would generally come first, and keep all items in one section of the whole test. c. Physical layout of the test- test items should be formatted in such as way that they are easy to read and answer. Do not crowd too many items onto a page. Page 24 of 27 Assessment of Student Learning Module 4 Activity1 1. Identify each of the following descriptions as declarative (D) or procedural (P) and as recall (R) or understanding (U).( Note that each item requires two answers) a. Define procedural knowledge b. What is the sequence of steps in preparing an objective test? c. Give an example of a multiple-choice item that measures application. d. List three suggestions for constructing matching items. e. Predict whether students will have questions about how to answer the items in the test. f. Review the strategy a teacher has used to construct binary-choice test items to determine if they can be improved 2. Match the suggestions or descriptions from Column A with the type(s) of objective tests in Column B. Each type of item may be used once, more than once, or not at all; each suggestions or descriptions may have more than one correct match. Column A __1. Generally more time-consuming to construct __2. Scoring may be a problem __3. Effectively measures relations __4. Conveniently constructed from instructional materials __5. Responses ordered logically __6. __7. __8. __9. Column B a. Completion b. Short answer c. Matching d. Binary choice e. Multiple choice Correct answers spread equally among all possible choices Verbatim language from textbooks is avoided Uses clear, concise statements Uses blanks of equal length Page 25 of 27 Assessment of Student Learning Module 4 3. Using the checklist for writing objective items, evaluate each of the following items and revise it so it will be improved. a. Earth is the fourth planet from the sun and is called the living planet b. Match the planets with each characteristics Column A __1. Earth __2. Venus __3. Jupiter __4. Mercury __5. Saturn Column b a. Nearest planet from the sun b. The ringed planet c. Largest planet d. Third planet from the sun e. Brightest planet c. Circle the correct answer. Biodegradable substances are a. Nonrenewable resources b. Materials that can be broken down into substances that are simpler and do not result in environmental pollution c. Becoming less popular d. Like fossil fuels d. ______ is the father of educational psychology. e. How does energy from the sun affect the Earth? Page 26 of 27 Assessment of Student Learning Module 4 Activity 2 Using the content of this module, construct sample items as directed. a. 5 binary-choice items b. 5 multiple-choice items c. 3 completion items d. 3 short-answer items e. 1 matching type (5 items) Note: Include in the test the directions and provide the answer key to the items. Remember that all the items will assess knowledge and simple understanding only. Activity 3 Write a brief reflection on the following: a. On critiquing test items b. On writing test items Page 27 of 27
© Copyright 2026 Paperzz