New SOL Test, Harder Thinking WHAT TO DO ABOUT THE “NEW” ENGLISH SOL ASSESSMENT: CHANGING COMPREHENSION INSTRUCTION Peter Dewitz, PhD Across the state of Virginia, scores on the Standards of Learning assessment in English declined for the 2012-­‐2013 academic year. Last year 75 percent of the students across the state passed the SOL English assessment compared to 89 percent the year before. African American, Hispanic and students from low-­‐income backgrounds struggled more with the test than did middle class white students. African American scores dropped 21 percent and Hispanic student’s scores dropped 19 percent. Students in Virginia did not become weaker readers from one year to the next, but several factors were at work that drove the test score decline. Research suggests that anytime a new test is introduced performance declines. Students are less familiar with the forms, items and demands of the test and teachers have less knowledge of the test to prepare the students. For years, the SOL English test remained largely the same and most teachers and their students learned to deal with its structure and idiosyncrasies. Sometimes the test scores decline because the state simply raises the passing score. This was the case in New York in 2010. Students’ actual performance did not decline, but students needed a higher score to “pass” the test, making it appear that student were not doing as well. Much of the recent test score decline in Virginia came about because the test developers redefined what it means to be a proficient or advanced reader. This redefinition was engineered by changing the demands of the test items. Students needed more and 1 New SOL Test, Harder Thinking deeper knowledge and more critical thinking skills. The items were changed so that tried and true test taking strategies became obsolete. In this paper I will describe how the SOL test changed and how teachers need to think differently about planning and teaching reading, and how students need to think differently while reading. My access to the new SOL test was limited, but a very close analysis of the practice items on the Virginia Department of Education’s website, plus some additional access, allows me to draw some insights about the test. Interviews with 50 strong students in 5th and 7th grades at the end of the academic year confirmed my assumptions about how the test changed. These students reported that the actual test was more difficult than the practice tests. A frequent comment from the best students was, “It was really hard to decide which was the right answer, even after reading them over several times.” From these insights I will suggest ways that teachers must change their thinking when planning and teaching reading. Then I will consider how students must think differently while reading and when taking the test. How the English SOL Test Has Changed To increase the rigor of the SOL English test the developers made a number of changes. Some of these changes involved the use of technology-­‐enhanced items that directed students to make multiple responses. Other changes increased the depth of thinking required when answering traditional multiple-­‐choice items. Before last year students were guided to “slash the trash” or eliminate the obviously incorrect answers. In the most recent version of the SOL test there was little or no trash to slash. At least five significant changes can be found in the new SOL test. 2 New SOL Test, Harder Thinking 1. Deeper Vocabulary Knowledge. It is not just the assessed vocabulary words that present a problem, but also the words in the passages that are not assessed. In a fifth grade passage students needed to know words like achievement, representative, devoted, advancement, appreciated, stunning, diversity and treasured. Each of these terms was essential to grasping the meaning of the text and none of these words was directly assessed. The test also demands that students understand words at a deeper level. It is not enough to know the definition of a word; the reader must also know characteristics of the concept the word represents. So at times the students have to sort words by categories searching for common features and underlying characteristics. Students have to compare and contrast words. Thus tasty and odorous are both sensory words, confident and stunning have positive connotations and stingy and dread have negative connotations. Students need to know more about a word than its definition. 2. Read Closely to Support Conclusions and Reasons. In the old SOL test the students were asked to make inferences, draw conclusions, find the main idea or the author’s purpose -­‐-­‐ difficult insights. In the new SOL test the inference or the main idea is often given in the question stem and the student is asked to find the text that supports the conclusion. “Which sentence supports the conclusion that Bob was feeling morose?” These questions require a much closer reading of the text, a careful analysis of the language in the text. 3 New SOL Test, Harder Thinking 3. Employ Multiple Strategies. In the old SOL test students were asked to compare and contrast in one question and make an inference in another question. In the new SOL test two strategies might be combined into one question. Thus a reader might be asked to make a comparison, but one of the conditions that were being compared had been implied. In another question the student might have to discover when the author first expressed a particular value, thus combining inference and sequence. The student’s task of comprehending the question has become more difficult. 4. Minimizing the Literal. In the old SOL test literal questions were common while in the new SOL test these literal questions are less common and they have become more complex. A passage might say that Mount Rushmore is a major attraction and the correct answer indicates that thousands of people visit it every year. This requires the reader to establish commonality between a major attraction and the number of visitors. This is actually an implied main idea/detail relationship. Thus what appears to be a straightforward literal question requires inferential thinking. 5. Change in Perspective. In the old SOL test the questions were almost always pitched at the established perspective of the narrator. In many stories an outside narrator tells the story from the perspective of an observer. In the new SOL test questions are phrased from different perspectives and these questions ask how different characters in the story might view a problem, an event or an interaction. 4 New SOL Test, Harder Thinking New Thinking Required of Teachers The changes to the SOL test require that teachers think differently about reading instruction and specifically how they plan and lead discussions to develop students’ comprehension and thinking. I present this as a three-­‐step process. Selecting the text, planning the discussion and leading the discussion. Selecting Texts Teachers need to select interesting and complex text. A complex text is necessary to develop comprehension because readers need something to think about. With no challenge, comprehension does not grow. The texts found in workbook activities are structured simply so that there is minimal thought or challenge for the reader. If a student practices using a strategy with easy and simple text, she is likely to be overwhelmed trying to apply the same strategy to complex text. You can evaluate a text’s complexity by considering the following factors. A text is complex because of the level of meaning, its structure, its language complexity and the knowledge demands it imposes on the reader. The four characteristics of complex text are illustrated in Figure 1 (Hiebert, 2012). Interesting complex texts can be found anywhere – novels, poetry, short stories, picture books, and some, but not all, selections in core reading programs. Complex text exists even with first grade material. When Frederick chooses not to work to prepare for winter, in the picture of that name, the author Lionni (1973) affirms the value of art. When Dr. Desoto and his wife outwit a fox they demonstrate insight in the mind of the villain (Steig, 2010). 5 New SOL Test, Harder Thinking Figure 1: Characteristics of Complex Narrative and Informational Text Narrative Text Informational Text Level of Meaning A complex text has more than The text not only presents one level of meaning. Beyond the information, but also basic plot a reader can infer includes overarching themes themes and the author’s purpose. and evaluative positions. Structure A complex text has a more Text employs multiple intricate structure. Multiple plots, structures (cause & effect; foreshadowing and flashbacks comparison & contrast) and make the structure more means of signaling these complex. structures. Language complexity A complex text might include Narrative, dialogue and metaphorical and figurative informative language might language. occur within the same text. Knowledge demands A complex text deals with Complex ideas and complex ideas and interconnecting concepts. interconnecting themes. The writer develops multiple concepts. Planning the Discussion Teachers need to understand how texts work to plan discussions and facilitate students’ use of strategies (Kucan, Hapgood, Palincsar, 2011). Teachers need to understand how ideas develop and are organized within a text. They need to know how an author develops an argument, how he implies main ideas and themes, and how he uses specifics to support his conclusions. The teacher must read and think critically at a high level to develop students’ thinking. 6 New SOL Test, Harder Thinking Planning means reading a text carefully looking for the characteristics that follow and then developing discussion questions and prompts. These characteristics require the attention of a careful reader and a successful test taker. •
What are the big ideas and the themes in the text? •
How does the author wield facts, details, language and inferences to develop big ideas and themes? Remember the SOL test will assess students’ understanding of how the details and clues contribute to the main ideas and themes. •
What are the major inferences that a reader must make to comprehend the text? The skilled adult reader makes inferences without even thinking about them, while younger and weaker readers do not. Some inferences are automatic while others voluntary. To comprehend the plot of a story, what happened, it is not necessary to always infer a character’s traits. A teacher needs to anticipate these critical inferences and pose questions that elicit them. •
What are the major hurdles to comprehension for the average students in the class? These hurdles might be difficult pronoun – noun relationships, noting semantic equivalence between two related but different terms (horses and stallions), lack of prior knowledge, complex vocabulary, and inferences that require thought on the part of the reader. After a careful reading of the text, the teacher should plan her discussion considering when to pause and discuss, how to initiate the discussiom through questions and comments, and how to sustain it. A few thoughts about each: 7 New SOL Test, Harder Thinking •
Stop as seldom as possible, but stop to discuss important ideas, themes, or actions. If you focus on the big ideas and the major inferences, the small facts and details take care of themselves. Sometimes one sentence or one paragraph is critical and requires everyone’s attention. Mark your stopping points. •
Begin most discussion with open-­‐ended questions. What is the author telling us? What is the story or article about? These open-­‐ended questions give you insight into the students’ thinking. Write out your questions. •
Sustain the discussion by validating students’ ideas, rephrasing vague responses, posing questions that require judgment and evaluations and questions that require students to justify answers with text evidence. Leading the Discussion Lead an engaging discussion that requires close reading and critical thinking (Boyles, 2013). Close reading will lead to success on the SOLs and to career and college readiness. “Close reading is a very critical style of reading, one that asks the reader to delve into the text, analyze it, determine the important ideas, consider how these ideas are organized and developed. Close reading often requires multiple readings of a text” (Brown & Dewitz, 2013, p. 64). For many, close reading is a new term, but not a new concept. You have all engaged in close reading. After that last thunderstorm toppled the large oak tree onto your garage you engaged in a close reading of your insurance policy to see if you were covered. In college, you read a biology text closely to determine the exact role of mitochondria in the cell’s nucleus. When you read in Shakespeare’s Henry VI, Part 2, “The first thing we do, let's
kill all the lawyers” you had to decide if that was a literal statement of intent or a figurative way of considering the value of a society that was less constrained by endless petty rules. 8 New SOL Test, Harder Thinking If close reading helps students understand the meaning intended by the author, critical reading does not take the text at face value but engages them to explore the author’s intent often with a skeptical eye. Critical reading asks student to analyze and evaluate the text while considering the biases of the author. Does this text tell all? How would another text give me a different slant on the issues? •
Posing initiating questions. The initial questions in a discussion should target overall understanding of the text, big ideas or themes, and important inferences. Initial questions should ask students to summarize and paraphrase what they have read (Kleitzen, 2000). This gives the teacher insight into what students understand. Stick to text dependent questions; questions where the answer comes from the text, but avoid questions that can be answered with one or two words. Avoid questions that put too much focus on the students’ prior knowledge and experiences. Use the questions in your core reading program with discretion – some may work, others may not. •
Use follow-­up questions to deepen students’ thinking. Ask students to analyze and evaluate what they read. A follow-­‐up question might ask a student to support his or her answer, find text evidence to support a position or argument. Ask questions that ask students to evaluate the comments of another student. Debate and disagreement among students builds critical thinking as they work to support their position from the text or refute the position of another. 9 New SOL Test, Harder Thinking •
Sustain the discussion. A discussion is sustained by a number of teacher moves including: acknowledging the importance and value of a student’s answer, expanding upon students’ thoughts, probing for clarification, turning the students back to the text to find justification, and rephrasing the students’ first thoughts into more elegant language. A discussion is halted when the teacher interprets the text for the students. A discussion should never be the transmission of information or the oral assessment of the students. Thinking Required of Students Students need a new set of thinking tools not just to pass the SOLS, but also to comprehend and think critically about texts they encounter in school, at their future workplace or as citizens seeking to understand the current political/economic environment. For many students, especially those who have been in school for a while, this means giving up some old habits and learning some new ways of thinking. This manner of thinking, stimulated by close reading, has a number of characteristics all of which demand that the reader closely attend to the ideas in the text, how these ideas are developed and the language the author uses to express his or her ideas. Students have to give up a few thinking patterns that are unproductive. Some students believe that it is better to contribute to a discussion than remain silent. This leads to students responding strictly from their prior knowledge, offering tangents rather than focusing on the meat of the text. Researchers have documented that the weakest readers rely too much on their prior knowledge and neglect what the text has to say (Dewitz & Dewitz, 2003; Neuman, 1999). Some students focus on details or tidbits, sometimes called seductive details, when they should be noting the more important ideas. A text with rich 10 New SOL Test, Harder Thinking sensory language can easily cause students to be seduced by these details ignoring the arguments in a text. In a close reading students should be less concerned with bringing their personal experiences into play and more concerned with understanding the ideas and arguments of the author. Close reading is less concerned with making text-­‐to-­‐self or text-­‐to-­‐world connections (Harvey & Goudvis, 2007) and more concerned with the structure of the text’s idea. Making text-­‐to-­‐text connections is an asset especially if students can find similarities in main ideas, themes or the author’s style. In a close discussion of the text, exploring the ideas within takes precedence over relating the text to one’s personal experience. Prior knowledge is important because new understandings can only be built on existing concepts. In the new orthodoxy of close reading it is common to say that the teacher should not focus or build the students’ prior knowledge. Instead the students should work to make sense of the text on their own. This is indeed nonsense, because prior knowledge provides the scaffold onto which new understandings are built. A teacher must tread lightly, developing knowledge that students need, but not giving them the knowledge that they can learn from the text. As students engage in a close reading of a text they should be able to (from Common Core State Standards, 2010): •
Read closely and determine what the text says explicitly. •
Infer main ideas of informational text and themes within literature. •
Follow an author’s arguments and cite the evidence from the text to support conclusions you draw. 11 New SOL Test, Harder Thinking •
Analyze the structure and organization of a text and indicate how the structure supports the author’s purpose. •
Understand how different points of view shape the meaning and the writing style. •
Evaluate the ideas, arguments and claims of a text. •
Compare and contrast two or more texts or a text and other media. When to Read Closely and When Not Closely reading a text is hard and does not resemble the highly engaged reading we experience with the latest thriller or fantasy novel. Because close reading is difficult most students and many adults do not readily take to it. Because close reading is hard we must be very mindful of student motivation. Because close reading is intellectually demanding we must engage students in the process only when the outcome makes sense. Close reading must take place only when the context is right and the purpose is clear. When students are studying Thomas Jefferson it makes sense to closely read his Virginia Act for Establishing Religious Freedom. When students are studying weather it makes sense to closely study the physical processes that produce a hurricane. With a novel like Holes (Sacher, 1999) it makes sense to closely study the intricate flashbacks in the plot. Close reading demands that teachers focus even more on the conditions that will motivate students to engage in hard thinking. Reading closely is a valuable tool not just for passing an SOL test, but also for coping with important and difficult material both in and out of school. Close reading is a bit like medicine or a diet; it is good for you. But reading should also taste good. We should not abandon the soft and comfortable engagement with books, the delight we get from a good 12 New SOL Test, Harder Thinking story or from fascinating new information for the single-­‐minded pursuit of analytic and rigorous close reading. A good reader has multiple purposes. References Boyles, N. (2013). Closing in on close reading. Educational Leadership, 70(4) 36-­‐41. Brown, R. & Dewitz, P. (2103). Building comprehension in every classroom. New York: Guilford. National Governors Association Center for Best Practices & Council of Chief State School
Officers. (2010). Common Core State Standards for English language arts and literacy in
history/social studies, science, and technical subjects. Washington, DC: Authors.
Dewitz, P., & Dewitz, P. K. (2003). They can read the words, but they can't understand: Refining
comprehension assessment. The Reading Teacher, 56(5), 422-435.
Harvey, S. & Goudvis, A. Strategies that work. Portland, ME: Stenhouse.
Hiebert, E. F. (2012). Text complexity multi-index. Retrieved from
http://textproject.org/professional-development/text-matters/the-text-complexity-multiindex/ September 12, 2013.
Keltzien, S.B., (2009). Paraphrasing an effective comprehension strategy. The Reading Teacher,
63 (1), pp. 73-77.
Kucan, L., Hapgood, S., & Palincsar, A. S. (2011). Teachers’ specialized knowledge for
supporting student comprehension in text-based discussions. Elementary School Journal,
112(1), 61-82.
Lionni L. (1973). Frederick. Boston, MA: Knopf
Neuman, S. B. (1990). Assessing children's inferencing strategies. In J.Zutell & S. McCormick
(Eds.), Literacy theory and research: Analysis from multiple paradigms 39th Yearbook
(pp. 267-274). National Reading Conference.
Sachar, L. (1998). Holes. Boston: Farrar, Straus and Giroux.
Steig, W. (2010). Dr. Desoto. New York: Fish Square.
13