Instructor Guide “Finding Reliable Information Online: Adventures of an Information Sleuth” by Leslie Stebbins Instructor Guide for Finding Reliable Information Online: Adventures of an Information Sleuth The following guide uses the new Association of College and Research Libraries Framework as a basis for hands-­‐on classroom exercises. Please feel free to modify exercises for the needs of your students. Please send feedback for future iterations of this guide to: [email protected]. Table of Contents Learning Activity 1: Understanding & Evaluating Health Information .......................... 3 Learning Activity 2: Understanding & Evaluating Science Information ........................ 8 Learning Activity 3: Understanding & Evaluating Social Science Information ........ 15 2 Learning Activity 1: Understanding & Evaluating Health Information Three Frameworks are addressed in these exercises:1 1. Authority Is Constructed and Contextual. Information resources reflect their creators’ expertise and credibility, and are evaluated based on the information need and the context in which the information will be used. Authority is constructed in that various communities may recognize different types of authority. It is contextual in that the information need may help to determine the level of authority required. Ask students to read Chapter 2 before 2. Information Has Value. Information possesses several dimensions of value, session if possible. including as a commodity, as a means of education, as a means to influence, and as a • Page 2: Instructor introduction and brief class means of negotiating and understanding the discussion world. Legal and socioeconomic interests influence information production and • Page 3 & 4: Group Work/Handout • Page 5: Extended classroom discussion dissemination. 3. Research as Inquiry. Research is iterative and depends upon asking increasingly complex or new questions whose answers in turn develop additional questions or lines of inquiry in any field. NOTE: Instructor can pick and choose which of the following items to assign to groups depending on time allotted. Ideally group work will be done during class time so that instructor can roam the room and consult with groups as they discuss their work. Exercises can also be done with the entire class led by instructor or can be modified for take-­‐home work or individual assignment. 1 Framework for Information Literacy for Higher Education. Association of College & Research Libraries. Note: Assessment of the frameworks listed above can be done through observation and discussion. The thresholds represent a continuum of understanding and the exercises are designed to move students along this continuum. 3 Introduction to Classroom Activity (instructor led discussion): Today we are going to focus on a certain type of information: health information. But before we get started I have a few questions for you. 1. What is one of your favorite series -­‐-­‐-­‐from Netflix or Amazon or HBO or TV or wherever? 2. If you need to know how many episodes there are for “[name of series student mentioned]” how would you figure that out? a. Where would you search? What would you search for? (terms) b. What kind of source would you get? (If able to -­‐-­‐ search live and show them results) 3. Is (Wikipedia or IMDb or one of the first hits) a reliable source for this information? After brief answer/discussion talk about how information is contextual in that the type of information you need will determine the level of authority required. For the number of episodes in “Game of Thrones”, Wikipedia is fine. But is it the best source for health information – like if you just got diagnosed with diabetes? (provide a few minutes for a brief discussion – allow student voices to be heard –this will help you determine their level of experience). 4. Divide students into groups of 2 or 3 and provide them with the following handout.
4 HANDOUT: Finding Reliable Health Information A company called PharmaForward estimated that the following 15 health conditions were the most frequently searched for keywords used on Google in relationship to health:2 Health Keyword Global Monthly Searches Cancer Diabetes Depression HIV Acne AIDS Stroke Herpes Arthritis Breast Cancer Hepatitis Autism Lupus Asthma Back Pain 24,900,000 9,140,000 6,120,000 6,120,000 5,000,000 5,000,000 4,090,000 4,090,000 2,740,000 2,740,000 2,740,000 2,740,000 2,240,000 1,830,000 1,830,000 The CPC (Cost per Click) for the “top organic result” ranged from $2.25 to $7.01. In other words, if someone searched Google for “Depression” and the top result was WebMD and the searcher then clicked on WebMD and clicked a sponsored ad for the prescription drug Cymbalta, then the makers of Cymbalta would pay WebMD $7.01 for connecting the searcher to the ad. A) As a group, choose a health topic from the list above. Do a Google search for treatment options for that topic and find examples of the following: 1. A commercial site. 2. An educational or non-­‐profit site (with or without sponsored links). 3. A site that provides information from a medical “expert.” 4. A site that provides information from an amateur, such as an amateur blogger (Google: blogger and disease name) or go directly to a Q&A site such as Answers.Yahoo.com, WikiHow, Quora and search for treatment options for the health issue. 2 http://www.pharmaforward.com/ 5 B) Choose two of the sites you found and answer the following questions about each. 1. Did the site come up near the top of a Google (or other search engine) search? If yes, why? 2. What is the purpose of the site? 3. Who owns the site? Or who sponsors the site? 4. Does the site provide reliable information? How do you know? And how do you define reliable? 5. Does the site provide information from an expert? How would you define an expert? (For example, if you wanted emotional support for a disease, would a blogger that had the disease be an “expert?”) C) Present Findings and Assess. Join with another group and take turns with each group presenting their findings from questions 1-­‐4 above about the most reliable or least reliable site they found. (Or groups will be asked to report and assess findings with entire class.) D) Start with a SOURCE. Go back to your original group. Search for the same health topic by starting with a source rather than Google. Use a library database or discovery system or search MedlinePlus (http://www.nlm.nih.gov/medlineplus/) or Healthfinder (http://www.healthfinder.gov) or a LibGuide if available from your library website. Choose one article that looks promising and reliable and answer the following questions. 1. What kinds of information are available on these sites referenced above? Who writes the information? 2. Can you trust this information? Is it “scholarly”? How do you know whether it is or is not “scholarly”? 3. Every piece of information has an agenda. What is the agenda for this information? ( i.e. what is its’ purpose?). Extended Work for Classroom Discussion on Scholarly Research and Open Access 6 1. Look up the following article on Google Scholar (or Web of Science or Scopus): Pathophysiology and treatment of type 2 diabetes: perspectives on the past, present, and future 2. When you look it up it says: “Cited by” followed by a number. What is that number? What does it mean? 3. Click on the HTML of the article on the far right of the screen (if in Google Scholar, or just link to full text). Then select the second tab (Citations). What are all these citations? How are they connected to the original article? 4. This article is available “Open Access.” What does that mean? Should all articles be available open access? 7 Learning Activity 2: Understanding & Evaluating Science Information NOTE: This learning activity is in two parts. The Advanced Discussion exercise on page 7 is optional and can be used if students are advanced and if there is sufficient time.
Ask students to read Chapter 5 before the session if possible. •
•
•
•
•
Page 2: Preparation information Page 3 & 4: Instructor introduction with PPT and brief class discussion Page 5: Group Work/Handout Page 6: Teacher-­‐led classroom discussion about group work Page 7: Advanced discussion Links to the articles used for the group exercise can be embedded in a Learning Management System or a Google Doc if students have access to computers, or this exercise can be adapted for classes with no computer access by providing paper copies of the sources to be evaluated. Three Frameworks are addressed in these exercises:3 1. Authority Is Constructed and Contextual. Information resources reflect their creators’ expertise and credibility, and are evaluated based on the information need and the context in which the information will be used. Authority is constructed in that various communities may recognize different types of authority. It is contextual in that the information need may help to determine the level of authority required. 2. Information Has Value. Information possesses several dimensions of value, including as a commodity, as a means of education, as a means to influence, and as a means of negotiating and understanding the world. Legal and socioeconomic interests influence information production and dissemination. 3. Research as Inquiry. Research is iterative and depends upon asking increasingly complex or new questions whose answers in turn develop additional questions or lines of inquiry in any field. Images below can be enlarged and placed in a PPT. 3 Framework for Information Literacy for Higher Education. Association of College & Research Libraries. Available: http://www.ala.org/acrl/standards/ilframework Note: Qualitative assessment of the frameworks listed above can be done through observation and discussion. The thresholds represent a continuum of understanding and the exercises are designed to move students along this continuum. 8 Preparation: (PPT slide images are in reduced size on the following page.) 1. Put the two meteorology articles from The Onion (first one without and second one with source information) on two PPT slides to project. 2. Prepare another PPT slide that lists the three heuristics discussed by the instructor next to the Adam Sandler image. 3. Choose a science (or social science) topic for the class to investigate. Don’t choose something too complex like climate change. The example used for this exercise is whether dogs appear to have some rudimentary form of empathy. Choose three articles that connect to this topic or use the links provided below. a. One article from a blog. b. One article from a newspaper or magazine such as the New York Times or TIME. c. One article from a scholarly journal. a. Blog post: New Study Links Dog Yawning to Empathy. Canidae.com. http://www.canidae.com/blog/2014/01/new-­‐study-­‐links-­‐dog-­‐yawning-­‐to-­‐
empathy.html b: Magazine article: Stanley Coren, “Canine Empathy: Your Dog Really Does Care If You Are Unhappy,” Psychology Today, June 7, 2012, http://www.psychologytoday.com/blog/canine-­‐
corner/201206/canine-­‐empathy-­‐your-­‐dog-­‐really-­‐does-­‐care-­‐if-­‐you-­‐are-­‐
unhappy c. Scholarly journal article: Alex J. Bartholomew and Elizabeth T. Cirulli, “Individual Variation in Contagious Yawning Susceptibility is Highly Stable and Largely Unexplained by Empathy or Other Known Factors,” PLOS One (March 14, 2014), doi: 10.1371/journal.pone.0091773. 9 Introduction (Instructor led) (slides below can be copied and enlarged onto a PPT) One of the biggest challenges we face when evaluating information we find online is “disintermediation.” This is when the source or publisher or author is separated from the information itself because someone else is rebroadcasting the information. Tracking down the source of a piece of information, or paying attention to source information if it is available, is essential for being able to evaluate a piece of information. [show first ppt slide:] Ask: What is the difference between this piece of information…[give students one minute to glance at it] [show second ppt slide] Ask: And this piece of information? [After students mention that the second slide indicates the article if from “The Onion” say:] Yes, granted we were probably a little suspicious of this article to begin with based on its unlikely headline, but quickly glancing at the source tells us this is a parody news cite: that most of what we find at “The Onion” is trying to be funny and is unlikely to be true. [Show third ppt slide—Adam Sandler image next to it the following list of phrases: Reputation Heuristic (shortcut), Bandwagon Heuristic (shortcut), “Google Effect”] If we can locate the source of a piece of information we can use what is known as the “reputation heuristic” as a shortcut – telling us the piece of information is likely true, or in the case of The Onion article, not true based on the reputation of the site or publisher. 10 Though we can only use this if we have some background knowledge about the source. So if we see that the source is National Geographic or The New York Times we can use this shortcut to judge the information as more likely to be reliable because these sites use fact checking, editors and so on, but it’s not 100% effective, and sometimes we have to dig a little deeper and we have to actually think more about the information. This site looks like CNN, and it appeared in a Facebook feed looking very reliable, but if you click on it, it doesn’t go to CNN. We have to be very careful when we use the reputation heuristic as a shortcut -­‐-­‐-­‐sometimes we have to go a little further and corroborate information or check more deeply for source information. There are other shortcuts we use, such as the ”bandwagon” heuristic. This shortcut is a little dangerous. It means that if people rate something highly, give it 5 stars, recommend it etc. that we are more likely to believe that the information is valuable or true. Social endorsement is extremely powerful in influencing people’s decisions about whether to trust information. In fact, research has shown that people place more value on social endorsements even when their own first-­‐hand information runs counter to what they are reading! The last type of shortcut I want to talk about is the “Google Effect.” Research has shown that many of us assume that the first few hits turned up in a Google search are the most reliable and credible pieces of information on a topic. What makes this tricky is often Google does turn up incredibly reliable information –typically about factual information—What is the boiling point of water? When was Michelle Obama born? But Google ranks pages based on hundreds of data points and though they aim for some quality and reliability, there are so many factors that feed the algorithm, so much information to plow through, and so many ways to game the system in order to profit, that Google’s ability to surface the most reliable information is uneven at best. Google has also been accused of ranking information that they have a financial stake in at the top of their results list as well. Today we are going to put under the microscope three pieces of information. Two came up in the first 5 hits of a Google search, and one piece of information was buried under layers of information, the researcher had to choose a specific database to look in to find this piece of information. Let’s break up into groups of two or three people each. Each group will be given one piece of information to evaluate whether the information is reliable or not by investigating the questions in the handout. [Several groups will receive the same piece of information –
either online or a print handout] 11 Student Handout: Understanding and Evaluating Science Information Start by looking closely at the piece of information you have been given (online or in print). Skim the article and get a feel for the tone, the conclusions, the author(s), and the site on which the article appeared. 1. What kind of information is this? (e.g. a scholarly journal article? A news article? An opinion blog or article? Something else?) 2. Is the author an expert? If yes, what kind of expertise does this person have? Can you tell what their credentials are? Are their credentials relevant to the topic being written about? Do they have a “track record” writing about this topic? 3. Can you conclude from reading this article that dogs have the ability to be empathic? 4. What is the “backstory” for this piece of information? Why does it exist and what is its purpose? Could it be biased? 5. Did this article cite its sources? Was it likely to have been reviewed or peer reviewed or have an editor involved in the process of putting this information online? 12 Discussion – Teacher-­‐led Let’s hear about what you discovered about your piece of information. Group 1 can you tell us what your answers were to the 5 questions? (Then two other groups report so that the 3 different pieces of information are covered.) Notes on what to reinforce when hearing student answers: 1. What kind of information is this? (e.g. a scholarly journal article? A news article? An opinion blog or article? Something else?) Good opportunity to talk about the differences between popular and scholarly information. Also point out that some blogs can be really reliable and almost scholarly in nature (such as when they are part of academic sites, cite sources, etc.) and other blogs can be opinion pieces or marketing copy generated to drive commercial traffic to a website. 2. Is the author an expert? If yes, what kind of expertise does this person have? Can you tell what their credentials are? These conversations are tricky. The author of the Psychology Today piece has a Ph.D. and he has written some scholarly articles on various topics, but most of his writing on dogs is light and fun and in the style of a popular journalist. He draws conclusions in his article that are not really supported, and is guilty of “single study syndrome”—presenting one study as if it is the final word about a science topic and as if it were “true.” (More on this can be discussed in #3 and read about in Chapter 5) 3. Can you conclude from reading this article that dogs have the ability to be empathic? Be careful – the answer to #3 is always “no” –science only tells us the current state of research on a particular topic. Research questions are rarely 100% settled. So far there have been dozens of scientific studies on dogs and empathy and so far most scientists conclude that we don’t really have significant repeatable findings that prove it to be true. That doesn’t mean it’s not true, but it means we don’t know and have not been able to prove it with repeated experiments. 4. What is the “backstory” for this piece of information? Why does it exist and what is its purpose? Could it be biased? It’s important to understand that every piece of information has a backstory. Even if the backstory is that it was created by a group of researchers to investigate a hypothesis. 5. Did this article cite its sources? Was it likely to have been reviewed or peer reviewed or have an editor involved in the process of putting this information online? See Framework 3 and Chapter 5 to aid discussion. 13 Advanced Discussion: Ask students how they would go about finding an article such as the scholarly article on dogs and empathy and ask them how they would find out if additional research exists relating directly to this piece of research. (This discussion should only take place if students have some research experience and if they have read chapter 5 of “Finding Reliable Information Online.” It provides an opportunity to discuss science databases, Google Scholar, and citation tracing through Scopus, Web of Science, or Google Scholar and the scholarly research process generally. Show the students the library website, databases, libguides etc.) 14 Learning Activity 3: Understanding & Evaluating Social Science Information NOTE: Students should be assigned chapter 3 before this one-­‐hour session and need to bring their books (or have access to the ebook online). This project involves doing research online. Students will need to have different research topics in the social sciences assigned to them beforehand or will be able to choose from a list that the librarian or faculty member provides or from the link below. The students can use these topics during class time, or they could go on to use the sources they find in order to complete a research paper or project. Question 7 on the handout is tricky: Why can you be sure that if you search regular Google most of these scholarly articles would not come up in the first 5 search results? The answer is that Google rarely provides people with scholarly resources at the top of the Google search results – in part because most scholarly articles cost money and this pushes them down in search results. (Good opportunity to discuss information privilege and open access issues)
Three Frameworks are addressed in these exercises:4 3. Research as Inquiry. Research is iterative and depends upon asking increasingly complex or new questions whose answers in turn develop additional questions or lines of inquiry in any field. 4. Scholarship as Conversation. Communities of scholars, researchers, or professionals engage in sustained discourse with new insights and discoveries occurring over time as a result of varied perspectives and interpretations. 5. Searching as Strategic Exploration. Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops. Use topic areas provided by the faculty member or already assigned to the students. If these are not available, devise your own or use subjects from the following site: http://www.library.illinois.edu/ugl/howdoi/topic.html 4 Framework for Information Literacy for Higher Education. Association of College & Research Libraries. Available: http://www.ala.org/acrl/standards/ilframework Note: Qualitative assessment of the frameworks listed above can be done through observation and discussion. The thresholds represent a continuum of understanding and the exercises are designed to move students along this continuum. 15 Fill in the following Research Log 1. From topic area to research question: Write a question about your topic area. Something you wonder about in relationship to the topic. E.g. if your topic was “empathy and young people” you might ask: “Do young people have more or less empathy than they did twenty years ago?” If you are struggling – go to Step 2 and then come back to step 1. 2. Poke Around by Google Searching. What comes up in autocomplete in Google? (pg. 59 in book). Is it helpful? List a few articles you found searching Google: Just author, site name/source, and title. 3. Choose one article from #2 and investigate it. Is it reliable? Is it written by an amateur or an expert? Explain. What is the purpose of the article? (i.e. what is the article’s “back story”—why was it created?) 16 4. Next, Start at the Source: select an appropriate library database to find scholarly information. List one article you find particularly interesting/relevant. (It is ok to shift your research question as you search: research is an iterative process.) 5. Is this article reliable? Put your research article that you found in #4 through the paces. What can you find out about the author(s)? What methodology was used in the research described? What type of journal published the findings? Does the article seem objective or is it biased? 6. Is it connected? Take the article you found above, skim the references at the back of the article and list the two most relevant ones. Now look to see if anyone cited your main article – if others have cited your original article pick one of these articles that is the most relevant for updating the research on your topic and list it. If no one cited your original article –why might that be? 17 7. Next, search Google Scholar on the research question. Did you find new articles or the same articles you found in the library database? Google Scholar and library databases provide access to the “Deep Web.” Why can you be sure that if you search regular Google most of these scholarly articles would not come up in the first 5 search results? (see pg. 136 for help) 18