Teaching Popular Source Evaluation in an Era
of Fake News, Post-Truth, & Confirmation Bias
Lane Wilkinson, Director of Library Instruction, Univ. of Tenn. at Chattanooga
LOEX 2017, Lexington, KY, May 12, 2017
Definitions
Fake News:
Disinformation made to appear credible in order to maximize advertising revenue through viral sharing;
Fake News is the monetization of confirmation bias in the online environment.
Post-Truth:
Describes circumstances in which objective facts are less important in shaping public policy than appeals
to emotion or personal opinion; the “rhetoric” of fake news that encourages skepticism towards
traditional news sources; the attitude that traditional popular sources are more biased and
untrustworthy than they may actually be.
Confirmation bias: The tendency to search for, interpret, favor, and recall information that confirms a preexisting belief.
Directional reasoning: The tendency to select cognitive processes aimed at confirming a pre-established belief.
Accuracy-based reasoning: The tendency to select cognitive processes aimed at uncovering the truth.
Hostile Media Effect: The tendency to view mainstream media sources as biased against a pre-established opinion.
Strong priors: information evaluation is often shaped more by prior opinions than capacities to reason.
Dunning-Kruger Effect: The tendency to be over-confident in an opinion if less knowledgeable about the issue.
Availability Cascades: The heuristic of familiarity; we tend to ascribe more importance to a belief if it is easier to recall.
Defensive rationalization: The tendency to cling more strongly to a belief in the face of countervailing evidence.
Bayes’ Theorem (simplified): The likelihood of A, given that B reports it, is proportional to the belief that B will
accurately report A; our ability to ascribe reliability to a source’s reporting on an issue is inversely
proportional to the strength of our commitment to that issue
1
Post-truth Pedagogy
1. Timing matters


Put library instruction at the very beginning of the research process.
o Confirmation bias increases the further students get in the research process.
Library instruction should begin before the research question is selected.
. 2. Encourage accountability



Asking students to justify their search and evaluation strategies encourages accuracy-based motivation.
Students show more complex, integrative reasoning when asked to justify their information-seeking processes.
Peer-instruction encourages more complex reasoning about popular source evaluation.
3. Google is not the enemy




An adversarial “library vs. Google” approach encourages defensive rationalization.
Teach how Google works, not just how to evaluate websites.
Use Google “hacks” like the site: tag to encourage reliability considerations during initial search.
Flip the script: encourage consideration of reliability while they search, not just in response to individual sites.
4. Avoid controversial topics (at first)




Polarizing topics only encourage confirmation bias.
Work with faculty to diminish polarizing topics on initial assignments.
In class, start with the least controversial issues as examples.
Students cannot evaluate controversial social issues if they haven’t first learned to evaluate.
5. Rethink reliability




The reliability (credibility) of a source is a reflection of how likely that source is to lead to true beliefs.
The CRAAP test only tests for the usefulness of an article.
Source evaluation involves reliability (credibility) which can’t be addressed by CRAAP tests.
Make reliability an integral part of the search process.
6. Focus on the search process


Accountability for the search process is more important than accountability for the search results.
Make information evaluation part of the entire search process, not a response to individual search results.
Works Cited
Cassino, D., Woolley, P., & Jenkins, K. (2012). What you know depends on what you watch: Current events knowledge
across popular news sources. Fairleigh Dickinson University’s Public Mind Poll. Retrieved from
publicmind.fdu.edu/2012/confirmed/final.pdf
Druckman, J. N. (2012). The politics of motivation. Critical Review, 24(2), 199-216. doi:10.1080/08913811.2012.711022
Goldman, A. (1999). Knowledge in a social world. Oxford: Oxford University Press.
Gunther, A. C., & Liebhart, J. L. (2006). Broad reach or biased source? Decomposing the hostile media effect. Journal of
Communication, 56(3), 449-466. doi:10.1111/j.1460-2466.2006.00295.x
Gunther, A. C., & Schmitt, K. (2004). Mapping boundaries of the hostile media effect. Journal of Communication, 54(1),
55-70.
2
Habermas, J. (2006). Political communication in media society: Does democracy still enjoy an epistemic dimension? The
impact of normative theory on empirical research. Communication Theory, 16(4), 411-426. doi:10.1111/j.14682885.2006.00280.x
Habermas, J. (1989). The structural transformation of the public sphere : an inquiry into a category of bourgeois society
(T. Burger, Trans.). Cambridge, MA: MIT Press.
Jonas, E., Schulz-Hardt, S., Frey, D., & Thelen, N. (2001). Confirmation bias in sequential information search after
preliminary decisions: An expansion of dissonance theoretical research on selective exposure to information.
Journal of Personality and Social Psychology, 80(4), 557-571. doi:10.1037/0022-3514.80.4.557
Kaplan, J. T., Gimbel, S. I., & Harris, S. (2016). Neural correlates of maintaining one’s political beliefs in the face of
counterevidence. Scientific Reports, 6.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence
lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134.
doi:10.1037//0022-3514.77.6.1121
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108(3), 480-498.
Lenker, M. (2016). Motivated reasoning, political information, and information literacy education. portal: Libraries and
the Academy, 16(3), 511-528.
Taber, C. S., & Lodge, M. (2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political
Science, 50(3), 755-769. doi:10.1111/j.1540-5907.2006.00214.x
Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and Social Psychology: Attitudes
and Social Cognition, 45(1), 74-83. doi:http://dx.doi.org/10.1037/0022-3514.45.1.74
Tetlock, P. E., & Kim, J. I. (1987). Accountability and judgment processes in a personality prediction task. Journal of
Personality and Social Psychology: Attitudes and Social Cognition, 52(4), 700-709.
doi:http://dx.doi.org/10.1037/0022-3514.52.4.700
Other stuff that got cut for time
Kuran, T., & Sunstein, C. R. (1999). Availability cascades and risk regulation. Stanford Law Review, 683-768.
Lavine, H. G., Johnston, C. D., & Steenbergen, M. R. (2012). The ambivalent partisan: How critical loyalty promotes
democracy. New York: Oxford University Press.
Schaffner, B., & Luks, S. (2017). This is what Trump voters said when asked to compare his inauguration crowd with
Obama’s. The Washington Post. Retrieved from https://www.washingtonpost.com/news/monkeycage/wp/2017/01/25/we-asked-people-which-inauguration-crowd-was-bigger-heres-what-theysaid/?utm_term=.db79a652500f
Seeber, K. (2017). Wiretaps and CRAAP. Retrieved from http://kevinseeber.com/blog/wiretaps-and-craap/
Vraga, E. K., Tully, M., Akin, H., & Rojas, H. (2012). Modifying perceptions of hostility and credibility of news coverage of
an environmental controversy through media literacy. Journalism, 13(7), 942-959.
doi:doi:10.1177/1464884912455906
In-class activities
The following two activities are used to teach popular source evaluation for the ENGL 1010 (Rhetoric & Writing I)
program at the University of Tennessee at Chattanooga. The typical 50 minute section is as follows:
1. Teach Me How to Google activity
(10 minutes)
2. Information Needs, Types, and Qualities activity
(10 minutes)
3. Scholarly sources and multisubject databases activity (15 minutes)
4. Discussion and Time to Search
(15 minutes)
3
Teach Me How to Google
Learning Outcomes
Students will be able to limit Google results to known, reliable sources.
Students will be able to explain how Google relevance ranking works.
Students will be able to identify unreliable sources in their Google results.
Module Type
Group discussion; student-led demonstration; peer-discussion; reflection
Time
10-15 minutes
Outline
This activity proceeds via Socratic questioning. The goal is to have students demonstrate how they use Google to find
information and to provide them with search-engine concepts that foster critical thinking about popular search
methods. The end goal is to provide students with Google search methods that increase the likelihood they will
encounter reliable, instructor-approved popular sources.
1. Begin by asking students where they usually go for their information needs.
a. If they say “the library,” ask for further clarification: movie reviews through the library? Driving
directions through the library? Product reviews through the library? Campus/local news through the
library?
2. Establish that most searching takes place in Google, even “scholarly” research.
3. Ask students to share their research topics. Write a few on the board.
4. Select the least controversial topic (if needed, prompt for something less controversial) and invite all students to
Google it.
a. If topics tend towards controversial issues (e.g., death penalty, abortion, etc.), respond with something
like “Hold up, I don’t want to lost my job because y’all started a fight in the Library! What’s something
that’s not going to get us hating each other?!”
5. Key question: With an acceptable, non-controversial issue, ask all students to pretend they are writing a paper
on that topic and have them look it up in Google. (Search strategies will vary).
6. Ask who has a good result and is willing to share. Invite that student to either recreate the search using the
wireless keyboard on the projector, or ask for the search terms and recreate from the podium.
7. Results will almost always be a mix of Google Scholar, Wikipedia, news, ads, and rando websites.
a. The key is to anticipate. Know what kind of results a given topic will most likely result in and steer
students to select that topic.
8. Key question: Examine search results as a group: “Which one would you choose? What else is there? Why did
you choose that one?
a. OPTIONAL:
i. Ask students how many scrolled down the page before selecting a good result. Less than half
usually do.
4
ii.
iii.
iv.
v.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
Explain that 90%+ of Google searches (not searchers) do not scroll.
Ask how many went to Page Two of their results.
Explain that less than 1% of Google searches (not searchers) make it to Page Two.
From the podium, start clicking the highest page number on each page (e.g., 10, then 20, then
30, etc.) while asking how far we usually go. (“How desperate do you have to be to be on page
14 of a Google search?!”)
vi. Within 10-15 seconds, pages will stop being offered and you will reach the end of your results.
Usually between page 20 and 50. The number of results at the top of the page will change from
millions to a few hundred. (Blows their minds; causes hubbub and convo)
vii. Explain that Google can’t actually deliver millions of pages, and who could go through that many
anyway?
Ask students how Google decides to present the sources it does.
a. May have to tease out AdWords, etc.
Key Question: Can you tell just by looking which websites are going to be acceptable for Professor [Name]?
Key Question: How many of you want to spend all day going through and trying to find something you still have
to wonder about?
Ask if anyone knows of a way to make sure that Google only gives sources that Professor [Name] will accept.
Key Question: What sorts of sources are reliable enough for your Professor?
a. Check assignment ahead of time to see if suggestions for reliable popular sources are already given.
b. May need to tease out reliable sources (timesfreepress.com, nytimes.com, cnn.com, etc.)
c. Do not challenge students who bring up unreliable sources like FOX News or Breitbart.
Select a provided popular source like cnn.com (most common)
Recreate search with ‘site:.cnn.com’ tag (or whatever is provided); ask students to do the same.
Show that all results are now from cnn.com (or whatever) and that they don’t have to worry as much.
Distribute reflection worksheet.
Ask students to pair, Google each other’s topics, and discuss results. Have them write down thoughts on Google
search results for the other person’s topic and share back.
Finish with a brief acknowledgement that Google is a great tool…if you know how to manipulate it.
Reflection
This activity gets a great response from students and allows them to really open up about their daily search habits. The
site: tag trick (if used) almost always grabs their attention (instructors too) and really forces them to reconsider how
they Google.
5
Information: Needs, Types,
Qualities
Learning Outcomes
Students will be able to articulate the type of information they need to complete a given
task.
Students will be able to identify the appropriate uses for various information formats.
Students will be able to explain the criteria by which we identify the credibility of an information source.
Students will be able to identify which attributes of a given information source should be included in an annotated
bibliography.
Module Type
Group discussion; whiteboard
Time
10-15 minutes
Outline
This activity proceeds via Socratic questioning. The goal is to have students explain the common stumbling blocks they
encounter as they look for information online and as they write papers (if they have). The role of the librarian is to
facilitate the discussion by providing a contextual framework for student experiences. By showing students that
their research process follows a common pattern, they can make better choices about how, when, and where to
look for information (e.g., not jumping straight to peer-reviewed articles when they can barely define their topic)
1. Ask students for a handful of volunteer questions. Write on the whiteboard.
2. ASK: “Let’s say I was researching X and I didn’t know anything about it. Where would you have me go to learn a
little bit more?”
a. DESIRED ANSWERS: Wikipedia, google, a book, dictionary, etc.
b. Introduce students to concept of Background Information as an information need. This includes
definitions, important events, important people, related issues, etc. Write this need on the board.
3. ASK: “Okay, now I know more about what I’m interested in, but why should my readers care?”
a. DESIRED ANSWER: Because topic X is relevant to something going on right now
b. Remind students that a persuasive essay has to persuade and that they need a “hook” to make their
ideas resonate with their readers (ethos). They need Current Events. Write this need on the board.
4. ASK: “But isn’t this a small issue? Does it really affect everyone? How can we prove that this issue affects lots of
people?”
a. DESIRED ANSWER: Show how many people are affected.
b. Students have a need for Data/Statistics. Write this on the board.
5. ASK: “Okay, but people argue about this. Can’t I just grab the first opinions I sort of agree with and call it a day?”
a. DESIRED ANSWER: No. You need reliable and credible opinions.
b. Students have a need for Analysis. Write on the board
6
6. ASK: “Sweet. I’ve got the basics, the contemporary angle, some stats, and some really good opinions. But what
do I make of this article that says ‘A recent study has shown…’?”
a. DESIRED ANSWER: Find the study!
b. The final need is for the actual Research. Write on the board.
c. Reading through the research will often lead to new needs for background information and the cycle will
start over.
7. Note that not all information can help with all five of these needs.
8. Go through each of the five needs and ask what Information Types best meet each need. List these on the
board.
9. Note that sometimes a piece of information is crap. Introduce a metric like the CRAP test as a measure of
Information Quality.
10. See the final table below. The whiteboard should look something like this:
Information Needs
Information types
Background
Dictionaries/Encyclopedias
Wikipedia
Websites
Books
Newspapers/websites
Radio
Television
Social media
Government websites
Non-profits
Blogs
Magazines
Essay websites (e.g., Salon)
Scholarly books
Scholarly articles
Current events
Data/Statistics
Analysis/Opinion
Research
Information Quality
C – Currency
R – Relevance
A – Authority
P – Purpose
11. Show students that they now have the concepts they need to start annotating their articles. If they went
through a method to find their articles they can now say:
a. What type of information their source is (book, article, etc.)
b. What need the source fills in their paper/argument.
c. How relevant, current, authoritative the article is in the context of their assignment
d. How they intend to use the source.
7