Changes to the media over the past few decades have brought many distressing trends. Social media tend to reward emotional manipulation over truth-telling. News outlets, even well respected ones, rush to publish rumors and poorly substantiated claims. The fragmentation of news into ideological islands and search engines’ tendency to cater to our preferences have helped to polarize viewpoints.
But while we’re arguably awash in more misinformation than ever before, online media have also enabled tools and sources that help us evaluate dubious claims. I recently spoke at the Skeptic’s Toolbox, the annual skeptical workshop in Eugene, OR, about the variety of fact-checking tools available. It was less a lecture than a conversation, as the participants provided many of their own suggestions. While we can’t pretend to be comprehensive, our combined lists should provide a good starting point for your fact-checking needs.
What is “Fact Checking”?
![person at a protest holding a sign that reads '[citation needed]'](http://www.csicop.org//uploads/images/si/wilner-online-tools-1.jpg)
Photo by futureatlas.com on Flickr
There are a variety of ways to answer this question. One way to think of fact checking is in light of its role in forming and evaluating arguments. As skeptics, we spend a lot of time looking for logical fallacies. But it takes more than sound logic for a conclusion to be true: the premises of the argument must be true as well or the entire argument falls apart.
We can investigate the soundness of our premises by examining internal consistency, external plausibility, and source validity.
- Internal consistency: If you read a claim and there are points of disagreement within the claim, that’s a sign of something wrong.
- External plausibility: We should check the story against what we already know, and we may also do more research to check it out further.
- Source validity: We must identify the source of the claim, verify the source’s identity, and consider the source’s motives.
The Fact-Checking Toolbox
Another way to think about fact checking is in terms of the tools at our disposal (all of which will also fulfill one or more of the roles above). I tend to think of these as a rough continuum, starting with the easiest types of checks, moving to the more involved. These categories are:
- Internal validity checks
- Looking up “facts”: fake news sites, fact-checking sites, reference sites
- Media-based investigations: photos, videos, originality checks, social media users
Internal Validity Checks
This basically boils down to being a critical reader. As you read a claim or article, look for features that can give you clues as to its reliability, such as:
- Does it make sense? Is there internal disagreement?
- Date of publication: It’s very common for an article to make the rounds, and people to get riled up about something they think is happening now—and then it turns out that it happened years ago.
- Does the text justify the headline? You’d be surprised how often it doesn’t! In the report Lies, Damn Lies and Viral Content, Craig Silverman of Columbia’s Tow Center for Digital Journalism found large numbers of articles where a headline would treat a rumor as if it were true, but the article body would say the claim was unverified or even false. And as we know, many people don’t read beyond the headline.
- What’s the source? This can help us judge what’s in front of us, and can alert us if we need to dig deeper, for example if the claim originated from an earlier source.
This leads us to the easiest kind of source to evaluate: the fake news or satire sites.
Fake News and Satire Sites
Fake news sites have many aims. Some are cynical ploys to make as much money as possible off of “clickbait” headlines. Some purveyors of fake news seem to actually see themselves as doing a public service by exposing how gullible people can be. Some of the sites I list in this section are genuinely funny or satirical sites, like The Onion, that people have mistaken for real news; some claim the mantle of “satire” even though there’s really nothing humorous about them. (I’m looking at you, National Report, with your hilariously po-faced report that an entire Texas town had been quarantined due to Ebola.)
So Step 2 in your debunking process should always be to make sure the story you’re looking at isn’t simply fake. To help, here’s a list of fake and satirical news
sites:[1],[2],[3],[4]
- The Onion
- The Daily Currant
- The Daily Mash
- The Borowitz Report (The New Yorker)
- News Examiner (they run some true stuff, to fool you)
- National Report
- World News Daily Report
- Daily Buzz Live
- Huzlers
- Empire News, Empire Sports News
- Stuppid
- The News Nerd
- Newswatch28
- Newswatch33
- Naha Daily
- The Stately Harold
- NewsBuzzDaily
- Free Wood Post
- Betoota Advocate
- MediaMass
- United Media Publishing
- Super Official News
- The Beaverton
- The Landover Baptist Church
But remember that new fake sites pop up all the time. Don’t see the site you’re evaluating on this list? Here are some things to try:
- Look at the website’s “About” page. Sometimes these sites do disclose if they’re meant as satire.
- Google “fake news” and the publication name.
Finally, if a story mentions “Paul Horner,” run like the hills—he’s a fake news maven (founder of News Examiner, contributor to National Report) who likes to insert himself in stories.
Fact-Checking Sites
The next stage is to see if someone has already debunked the story or claim. If the debunking site you use is reliable, I see no shame in this—there’s plenty of bullshit out there for us all to investigate, without repeating someone’s else’s work.
Of course, that “if” is important. Even serious news sites that we generally trust can get a story dead wrong. I therefore limit my definition of “fact-checking sites” only to dedicated organizations whose raison d’etre is to thoroughly investigate claims, and who have a track record of doing so well.
Fact checking has really picked up steam as a movement within political reporting: There are now sixety-four sites around the world active in political fact checking, compared to forty-four a year ago.[5] Today in the United States, there are over two dozen fact checking organizations, though fact checking of national politics is dominated by the big three organizations: FactCheck.org, the Washington Post Fact Checker, and PolitiFact, which has affiliates in several states. These are great places to check on claims made by politicians.
But fact checking occurs in other realms too. Here are a few of my favorite fact-checking websites:
This site was launched about six months ago by FactCheck.org, which is based at the University of Pennsylvania. SciCheck focuses on addressing scientific claims that are made within the political arena. They can’t get to every such claim, of course, but if the claim you’re checking falls in this arena, SciCheck is certainly worth a look.
Fake photos are a big problem online. People will post a picture of one thing claiming that it depicts something else; or Photoshop a photo to depict something that never was; or circulate a picture without proper attribution. There are Twitter accounts devoted to sharing pictures without any regard for whether the images are actually real. Fake astronomy pictures (such as those ever-present giant moons) seem to be a popular category, and to counteract that we have the fine folks at @FakeAstropix, a Twitter account. Another good Twitter account, focusing on fake pictures of all sorts, is Pic Pedant.
I really like this site, run out the University of Minnesota, which tries to address the very real and pressing problem of terrible health reporting. Health News Review has a team of over forty reviewers, mostly medical professionals, who grade mainstream health news articles against ten criteria—things like: Does the article engage in disease-mongering? Does the article discuss the costs of the proposed intervention? Was the study in question done on mice rather than humans? They spell all this out for the reader and they give the article a rating.
Other fact-checking sites:
Snopes
Hoax Slayer
Antiviral (Gawker) - note that Antiviral posts from April 2014 and before are available here.
What Was Fake on the Internet This Week (Washington Post)
Sense About Science: For the Record
HoaxBusters
PicPedant
Emergent: Sadly, this site is no longer active, but since hoaxes tend to crop up again and again, there’s a good chance what you’re trying to debunk may be listed here.
The following sites do excellent fact checks, though it’s not all that they do; use their menus or search functions to see if they’ve fact checked the claim you’re interested in.[6]
Science-Based Medicine
Bad Astronomy
Bad Science
Reference Sites
What if no one has debunked the claim already? There’s lots of reference sites that can help you check out the claim against an existing body of knowledge. It’s good to have a list of them at your fingertips, which is why my fellow Skeptic’s Toolbox attendees and I brainstormed the items below.
But the scope of human knowledge is vast, and sometimes a reference that works great for one type of claim just doesn’t help on another. What are some good general principles for finding reference works?
My top tip is to be creative with your Googling. Rethink the aim of your search, and the various ways the desired information may be presented. For example:
- Look for the organizations that are connected with your topic. Let’s say I have a question within pediatrics. If I don’t know what organizations cover pediatrics, I can Google “pediatrics” or “pediatrics organization” to find that the American Academy of Pediatrics is a leading authority. If I want to Google search within their website, I append the search string “site:aap.org” to any search terms I want to use.
- Look for people. For example, you could combine the term or topic you’re interested in with “expert” or “professor” (always using critical reasoning to judge the qualifications of the people you find, of course!). If you’re trying to understand something complex, consider sending an expert a polite email to ask if you can talk by phone for a few minutes. Human conversation is invaluable because when you’re exploring a topic, it’s easy to form misconceptions—and not to know what it is that you don’t know. A real, live person can point out where you’ve gone astray. Lay out your preconceptions and let the expert correct you.
- Look for journals and journal articles, using Google Scholar. Even if you’re not prepared to dig deep into the journal articles themselves, oftentimes a scan of study titles can give you an idea of how a subject is addressed by experts, which can enable further searches. Also try searching for “review” or “meta-analysis” to find studies that synthesize large bodies of research.
One more note on finding good references: while Wikipedia should never be treated as a primary source, its footnotes are actually a great way to find works that you can rely upon.
Here are some reference sites to try, grouped loosely by category.[7]
Medicine:
Quackwatch has a wealth of information—including lists of further great resources. Check out their lists of “Comprehensive Medical Sites,” “Specialized Information Indexes” (click and scroll down a bit), “Interesting Web Sites.”
Cochrane Library (database of systematic reviews)
Science-Based Medicine (overviews and resource links for acupuncture, vaccines/autism and more)
Mayo Clinic (personal health information)
PubMed (medical studies)
Dietary Supplement Label Database - National Institutes of Health
Physician’s Desk Reference (prescription drug information)
Other science and math:
Skeptical Science (all about climate change)
Science Reference Services - Library of Congress (a list of reference guides, by subject)
Institute of Physics (search across a suite of pre-reviewed sites)
PLOS ONE (open access journals)
Dino Directory (identifying dinosaurs!)[8]
Other topics:
education.iseek.com (a knowledge-focused search engine that looks across thousands of pre-approved sources, including universities, governments, and non-profits, on any academic topic imaginable)
Journalist’s Resource (curated studies on a variety of topics)
Oxford English Dictionary (can be accessed from libraries)
OneLook (search across over 1,000 dictionaries)
Google Ngram (see how use of a phrase has changed over time)
CIA World Factbook (facts about countries)
Global Edge (statistical information about countries, and business stats by state, industry and more)[9]
National Security Archive (declassified U.S. government documents, hosted by George Washington University)
Sourcebook of Criminal Justice Statistics[10]
U.S. Census Bureau: Economic Indicators
Bureau of Labor Statistics: Labor Force Statistics
Give.org and Charity Navigator (info on charities)
Khan Academy (videos on math, science, art history, and more)
Internet Archive “Wayback Machine” (see caches of websites saved earlier)[11]
AMBER Alert Program (list of active alerts, from the National Center for Missing & Exploited Children)[12]
Photos
What happens when can’t check a claim against a database of facts, but have to compare the claim against reality yourself? This is often the case when it comes to photos, videos, and claims made on social media.
When you want to check out the veracity of a photo, there are a few key questions to ask:
- Does this image show what it purports to show? (One of the most common types of photo hoaxes is simple misattribution: a photo of Japanese tsunami victims, say, reused to supposedly show earthquake survivors in Nepal.)
- Has this photo appeared anywhere else online?
Further questions that might help to ask are:
- When was this photo taken?
- What type of camera was used?
- Was the photo altered?
As with text, you can save yourself grief by first going to fact-checking sites (see above) to see if someone has already debunked the photo.
After that, your next step should be a reverse image search to see where else the image has appeared. To use these sites, you can upload an image or enter its URL. (The URL should be for the image itself, not a website containing the image.)
Key tools for this are:
Next, try one of several tools that will check the image’s EXIF data to tell you the date and time the photo was taken and modified. These tools can also tell you what kind of camera was used, camera settings employed, and sometimes GPS location. These tools can, however, have trouble with social media sites, which strip out a lot of metadata.
Key tools:
Then you can do more detailed work using tools that examine the photo file itself, to determine when it was taken and look for alterations. Some of these tools have a bit of a learning curve.
The tools include:
Videos
There are a few tools to help you with video. YouTube Data Viewer gives you the upload time of a video and can help find earlier versions of the same video, using a reverse image search. If you need to slow down or brighten a video to try and make out certain features within it, try Tracker and VLC Media Player.
Originality Checks
It’s often important to see if the text you’re looking at has been copied from another source. For example, Googling text from badly written science articles can often turn up the original press release. Anti-plagiarism tools like Turnitin are also useful for this purpose. (You might need to sweet-talk an academic friend in order to access this tool.)
Verifying People
The logo for Bot or Not
If you’re looking at a post on social media, you may need to ask about the poster:
- Are they who they say they are?
- Are they near the action they’re describing?
- Are they even human—maybe they’re a bot? According to one study, about 13 percent of Twitter users are bots.[13]
Some of these checks don’t even require wheeling out special tools. For example, if you’re looking at a social media account of a supposed public figure, does the account have a blue “verified” tag? (If so, hover over the check mark with your mouse to see if the text “verified account” pops up.) When was it created? (Very recently created accounts are a good sign of a hoax.)
Some of the tools you can use are:
- LinkedIn Advanced Search
- Twitter Advanced Search
- Topsy - This is a good tool for seeking tweets from a specific time period.
- WebMii - This tool lets you search by name to find social media profiles, photos, video, and more. It has a “web visibility” score that can help identify fake profiles.
- Bot or Not? - This initiative of the Truthy Project at Indiana University evaluates the likelihood that a given Twitter user is actually just a bunch of code.
- Google Maps and Google Earth - These tools can help you figure out if a photo or video was shot where its promoters claim.
- Wolfram Alpha - This knowledge engine has lots of uses. Among them, it can tell you what the weather was like in a certain location on a certain day—again, useful for debunking photos and video.
- WhoIs - Lets you look up who owns a website, and when it was last registered. Note that you want https://who.is, not the similarly named whois.com or whois.net.
Finally, sometimes you may even want to reach out to someone in real life, but can’t find contact details. Or you have a number, but need their name.
These services will help you do that:
Further Resources
If you’re hungry for more fact-checking help, here are some resources you might find useful:
The Verification Handbook - This resource, edited by Craig Silverman, is designed for journalists. But don’t let that stop you; it’s got loads of handy tips, case studies of fact-checking tools at work, and deeper explanations of some of the tools described above.
Skeptools - A blog by Tim Farley, with excellent examples of how to use fact-checking tools.
Verification Junkie - A listing of verification tools, maintained by journalism foundation executive Josh Stearns.
Fact-checking resources - A guide I wrote for the Fact-Checking Project of the American Press Institute. It goes into more detail on some of the social media tools above, and also gives resources for several hot-button political issues.
I hope you’ve found this overview helpful. To my mind, the project of cataloguing reliable online sources is an important one—but one that is just beginning. What are your favorite fact-checking tools and resources?
[1] http://factually.gizmodo.com/the-9-worst-fake-news-sites-1681729157
[2] http://antiviral.gawker.com/the-antiviral-guide-to-the-worst-hoaxers-and-liars-on-f-1676552746
[3] http://www.washingtonpost.com/news/the-intersect/wp/2015/07/10/what-was-fake-on-the-internet-this-week-beyonce-and-jay-zs-quest-to-buy-the-confederate-flag-and-pea-guacamole/
[4] Skeptics’ Toolbox participant suggestions
[5] http://reporterslab.org/snapshot-of-fact-checking-around-the-world-july-2015
[6] Skeptics’ Toolbox participant suggestions
[7] Thanks again to Skeptics’ Toolbox participants for their suggestions, incorporated here.
[8] http://www.ala.org/rusa/sections/mars/marspubs/marsbestfreewebsites/marsbestref2014
[9] http://www.ala.org/rusa/sections/mars/marspubs/marsbestfreewebsites/marsbestref2014
[10] http://www.ala.org/rusa/sections/mars/marspubs/marsbestfreewebsites/marsbestref2014
[11] Tim Farley has a nice overview of how this and similar tools can be used: https://skeptools.wordpress.com/2009/06/10/web-archive-skeptic-tool/
[12] https://sweetyetunrefined.wordpress.com/2015/07/09/science-thursday-fact-checking-and-sharing-on-social-media/
[13] Chu, Z., Gianvecchio, S., Wang, H., and Jajodia, S. (2010). Who is tweeting on Twitter: Human, bot or cyborg? Proceedings of the 26th Annual Computer Security Applications Conference (pp. 21-30). Austin, Texas.