We are inundated with evidence

Welcome new subscribers! If you like what you read, please consider forwarding this newsletter to someone else you think would enjoy it. And if someone forwarded this newsletter to you please consider subscribing!

When I did research in high school I had access to the school library’s print collection and… that was pretty much it. If it wasn’t in the encyclopedia or in one of the few thousand books in the collection, I wasn’t going to find it. My challenge was that I had too little access to information; that is not the challenge we face anymore. This is, in many ways, a good thing. I can find resources and experts and perspectives I never would have been able to find before. The downside is that I am inundated with so many resources, experts (and “experts”) and perspectives it’s difficult to make sense of it all.

There is evidence for everything

The abundance of available evidence can make us feel like we’re able to understand events and trends that are far outside our areas of expertise. It has become something of a meme: as soon as a disaster happens amateurs start analyzing images and information and making claims about what the evidence “proves.” The latest example is the collapse of the Francis Scott Key Bridge in Baltimore. Within hours, and before we had all the information about what happened, people were offering explanations about what had happened. Some of these explanations turned towards the conspiratorial, but many others were people trying to make sense of what had previously been unimaginable. 

Other folks, of course, use events like this as evidence for a larger claim about the way the world works. Boeing’s issues with safety have been used evidence that the company is prioritizing diversity efforts over safety. There is, of course, no connection between DEI programs and airplane safety, but the evidence about lack of regulation and prioritizing profits can be easy to cast aside when it’s just as easy to find “evidence” that fits your already established position. The adage that “everyone is entitled to their own opinions but not to their own facts” is put under serious strain when there are so many facts available. 

A lot of the evidence we encounter is also attached to unstated questions and open claims. Mike Caufield has a great post about how open claims work; we all engage in and encounter open claims all the time. I have a friend with whom I frequently exchange posts and articles about efforts at integrating AI going (predictably) awry. I don’t have to tell him why I’m sharing it or what claim I’m using it to support; we each know what the open claim is (roughly: the marketing and promotion of AI tools is way out ahead of what these tools actually can do or should be doing) and can fit the evidence in accordingly. Evidence is also used to support open claims that exist as part of the discourse; Taylor Swift and Travis Kelce’s relationship was offered as evidence that Democrats were trying to manipulate the voting public, which fits into larger claims about election fraud and interference. In cases like mass shootings, the same evidence can be used to support different open claims: that we need more gun control, that we need more armed citizens, that we need to focus on mental health, etc. The open claims persist, just waiting for evidence to match.

This abundance of evidence joins forces with confirmation bias to create evidence maximalism, which Caufield says has three rules: 

1. Any small thing can be evidence of my thing

2. Any big thing is always evidence of my big thing

3. All your evidence against my thing is, on closer inspection, very strong evidence for my thing

What the evidence is seems to be beside the point. We seek out evidence because we need a narrative to fit events into. There are lots of things that contribute to that instinct, but I do think there is something cyclical about it: we know more and more about the world; it’s hard to make sense of everything we know; we look for evidence to help explain what we’re seeing; that evidence confirms our beliefs and gives us a sense of certainty; we learn something else and need to make sense of it. 

Evidence needs context 

One of the challenges we face is that a lot of the evidence we encounter has been stripped of context or nuance. This happens in lots of places (articles that report what “studies say” without detailing the parameters of a study) but today I want to highlight ProCon.org in part because it’s easy to show what’s happening, and also because I know lots of teachers and librarians (myself included) who have used it with students when trying to understand the different sides of an argument.

I’m going to put aside the fact that pro vs. con is a false binary and look at an example of the evidence being shared on the site (I have written about my frustrations with ProCon previously, and am adapting some of that for this post). 

The page for Book Banning does a fairly good job representing the arguments that exist about allowing students to freely access books. These are some of the open claims in this larger argument, which is an important part of understanding a topic. There is definitely an opportunity for rhetorical analysis for some of the claims being made, but I want to focus specifically on the evidence being cited. 

These two arguments appear to be citing evidence from academic studies, which we tend to think of as being reliable. There is an appeal to authority happening here. But when we follow the footnotes to the sources we don’t find actual studies.

Okay. If we’re talking about opinions it’s not surprising that there’s some charged language in the headlines, and research involves tracing claims back to their original source. 

A link to a study… about pornography usage. Which is not what I would have assumed based on how ProCon summarized it. White, the author of the article, has used this evidence in support of her broader claim but both she and ProCon have stripped it of context. 

The article from footnote 17 does not link to any information from the AAP, and while I did not do an exhaustive search of AAP recommendations I was unable to find anything that supported restricting access to books. In fact, I found a report promoting reading as an alternative to more violent media.  

Is the influence of media consumption on children a complex question? Yes. Does the evidence ProCon lists help us answer that question? No. What’s being prioritized here is not the evidence, but the claim it’s supposed to be supporting; the reliability of the evidence is, at best, secondary. 

What do we do with all this evidence?

The ways that evidence can be used (or misused) to support a belief system is not typically covered under instruction on misinformation, in part because using evidence in this way is far more about truth than it is about fact. It’s hard to prioritize information when there is so much evidence that it can be hard to make sense of it all; we either accept it all because there’s too much to sort through, or shut down and reject all of it. Neither option is great. 

Our conversations are full of open arguments and unstated claims, and many times we know what those claims are; if we’ve been having an ongoing conversation with a colleague about an issue at work you don’t need to restate your claim every time. But if you’re encountering (or presenting) an argument with someone new or with a larger audience it’s worth it take the time to state the claim you’re providing evidence for. 

We also need to take the time to understand the open claims other people are providing evidence for. “Claim, evidence, reasoning” is a familiar framework for teaching students to support their conclusions and we could probably benefit from making more use of it outside the classroom. When you encounter evidence unattached from a claim it’s worth taking the time to see if you can determine what the unstated claim is; without that, it’s very difficult to evaluate the evidence or the reasoning. Knowing what the claim is also provides parameters for what kind of evidence is actually useful. 

When everything is evidence it’s even more important to understand what we’re talking about.

Thanks for reading! Want to work with me to help your community develop the skills they need to be savvy consumers and creators of information? Get in touch!