• Examine the McCabe and Castel study
• Explain the replication crisis

Examining Psychological Research

Let’s practice our research skills by looking at an example of some research. First, read an excerpt from this article and think about the claims made by the research:

Watching TV is Related to Math Ability

Recently, researchers Ian McAtee and Leo Geraci at Harvard University did some research to examine if TV watching might have beneficial effects on cognition. The approach was fairly simple. Children between the ages of 12-14 were either asked to watch a television sitcom or do arithmetic problems, and while they were doing these activities, images of their brains were recorded using fMRI (functional magnetic resonance imaging). This technique measures the flow of blood to specific parts of the brain during performance, allowing scientists to create images of the areas that are activated during cognition.

Results revealed that similar areas of the parietal lobes were active during TV watching (the red area of the brain image on the top) and during arithmetic problem-solving (the red area of the brain image on the bottom). This area of the brain has been implicated in other research as being important for abstract thought, suggesting that both TV watching and arithmetic processing may have beneficial effects on cognition. “We were somewhat surprised that TV watching would activate brain areas involved in higher-order thought processes because TV watching is typically considered a passive activity,” said McAtee. Added Geraci, “The next step is to see what specific content on the TV show led to the pattern of activation that mimicked math performance, so we need to better understand that aspect of the data. We also need to compare TV watching to other types of cognitive skills, like reading comprehension and writing.”

Okay, let’s pause here. Would you say that the article above was well-written? Do you think the title, “Watching TV is Related to Math Ability” was a good description of the results? Do you think the scientific argument makes sense?

It would be pretty surprising to learn that watching television can improve your math ability…

But can it?

Surprise!

The article you just read was not an account of real research. Ian McAtee and Leo Geraci are not real people and the study discussed was never conducted. The article was written by psychologists David McCabe and Alan Castel for a study they published in 2008.^[1] They asked people to do exactly what you just did: read this article and two others and rate them.

McCabe and Castel wondered if people’s biases about science influence the way they judge the information they read. In other words, if what you are reading looks more scientific, do you assume it is better science? Did you notice the pictures of the brain next to the article that you just read? Do you think that picture had any influence on your evaluation of the scientific quality of the article? The brain pictures actually added no new information that was not already in the article itself other than showing you exactly where in the brain the relevant part of the parietal lobe is located. The red marks are in the same locations in both brain pictures, but we already knew that “Results revealed that similar areas in the parietal lobes were active during TV watching…and during arithmetic solving.”

McCabe & Castel Experiment

McCabe and Castel wrote three short, fake, scientific articles that appeared to be typical reports like those you might find in a textbook or news source, all with brain activity as part of the story. In addition to the one you read (“Watching TV is related to math ability “) others had these titles: “Meditation enhances creative thought” and “Playing video games benefits attention.”

All of the articles had flawed scientific reasoning. In the “Watching TV is Related to Math Ability” article that you read, the only “result” that is reported is that a particular brain area (a part of the parietal lobe) is active when a person is watching TV and when they are working on math. The highlighted part of the next sentence is where the article goes too far: “This area of the brain has been implicated in other research as being important for abstract thought, suggesting that both tv watching and arithmetic processing may have beneficial effects on cognition.”

The fact that the same area of the brain is active for two different activities does not “suggest” that either one is beneficial or that there is any interesting similarity in mental or brain activity between the processes. The final part of the article goes on and on about how this supposedly surprising finding is intriguing and deserves extensive exploration. This is a classic case of “correlation is not causation.”

McCabe and Castel hypothesized that seeing a brain image next to a poorly-written academic article would increase its legitimacy and make people think it made a more convincing argument.

---