Statistical Thinking: Apply It

  • Explain how statistics help to examine data
  • Explain statistical significance and p-values
  • Explain reliability and validity
  • Understand how to read a research article
Coffee cup with heart shaped cream inside.
Figure 1. Coffee is wildly popular in many countries around the world. But can it make you live longer? [Image: Duncan, https://goo.gl/vbMyTm, CC BY-NC 2.0, https://goo.gl/l8UUGY]
Psychological studies often lead to interesting and intriguing discoveries about human behavior. Researchers are generally careful to jump to conclusions about their discoveries, but news outlets and popular media sources are less careful and they sometimes make claims that exaggerate the reach of the original study. One example of this is a 2012 study (Freedman, Park, Abnet, Hollenbeck, & Sinha) that found that men who drank at least six cups of coffee a day also had a 10% lower chance of dying (women’s chances were 15% lower) than those who drank none. Does this mean you should pick up or increase your own coffee habit?
Remember, correlation is not causation! What other things would you want to know about this research before insisting to everyone that they need to drink more coffee?  
Let’s think critically about the study together:

1. What questions should you ask about how the study was conducted?

2. What types of questions should you ask to examine the data?

3. What other things should you consider before increasing your personal coffee consumption based on this study?

The good news is that we can answer many of these questions about this study:

  • This was a 14-year study conducted by researchers at the National Cancer Institute.
  • The results were published in the June issue of the New England Journal of Medicine, a respected, peer-reviewed journal.
  • The study reviewed the coffee habits of more than 402,000 people ages 50 to 71 from six states and two metropolitan areas. Those with cancer, heart disease, and stroke were excluded at the start of the study. Coffee consumption was assessed once at the start of the study.
  • About 52,000 people died during the course of the study.
  • The sample sizes were fairly large and so the p-values are quite small (less than .05).
  • People who drank between two and five cups of coffee daily showed a lower risk of death, but the amount of reduction increased for those drinking six or more cups.
  • Whether coffee was caffeinated or decaffeinated did not appear to affect the results.
  • This was an observational study, so no cause-and-effect conclusions can be drawn between coffee drinking and increased longevity, contrary to the impression conveyed by many news headlines about this study. In particular, it’s possible that those with chronic diseases don’t tend to drink coffee.
While these results do show a negative correlation between coffee and death, the actual data from the study is not as optimistic. Look at these quotes from the research[1]:

4. What does this quote from the research mean?”In age-adjusted analyses, coffee consumption was associated with increased mortality among both men and women. However, after multivariate adjustment for potential confounders, particularly smoking, a modest inverse association between coffee drinking and total mortality was observed for both sexes.”

5. What does this quote from the research mean?

“After multivariate adjustment, coffee appeared to be inversely associated with most major causes of death in both men and women, including heart disease, respiratory disease, stroke, injuries and accidents, diabetes, and infections.”

6. What does this quote from the research mean?

“In contrast, there was no significant association between coffee consumption and deaths from cancer in women. There was a borderline positive association in men: among 13,402 deaths from cancer, 880 deaths were reported among men who drank 6 or more cups of coffee per day.”

7. What does this quote from the research mean?

“Although coffee consumption was inversely associated with diabetes, it was also positively associated with a number of behaviors that are considered unhealthy and are associated with an increased risk of death, such as tobacco smoking, consumption of red meat, and heavy alcohol use.”

This study needs to be reviewed in the larger context of similar studies, with the constant caution that this was not a randomized experiment. Whereas a statistical analysis can still “adjust” for other potential confounding variables, we are not yet convinced that researchers have identified them all or completely isolated why this decrease in death risk is evident. Researchers can now take the findings of this study and develop more focused studies that address new questions. In fact, additional studies support this original research, though a more recent 2022 study on coffee consumption and health lists 2-3 cups as the most beneficial daily consumption amount instead of 6 cups.

 


  1. Freedman, N. D., Park, Y., Abnet, C. C., Hollenbeck, A. R., & Sinha, R. (2012). Association of coffee drinking with total and cause-specific mortality. The New England Journal of Medicine, 366(20), 1891–1904. https://doi.org/10.1056/NEJMoa1112010