- Recognize statistically unlikely outcomes.
Surprising Water Samples
The Flint water crisis is a public health crisis that started in 2014 after the drinking water for the city was contaminated with lead.[1] During that event, the city of Flint switched water sources, and the residents began to suspect that their water was contaminated with lead. The Michigan Department of Environmental Quality (DEQ) claimed that the water was compliant with federal regulations and that there was not a contamination problem. The federal guidelines for water safety state that a city is compliant if at least [latex]90%[/latex] of water samples obtained from residences have lead in the water under [latex]10%[/latex] parts per billion (ppb). A water sample is “contaminated” if it contains lead at [latex]15[/latex]ppb or above. In other words, under [latex]10%[/latex] of residences would need to be contaminated in order for the city to be compliant.
In the Flint water crisis context, the baseline assumption was that the DEQ was correct and the city of Flint was compliant with federal water safety regulations, so the proportion of residences with contaminated water was not over [latex]10\%[/latex]. It was the Flint Water Study (FWS)’s responsibility to provide evidence that the city was really not compliant and that there was actually a higher proportion of homes with contaminated water.
The question we want to ask here is, “If Flint was compliant with federal water safety regulations, was the FWS sample result surprising?”
An event is considered statistically unlikely if it has a low probability of occuring by chance alone. Researchers will set the threshold of what they consider unlikely, but a common standard is less than [latex]5%[/latex].