Simulation-Based Hypothesis Test for a Difference in Proportions – Fresh Take

  • Complete a randomization test involving a difference in proportions

Bootstrapping vs. Randomization Test

Both bootstrapping and randomization allow us to resample a data set and use it to generate new samples. Although bootstrap resampling is typically used to estimate confidence intervals, randomization resampling is typically used to test a hypothesis.

A bootstrap sample is a sample that is selected from the values in the original sample.
The bootstrap sample is selected with replacement, and the sample size is the same as the sample size of the original sample.
Bootstrap distribution is mainly used to estimate population parameters. 
Randomization procedures use resampling techniques to construct a sampling distribution that can be used to make inferences about the population.
Randomization is constructed given that the null hypothesis is true, and its distribution will be centered on the null hypothesis value. 
[1]Consider, for example, two groups of participants who have been randomly assigned to view a stimulus either monocularly or binocularly, and estimate its distance.

  • The bootstrap approach would focus primarily on estimating population differences in distance perception between the two conditions, and its standard error, and would probably result in a confidence interval on the mean or median difference in estimated distance.
  • A randomization test, on the other hand, would ask if it is likely that we would obtain a difference as large as the one we obtained if the monocular/binocular condition had no effect on the apparent distance.

Notice that the resampling approach is not concerned with what the estimated distances (or differences in mean distance) were, nor is it even particularly concerned about population parameters. The bootstrap approach, on the other hand, is primarily concerned with parameter estimation. It turns out that these differences have very important implications.

 

  1. https://www.uvm.edu/~statdhtx/StatPages/ResamplingWithR/ResamplingR.html#:~:text=Bootstrapping%20is%20primarily%20focused%20on,populations%20and%2For%20their%20parameters. ↵