Qfyilyi

Why does non-parametric bootstrap not return the same sample over and over again?

My notes write:

Assume data $X_1,...,X_n$.

Sample data with replacement to produce $X_1^{(p)},...,X_n^{(p)}$

Now since both are length $n$, then how does this not produce always the same sample? I'm missing something.

Each member of the bootstrap sample is selected randomly with replacement from the data set. If we were to sample without replacement, then every sample would simply be a re-ordering of the same data. But, as a consequence of replacement, the bootstrap samples differ in how many times they include each data point (which may be once, multiple times, or not at all). On average, ~63% of data points appear at least once in a given bootstrap sample.

@user20160's explanation is fine. Here's an example of 10 bootstrap samples of the sequence from 1 to 5, showing that some values will be represented more than once and other values will not be represented (x <- 1:5; t(replicate(10,sort(sample(x,replace=TRUE)))))

      [,1] [,2] [,3] [,4] [,5]

 [1,]    2    2    4    4    5

 [2,]    1    1    1    2    4

 [3,]    3    3    3    5    5

 [4,]    1    1    1    2    3

 [5,]    1    1    2    3    3

 [6,]    1    2    3    4    4

 [7,]    2    2    3    4    5

 [8,]    3    3    3    4    4

 [9,]    1    1    2    3    5

[10,]    1    1    2    4    4