Different sampling schemes

In a random sample of size n from a finite population of N values, each population value has the same chance of being in the sample. Two different types of random sample are common in practice.

Both sampling methods can be performed by sequentially selecting values until the required sample size is reached. They differ in how the second and subsequent values are selected.

Sampling with replacement (SWR)
In SWR, the first selected value is returned to the population and the second value is randomly selected from all N population values. Values are selected in this way until there are n values in the sample.

A sample with replacement can contain the same population value more than once.


Sampling without replacement (SWOR)
In SWOR, the first selected value is removed from the population and the second value is randomly selected from the remaining N - 1 population values. Each selected value is removed and another selected from the remaining population values until there are n values in the sample.
 
Each possible subset of n values from the population has the same chance of being selected.

Since no value can appear more than once in the sample, SWOR covers more of the population and gives more accurate estimates than SWR.

However occasionally the sampled individuals cannot be removed from the population and SWR is necessary. An example would be a biologist who records characteristics of animals that are sighted within a region; there may be no way to tell whether a bird has already been spotted. (The statistical theory for analysing SWR is also easier than for SWOR, though this should not affect the sampling scheme that you use!)

Practical differences

If the sample size, n, is much smaller than the population size, N, there is little practical difference between SWR and SWOR — there would be little chance of the same individual being picked twice in SWR.

When the population is large (and considerably larger than the sample size), SWR and SWOR are almost identical.

In particular, if the population size is infinite, SWR and SWOR are identical.

Illustration

The distinction between sampling with and without replacement is shown in the diagram below. The values in the diagram are times (days) between a garage sending invoices for car repairs and receiving payment, for the 80 invoices that were delivered in one week.

Click the button Take sample to randomly select 5 of the 80 invoices with replacement. Take a few more samples and observe that it is possible to select the same invoice twice or more.

Use the pop-up menu to increase the sample size and select a few more random samples. The bigger the sample size, the greater the chance of selecting the same invoice two or more times when sampling with replacement.

Select the option Without replacement, then take a few more samples. Since we can no longer select any invoice more than once, the samples cover more of the population.