When we select a simple random sample X1,…,Xn from a fixed population of
interest, what makes resulting statistical inference procedures appropriate is not the fact
that each individual measurement in the sample is likely to be representative of the
population characteristic, say mean or median, of interest. Rather it is through the
concept of sampling distributions of the relevant statistics that we should, "on the
4
average", obtain a set of sample observations that are truly representative of the entire
population. However, in practice we obtain only a single random sample and the "on
the average" concept does not help much if the particular population items selected for
our sample are, in fact, not really very representative of the entire population. We are
simply bound by the statistical inferences for this particular sample that go with the "on
the average" concept unless we are willing to increase our sample size and expand the
number of sample observations.